Row Partitioning: 4.5× Faster Narrow SAP Extracts
· 9 min read
Most SAP extracts are narrow. You want three columns out of a table with a hundred
million rows, on a schedule, and you want it to finish before the batch window closes.
That shape is the normal case for an incremental load — and it was the one shape
sap_read_table could not read in parallel at all.
partitions fixes that by splitting the scan across rows. On 2.9 million rows it is
4.5× faster at eight workers. Getting there meant taking apart the invariant the
scan had been built on, which is the part worth writing about.