4 posts tagged with "Performance"

Most SAP extracts are narrow. You want three columns out of a table with a hundred million rows, on a schedule, and you want it to finish before the batch window closes. That shape is the normal case for an incremental load — and it was the one shape sap_read_table could not read in parallel at all.

partitions fixes that by splitting the scan across rows. On 2.9 million rows it is 4.5× faster at eight workers. Getting there meant taking apart the invariant the scan had been built on, which is the part worth writing about.

In June I wrote up erpl-rev — the little registered RFC server that lets ABAP call out into DuckDB — and it replicated 10 million rows of a 400-column BSEG-shaped table in about a minute. That post did not just claim a number, it explained why the number was what it was.

Since then I put it under a profiler to find out what was still on the table. Two things came back, pulling in opposite directions. A step that genuinely earned its place in June still had about a third of the ingest path left in it. And a constraint I had treated as fixed — SAP's proprietary RFC library — turned out to be removable altogether.

Picture the data team at a mid-sized manufacturer. Every night they need a fresh copy of the FI line items — BSEG and friends — in their lakehouse, where the analysts actually live: DuckDB, Parquet on object storage, the odd Iceberg table. The table is wide (hundreds of columns) and deep (tens of millions of rows), and the window is tight. The "official" answer is SLT: stand up a replication server, license it, configure the per-table settings in LTRS, babysit the queue. It works. It is also a lot of machinery — and budget — for what is, conceptually, "read this table fast and write it somewhere open."

I wanted to know how far the cheap path goes. SAP already exposes a perfectly good remote-function interface; DuckDB already ingests at memory bandwidth. What if ABAP just called out into DuckDB — no extension inside SAP, no SLT, no staging files — and we made the read fast enough to matter?

So I built it — two days, with Claude Code as the pair programmer. The result is erpl-rev, and on a throwaway SAP developer trial (single host, over loopback) it replicated 10,000,000 rows of a 400-column BSEG-shaped table in about a minute. It's an early release — a research prototype we're opening up, not an SLT replacement yet — but the numbers were enough to convince me the cheap path is real. Here is the story, the tools, and the numbers.

Summary​

This article compares ERPL's approach to SAP integration with traditional methods, highlighting performance improvements, ease of use, and cost benefits. We'll examine real-world scenarios and provide concrete examples of how ERPL transforms SAP data integration workflows.