2 posts tagged with "Parquet"

Every integration tool has two speeds: the one in the benchmark, and the one you actually experience on the first day. For erpl-rev those two numbers were embarrassingly far apart. It replicates 10 million BSEG-shaped rows in about a minute, and 2.5× faster than that since August. Getting to the point where you could run that benchmark took an afternoon.

Not because anything was hard. Because it was scattered. A type-T destination here. A function group there. Eight Z_DUCKDB_* function modules. Fourteen ABAP objects. A reginfo line in a file only Basis can touch. Four documents, each correct, none of them the whole path.

This post is the whole path, and it is now short enough to time with a stopwatch.

Summary​

• The article guides on integrating Excel with Parquet files using DuckDB, highlighting the efficiency of DuckDB for large data sets and how it surpasses Excel's normal data handling limits.
• It includes step-by-step instructions for installing the DuckDB ODBC driver, configuring Excel, setting up the ODBC connection, and suggests using Power Query Desktop for data transformation.