10 posts tagged with "DuckDB"

Picture the data team at a mid-sized manufacturer. Every night they need a fresh copy of the FI line items — BSEG and friends — in their lakehouse, where the analysts actually live: DuckDB, Parquet on object storage, the odd Iceberg table. The table is wide (hundreds of columns) and deep (tens of millions of rows), and the window is tight. The "official" answer is SLT: stand up a replication server, license it, configure the per-table settings in LTRS, babysit the queue. It works. It is also a lot of machinery — and budget — for what is, conceptually, "read this table fast and write it somewhere open."

I wanted to know how far the cheap path goes. SAP already exposes a perfectly good remote-function interface; DuckDB already ingests at memory bandwidth. What if ABAP just called out into DuckDB — no extension inside SAP, no SLT, no staging files — and we made the read fast enough to matter?

So I built it — two days, with Claude Code as the pair programmer. The result is erpl-rev, and on a throwaway SAP developer trial (single host, over loopback) it replicated 10,000,000 rows of a 400-column BSEG-shaped table in about a minute. It's an early release — a research prototype we're opening up, not an SLT replacement yet — but the numbers were enough to convince me the cheap path is real. Here is the story, the tools, and the numbers.

← Part 1: Your SAP Data Is Already Real-Time

Last week the forecasting team at our Mittelstand walked away with something they'd never had: a fresh, validated, DuckDB-native view of every order line and customer record they cared about, no warehouse landing in sight. Latency under five minutes, anomalies caught at the gate. Good ergonomics. Good numbers.

Then the AI engineer asked the question that always comes next.

"Great. How does the agent reach this?"

Picture a forecasting team at an industrial-controls Mittelstand. Their job for the next quarter: launch an AI-driven inventory and demand-forecasting tool that the operations team will actually trust enough to act on. They have the model. They have the GPUs. They cannot ship.

The reason: their nightly ETL job runs at 02:00 and finishes around 04:00, by which point the model trains on data that's already half a day old. By the time the forecast reaches the procurement system at 08:00, it's reasoning about a world that existed fourteen hours ago. When the CFO asks why the model recommended ordering more of an item that just sold out at lunchtime, nobody has a good answer — because the data the model saw never knew the lunchtime existed.

The fix isn't a better ETL. The fix is removing the ETL.

Summary​

This article compares ERPL's approach to SAP integration with traditional methods, highlighting performance improvements, ease of use, and cost benefits. We'll examine real-world scenarios and provide concrete examples of how ERPL transforms SAP data integration workflows.

Summary​

Learn how to build real-time SAP dashboards using ERPL and DuckDB. This comprehensive guide covers everything from data extraction to dashboard creation, including practical examples and best practices for monitoring your SAP systems in real-time.

Summary​

Discover how ERPL transforms SAP data integration for data science workflows. This comprehensive guide covers everything from data extraction to machine learning model deployment, showing how ERPL enables data scientists to work directly with SAP data using familiar tools and frameworks.

Summary​

DuckDB is a high-performance SQL OLAP database system known for its efficiency in handling analytical queries. However, accessing and analyzing public holiday data across different countries and date ranges can be challenging. By leveraging the ERPL-Web extension, you can seamlessly fetch and process this data within DuckDB.

We demonstrate how to retrieve public holiday data for Germany, showcasing the ease and power of this approach. This example highlights how DuckDB and ERPL-Web can streamline your data workflows by connecting your SQL environment with external data sources efficiently.

Summary​

This article demonstrates how to integrate SAP BW data in Microsoft Fabric using ERPL, providing a comprehensive guide for connecting enterprise SAP systems to Microsoft's modern data platform.

Summary​

• Airbyte is one of the most popular integration solutions. However it is lacking access to enterprise data sources from SAP. We show how with our ERPL Airbyte connector one can bridge this gap.

Summary​

• The article guides on integrating Excel with Parquet files using DuckDB, highlighting the efficiency of DuckDB for large data sets and how it surpasses Excel's normal data handling limits.
• It includes step-by-step instructions for installing the DuckDB ODBC driver, configuring Excel, setting up the ODBC connection, and suggests using Power Query Desktop for data transformation.