Skip to main content

ERPL-IDoc Overview

Turn SAP IDoc files into SQL tables — and SQL back into byte-valid IDoc files.

erpl_idoc is a DuckDB extension for the SAP IDoc (Intermediate Document) format: the fixed-width flat files and the self-describing IDoc-XML that ALE/EDI interfaces exchange every day. Parse them, decode the opaque SDATA into typed columns, generate new IDocs from a query, and convert flat ⇄ XML — all in plain SQL, all offline.

It is the document/EDI layer of the erpl SAP family (erpl_rfc, erpl_odp, erpl_bics), and composes with erpl_rfc when you want live-SAP round trips.

erpl_idoc demo: read an IDoc file, decode SDATA into typed columns, write a byte-exact IDoc back, and convert flat to IDoc-XML — all in DuckDB SQL

✨ Highlights​

  • Read any IDoc file as a table — one SELECT over a flat IDoc or IDoc-XML file.
  • Typed decode — split the opaque 1000-char SDATA into named, typed columns via a segment dictionary. The control record (EDI_DC40, all 36 fields) reads typed too.
  • Write byte-valid IDocs from SQL — COPY (…) TO 'x.idoc' (FORMAT sap_idoc); the writer recomputes derived fields (SEGNUM, PSGNUM, HLEVEL, lengths).
  • Flat ⇄ IDoc-XML conversion — modernize a flat interface to XML or vice versa, losslessly.
  • Byte-exact round trips — read → write reproduces the input file bit-for-bit.
  • Offline & portable — the core needs no SAP and no network; works on a detached, air-gapped host. Linux, macOS, and Windows.
  • Framing & encoding — contiguous fixed-width or LF/CRLF ports (auto-detected); UTF-8 and latin-1 codepages; lenient mode for truncated files.
  • Composes with erpl_rfc — fetch the dictionary from a live system, or import a generated IDoc, using documented SQL — but erpl_idoc itself never speaks RFC.

🚀 Install​

INSTALL erpl_idoc FROM community;
LOAD erpl_idoc;

That's it — no SAP connection required to read, write, or convert IDoc files. New here? Start with the 5-minute Quick Start.

💡 Use cases​

  1. Land inbound IDocs in your warehouse. A partner drops ORDERS05/INVOIC02/MATMAS05 files on a landing zone. Query them straight into DuckDB for staging, validation, and analytics — no middleware.
  2. Decode & reconcile opaque payloads. Split SDATA into fields and reconcile values against the segment definition, or diff two IDocs field-by-field.
  3. Produce outbound IDocs from transformed data. Build IDoc content in SQL (joins, lookups, mappings) and emit a file a SAP file-port or inbound processing accepts.
  4. Migrate a flat-file interface to IDoc-XML (or back). Convert losslessly in one step — flat → xml → flat is byte-exact — so you can switch a port's format without touching the payload.
  5. Typed decode on an air-gapped host. Fetch the segment dictionary once from a connected system, persist it to Parquet, then decode IDocs on a detached host with no SAP and no erpl_rfc.
  6. Validate before you send. Catch structural problems early — bad offsets, overlaps, missing fields — and keep a byte-exact round trip as your safety net.

🔁 Round-trip guarantees​

  • Generic: sap_idoc_read_raw → COPY (FORMAT sap_idoc) reproduces the input file byte-for-byte.
  • Flat ⇄ XML: flat → xml → flat (and xml → flat) is byte-exact with the dictionary.
  • System-true: IDocs written/converted by erpl_idoc are accepted by SAP inbound processing (validated end-to-end against a live SAP AS ABAP system, not mocks).

🔌 Live SAP, cleanly separated​

erpl_idoc is a pure, offline file engine — it links no SAP libraries and makes no network calls. When you want a live round trip, you compose it with erpl_rfc in SQL:

  • Get a dictionary — sap_idoc_dictionary(sap_idoc_params('ORDERS05')) (see the dictionary guide).
  • Import a generated IDoc — hand the records to an erpl_rfc inbound call.

This keeps the file format portable and dependency-free while still giving you the full loop when a system is reachable.

🧭 Scope​

In scope: flat EDI_DC40/EDI_DD40, generic + typed read/write, the full control record, the segment dictionary (online/offline), both framings, multi-IDoc files, lenient/error handling, latin-1/UTF-8 decode, and IDoc-XML read/write + flat↔XML conversion.

Not (yet): EDIDS status-record side output, X12/EDIFACT conversion, and business-semantic validation (only structural validity is checked).

Next steps​