Skip to main content

APE Protocol Deep Dive

This guide covers CDS entity extraction in ERPL through SAP's ABAP Pipeline Engine. By the end you will know how to:

  • ✅ Find and describe the CDS entities a system exposes
  • ✅ Read one in full, or as a delta that includes deletes
  • ✅ Manage the subscriptions a delta read creates, and drop them cleanly
  • ✅ Recover an interrupted delta read without losing committed changes
  • ✅ Check, before you start, that the SAP user has what it needs
What is APE?

The ABAP Pipeline Engine (DHAPE_*) is a data-flow engine that ships inside the ABAP stack. It runs a small graph of operators — a reader, a channel, an outport — and hands the result back over RFC in packages. ERPL drives it with one operator: the CDS reader. Together with the ABAP Metadata Browser (DHAMB_*) for discovery, that is the whole surface. Nothing is installed in SAP — no transport, no Z objects, no ABAP.

Why a second path to CDS data​

ERPL already reads ABAP CDS through ODP, and for many entities that remains the right choice. Two things push the other way.

SAP Note 3255746 restricts third-party use of the ODP RFC API. erpl_ape exists because the pipeline engine is a different, supported interface for the same data. It reads CDS entities and nothing else — and that restriction is not a promise, it is enforced twice over: by a compiled-in allow-list of pipeline operators that no setting widens, and by the SAP role, whose S_DHAPEOPR object is checked per operator name. A role built as we document it cannot drive an ODP or SLT operator even if the software tried to. See the ZERPL_APE role.

Delta through ODP needs an annotation the entity may not have. ODP delta on a CDS view requires @Analytics.dataExtraction.delta.byElement — a design-time decision taken by whoever built the view, usually a change-timestamp element. That gives you inserts and updates. It does not give you deletes, because a deleted row has no timestamp to report. The pipeline engine uses trigger-based change capture (DHCDC) instead, so a delete arrives as a row marked D.

ODP (ABAP_CDS context)APE (pipeline engine)
InterfaceRODPS_REPL_*DHAPE_* / DHAMB_*
Delta prerequisitebyElement annotation on the viewnone; triggers are generated on first delta
Deletes in deltanoyes, as /1DH/OPERATION = 'D'
Server-side filtersyes, as ABAP range selectionsno — filter in SQL instead
Parallel package fetchyes, threads =>no, one package at a time
SAP footprintdelta queue entriesDHCDC logging tables and triggers, until the subscription is dropped

Neither replaces the other. ODP remains the better fit when the entity is already annotated, when you need server-side selections, or when you want parallel fetching. Reach for APE when you need deletes, or when the ODP RFC path is not available to you.

APE Architecture​

Like BICS and ODP, APE runs entirely over the SAP NetWeaver RFC SDK and reuses the RFC extension's sap_rfc secret. There is no separate endpoint to configure.

One structural point worth knowing early: DHAPE_GRAPH_ROUNDTRIP is the only call that moves data, and the engine commits each package as it hands it over. It cannot re-send one. That is why ERPL keeps a local copy in erpl_ape.delta_spill — see Recovering an interrupted read.

Connecting​

APE uses the same secret as every other ERPL protocol:

CREATE SECRET sap (
    TYPE sap_rfc,
    ASHOST 'sap.example.com',
    SYSNR  '00',
    CLIENT '100',
    USER   'ERPL_APE',
    PASSWD 'secret',
    LANG   'EN'
);

LOAD erpl loads erpl_ape alongside erpl_rfc, erpl_bics and erpl_odp, so there is nothing extra to install.

Before the first extraction, check that the user can actually do the job:

SELECT auth_object, checked_values, verdict, note
FROM sap_ape_check_authorizations('I_SALESORDER')
WHERE verdict <> 'ok';

An empty result means the role is complete. Anything else names what is missing, or — for one row — what is granted and should not be. See the role page.

Discovering entities​

sap_ape_show([search, released_only, max_depth, secret])​

Browses the metadata browser's CDS tree. With no arguments it walks from the root; search narrows it, which is much faster on a large system.

-- Find entities whose name contains SALESORDER
SELECT object_name, object_path, released
FROM sap_ape_show(search => 'SALESORDER');

-- Only entities with a release contract, to a bounded depth
SELECT * FROM sap_ape_show(released_only => true, max_depth => 4);

released matters: an entity without a release contract may change shape without notice, and ERPL refuses to extract from one unless you set erpl_ape_allow_unreleased = true.

sap_ape_describe(cds_name [, secret])​

Field structure — ABAP type, length, decimals, and the DuckDB type ERPL will produce.

SELECT field_name, abap_type, length, decimals, duckdb_type
FROM sap_ape_describe('I_SALESORDER');

The argument is an entity name or its browser path; a bare name is resolved by browsing.

sap_ape_preview(cds_name [, max_rows, secret])​

A bounded sample, read directly from the metadata browser rather than through the pipeline. That is the point: it needs no graph and no subscription, so it is the cheap way to look at an entity before committing to an extraction.

SELECT * FROM sap_ape_preview('I_SALESORDER', max_rows => 20);

Full extraction​

sap_ape_read_full(cds_name [, columns, filters, chunk_size, wireformat, recover, secret])​

-- Everything
SELECT * FROM sap_ape_read_full('I_SALESORDER');

-- A projection, pushed into the graph
SELECT * FROM sap_ape_read_full('I_SALESORDER',
    columns => ['SalesOrder', 'SoldToParty', 'TotalNetAmount']);

-- Smaller packages: more roundtrips, less memory per package
SELECT count(*) FROM sap_ape_read_full('I_SALESORDER', chunk_size => 10000);

A full read creates a subscription for the duration of the scan and erases it when the scan ends — including when you abandon it early with a LIMIT.

filters is refused, not ignored

The pipeline's CDS reader on the current release reads exactly seven configuration keys, and a filter is not among them. Passing filters therefore raises an error rather than being silently dropped: emitting a filter the engine does not read would return every row while your query looked filtered, which is worse than failing. Filter with an ordinary SQL WHERE — the scan streams, so DuckDB applies predicates as rows arrive.

Delta extraction​

sap_ape_read_delta(cds_name, subscriber_process [, columns, filters, chunk_size, wireformat, recover, secret])​

subscriber_process names the consumer. It is how SAP recognises the subscription on the next call, so the same value resumes where the last read stopped.

-- First call: initial load, and the subscription is created
SELECT count(*) FROM sap_ape_read_delta('I_SALESORDER', 'NIGHTLY_ETL');

-- Later calls: only what changed since
SELECT * FROM sap_ape_read_delta('I_SALESORDER', 'NIGHTLY_ETL');

Reading the change indicator​

Delta rows carry an extra column, /1DH/OPERATION:

ValueMeans
UThe row's current state — an insert or an update, both delivered as after-images
DThe row was deleted. Only the key fields are populated

Inserts and updates are not distinguished, because the engine sends after-images for both. That is the engine's semantics, not an ERPL simplification.

-- Apply a delta to a local table
BEGIN;
  CREATE TEMP TABLE chg AS
    SELECT * FROM sap_ape_read_delta('I_SALESORDER', 'NIGHTLY_ETL');

  DELETE FROM salesorder
   WHERE SalesOrder IN (SELECT SalesOrder FROM chg WHERE "/1DH/OPERATION" = 'D');

  DELETE FROM salesorder
   WHERE SalesOrder IN (SELECT SalesOrder FROM chg WHERE "/1DH/OPERATION" = 'U');
  INSERT INTO salesorder
    SELECT * EXCLUDE ("/1DH/OPERATION") FROM chg WHERE "/1DH/OPERATION" = 'U';
COMMIT;
The first delta read is slow, and that is the SAP side

A first delta on an entity makes SAP generate DHCDC logging tables and triggers on the underlying tables, in a background job. While that job is pending the engine returns neither data nor an error — the read simply waits. Batch work processes must be available or it appears to hang. erpl_ape_prepare_timeout (default 900 s) bounds the wait; the application log to check is object DHCDC, subobject ACP_JOB.

Subscriptions​

A delta subscription is durable server-side state: the triggers and logging tables persist until it is dropped. Treat it as something you own and clean up.

sap_ape_show_subscriptions([erpl_only, cds_name, secret])​

SELECT cds_name, subscriber_process, subscription_name, transfer_mode, status, created_at
FROM sap_ape_show_subscriptions();

-- Including subscriptions ERPL did not create
SELECT * FROM sap_ape_show_subscriptions(erpl_only => false);

Two name columns, and the difference matters. ERPL registers subscriptions with an ERPL_ prefix so the engine's inventory says which are ours: subscriber_process is the value you passed and the one that round-trips into sap_ape_drop, while subscription_name is what SAP stores and what a Basis administrator sees in DHCDC_MON.

PRAGMA sap_ape_drop(cds_name, subscriber_process)​

PRAGMA sap_ape_drop('I_SALESORDER', 'NIGHTLY_ETL');

Pass the unprefixed name — the one you used when reading. It reports rather than raises ('DROPPED' or 'NOT_FOUND'), because cleanup in SQL has no try/catch and a pipeline needs to be able to tidy up best-effort.

A client killed mid-scan

If a process dies while its graph is running, the graph stays running server-side and the engine then refuses to erase the subscription it holds. Recovery is the engine's own retention mechanism; there is nothing to do from the client but wait for it.

Recovering an interrupted read​

SAP commits each delta package as it hands it over, and cannot re-send it. So if your process dies mid-scan, those changes are gone as far as SAP is concerned — the pointer has moved. This is the one place APE needs help from the client.

ERPL writes every handed-over package to a local table, erpl_ape.delta_spill, before its rows are consumed, on its own transaction so a rollback in your transaction cannot discard it.

-- After a crash: replay what was handed over but never consumed
SELECT * FROM sap_ape_read_delta('I_SALESORDER', 'NIGHTLY_ETL', recover => true);

recover => true touches no SAP state at all — it reads the spill and nothing else, which means it works with SAP unreachable, and it is repeatable. It replays whole packages, so you may see rows you already consumed; the operation is at-least-once, not exactly-once.

The spill holds real business data

erpl_ape.delta_spill contains complete, unredacted rows for as long as the batch is the newest one, and anyone who can open the database file can read it. On a persistent database the last batch lives until the next delta read on that subscriber replaces it. Set erpl_ape_spill_enabled = false to opt out — at the cost of making delta extraction at-most-once.

Performance and good practice​

Use chunk_size to trade roundtrips against memory. It is rows per package. Smaller packages mean more calls and a smaller working set; the default suits most entities. Unlike ODP, APE fetches one package at a time — there is no threads parameter, because the engine hands packages over on a single session.

Project early. columns => [...] is pushed into the reader, so unwanted columns never cross the wire. A SQL-level projection does not help the transfer.

Prefer sap_ape_preview for exploration. It needs no graph and creates no subscription, so it costs the SAP system almost nothing compared with a full read you abandon.

Delta beats repeated full reads, but not for free. Weigh the ongoing cost of triggers and logging tables on the source tables against re-reading the entity. For a small, rarely changing entity a periodic full read is often the better engineering choice.

Don't leave subscriptions lying around. Every one you no longer read is trigger overhead on a production table. sap_ape_show_subscriptions() then sap_ape_drop is a good habit.

Troubleshooting​

Common Issues​

Entity not found

-- Search rather than guess; names are case-sensitive at the API
SELECT object_name, object_path FROM sap_ape_show(search => 'SALESORDER');

The read seems to hang

-- Usually SAP's preparation job. Bound the wait, then look in SAP.
SET erpl_ape_prepare_timeout = 1800;

Check that batch work processes are free, and read application log object DHCDC, subobject ACP_JOB. A first delta is much slower than a full load by design.

"already has a subscription" on the second call

-- The subscriber already exists. Resume it by using the same name...
SELECT * FROM sap_ape_read_delta('I_SALESORDER', 'NIGHTLY_ETL');

-- ...or discard it and accept a fresh initial load
PRAGMA sap_ape_drop('I_SALESORDER', 'NIGHTLY_ETL');

Permission denied, or nothing works at all

-- The whole requirement list, with a verdict per row
SELECT * FROM sap_ape_check_authorizations('I_SALESORDER');

If every row reports unknown, the connection or AUTHORITY_CHECK itself could not be reached — usually a missing S_RFC grant for function group SUSR.

An unreleased entity is refused

-- Deliberate: such an entity may change shape without notice
SET erpl_ape_allow_unreleased = true;   -- logged at WARN

Debugging Tips​

SET erpl_trace_enabled = true;
SET erpl_trace_level = 'DEBUG';
SET erpl_trace_output = 'console';

PRAGMA sap_ape_ping;          -- logs on AND asks the engine its version
SELECT * FROM sap_ape_system_info();

PRAGMA sap_ape_ping is deliberately more than a logon check: it also calls the engine, so it fails when the DHAPE_* function group is unreachable for this user rather than only when credentials are wrong.

For SAP Experts​

The wire-level mechanics for Basis administrators and ABAP developers — the DHAPE_* and DHAMB_* modules, the graph, the package format, and the change-capture semantics behind the SQL functions.

APE protocol deep dive — DHAPE/DHAMB modules, the graph, package format, DHCDC semantics, authorization call sites

The function modules​

ERPL functionRFC function module(s)
sap_ape_showDHAMB_SERVICE_DSET_BROWSE (once per folder level)
sap_ape_describeDHAMB_SERVICE_DSET_BROWSE (path resolution) → DHAMB_SERVICE_DSET_DEFINITION
sap_ape_previewDHAMB_SERVICE_DSET_PREVIEW
sap_ape_read_full / read_deltaDHAPE_GRAPH_MANAGER → DHAPE_GRAPH_ROUNDTRIP (repeatedly) → DHAPE_GRAPH_MANAGER
sap_ape_show_subscriptions, PRAGMA sap_ape_dropthe same two, driving an admin graph
PRAGMA sap_ape_pingSDK RfcPing() + DHAPE_GRAPH_VERSION
sap_ape_system_infoDHAMB_SERVICE_SYSTEM
sap_ape_check_authorizationsAUTHORITY_CHECK

The DHAMB_SERVICE_* modules are REST endpoints tunnelled over RFC: they take a path and return a JSON body with an HTTP status. The DHAPE_* ones are the engine proper.

The graph​

DHAPE_GRAPH_MANAGER takes a mode — Create, Run, Stop, or Open — and, for Create, a graph definition as JSON: a set of named processes, each with a Component naming its operator, and channels connecting their ports. Protocol v6 auto-starts the graph during Create, and the graph is bound to the RFC session.

ERPL builds one shape: a CDS reader (com.sap.abap.cds.reader.v2) feeding an outport (com.sap.abap.internal.outport, which the engine synthesises). The reader takes exactly seven configuration keys — subscriptionType, subscriptionID, subscriptionName, cdsname, action, chunkSize, wireformat. There is no filter key and no schema key, which is why filters is refused rather than emitted.

action selects the mode: Initial Load for a full read, Replication for delta. A third value, Delta Load, exists in the engine and is deliberately not exposed — it maps to a DHCDC process option that never prepares an initial load, so it would stall permanently.

The package format​

DHAPE_GRAPH_ROUNDTRIP returns packages as a JSON envelope with Encoding: "csv" and a self-describing Attributes.ABAP.Fields[] block, so each package carries its own schema. The body is RFC 4180 quoted — "comma,inside", "quote""inside" — and a backslash is data, not an escape. message.batchIndex counts packages and message.lastBatch marks the end.

A replication graph never sets lastBatch; a delta read ends when no further package arrives within erpl_ape_delta_quiet_seconds. Documented sentinel values (9999-99-99, NaN, ? and similar) map to NULL; any other value that will not cast raises an error naming the package, row, column and text, because a silent NULL is a wrong answer the caller cannot see.

Change capture​

Delta uses DHCDC: on first use SAP generates logging tables and database triggers on the tables underlying the entity, in a background job (DHCDC / ACP_JOB). They persist until the subscription is erased. The change column is /1DH/OPERATION, with U for insert and update after-images and D for a delete carrying only keys.

The engine commits each package on handover — ERPL calls commit_delta_data immediately after output_data, and there is no client acknowledgement in the protocol — so there is no engine-side recovery of an unconfirmed package. That is what the local spill is for.

Where authorizations are actually checked​

ObjectField(s)Checked by
S_DHAPEOPRDHAPEOPNM, ACTVTCL_DHAPE_AUTHORITY::check_operator, from CL_DHAPE_OPERATOR_REGISTRY, once per operator the graph names
S_DHAPEOPDHAPEOPIMPthe same method, as a fallback when the S_DHAPEOPR check fails
S_DHAPEAC2ACTVTCL_DHAPE_AUTH_ACTIVITY
S_DHAMBACTDHAMB_ACVTCL_DHAMB_AUTHORITY::check_activity
S_DHAMBCDSDHBASCDSNM, DHBASCDSRS, ACTVTCL_DHAMB_AUTH_CDS

Two consequences worth the attention of whoever builds the role. S_DHAPEOPR being keyed on the operator name is what makes the CDS-only restriction expressible in a role at all. And S_DHAPEOP must not be granted: it is the fallback consulted when the S_DHAPEOPR check fails, so holding it — from any role — authorises operators S_DHAPEOPR denies. The role page covers both.

Next Steps​

🚀 Ready for More?​

🔧 Advanced Topics​

💡 Examples​

Need help? Check our troubleshooting guide or browse more examples.