The SAP RFC Protocol, Byte by Byte

Every SAP integration built in the last two decades rests on one shared library: libsapnwrfc.so, SAP's closed-source NetWeaver RFC SDK. It is the thing that actually speaks to an ABAP system. It is also a black box — a binary you download, that pulls in ICU, that you cannot read, cannot audit, and cannot ship inside your own product without a redistribution dance.

We wanted erpl — our DuckDB extension for SAP — to not need it. So we wrote a pure-Rust replacement for the RFC protocol, saprfc: no SDK, no ICU, no post-install download, just Rust that opens a socket and talks. To write it, we first had to learn a protocol that has no public specification.

This post is what we learned: the SAP RFC wire format, byte by byte. You can read a real record below.

The claim, up front

There is a large academic literature on protocol reverse engineering — recovering the format of a closed protocol from observed traffic. The standard way to argue an inferred format is "correct" is to compare it against a Wireshark dissector and report a precision/recall number: the inference got 92% of the field boundaries right.

We are making a stronger claim, and it is worth stating precisely:

Every rule in our specification is derived from packet captures of SAP's own libsapnwrfc talking to a live ABAP system. And the implementation built from it is validated byte-for-byte — both by re-encoding captured records and diffing against the original, and by sending our bytes to a real SAP server and requiring it to answer exactly as it answered SAP's library.

That is not a statistical accuracy figure against a dissector. It is an executable, falsifiable specification with a differential oracle: SAP's library and a live server both get a vote, and a wrong byte fails the test. We'll come back to how that works at the end. First, the format.

How we watched

You cannot specify what you cannot observe, so the first question is: what could we see?

We recorded the byte stream at the socket boundary of SAP's own client, using an LD_PRELOAD shim that intercepts read/write/writev/sendmsg and writes each segment to disk before passing it through. It changes no addressing and alters no bytes; it just watches. Crucially it records a vectored writev as one segment under a single lock and aborts rather than truncate — the captures are the specification, so a torn record would be invisible corruption.

What we could not see: SAP's source, and SAP's disassembly. We deliberately did not read either. Where we needed a second opinion on behaviour we used open-rfc, an Apache-licensed TypeScript implementation, strictly as a black box — send a call, compare the bytes it produced — never as source to read while implementing. This is the same posture the Samba team took two decades ago, and for the same reason: behavioural observation of a network protocol is the clean, defensible way to build an interoperable implementation. More on that below.

The onion

Here is a real logon record — the very first thing an RFC client sends after the handshake — captured from libsapnwrfc, then decoded by our parser (the same code saprfc ships). Hover a byte to find the field it belongs to; hover a field to find its bytes. Every field carries the specification rule that defines it.

The colours show the structure is an onion. Four layers, outside in:

1. a 4-byte NI length,
2. an 80-byte APPC record header,
3. the RFC_PRO message — a run of tag/length/value fields,
4. and inside those fields, the actual values.

Those four records — connect, log on, ask for the interface, call — are the whole conversation. Here is the shape of it, and where the pieces above fit:

Let's peel the layers.

Layer 1 — NI framing

The outermost wrapper is trivially simple and easy to get subtly wrong. Every message is a 4-byte big-endian length followed by exactly that many bytes:

+--------+--------+--------+--------+----------------------------+
|         length (u32, big-endian)  |   payload (length bytes)   |
+--------+--------+--------+--------+----------------------------+

In our logon record the frame opens 00 00 01 74 — 372 — and 372 bytes follow, with nothing left over. That "nothing left over" is a property we test in both directions: the stream frames exactly, with zero residue. If your decoder ever has leftover bytes, it has already desynchronised and every later frame is garbage.

The very first frame of a connection is special: it is not APPC at all but a gateway routing handshake beginning 02 03, carrying the client's IP as raw octets, the program name, and the codepage as ASCII digits. The client offers 1100 (Latin-1); the server answers 4103 (UTF-16LE). That codepage answer matters later — it changes how every subsequent string is encoded.

Layer 2 — APPC records

Inside the frame is an 80-byte APPC/CPIC record header. Most of its 80 bytes we never needed to understand, but three positions carry their weight:

• byte 0 — a version, always 0x06;
• byte 1 — the function: Initialize, Data, and a handful of others;
• bytes 40–47 — an 8-character ASCII conversation ID.

The conversation ID is the one detail worth remembering. The client does not invent it — the server assigns it, in its reply to the client's Initialize, and the client echoes it on every record thereafter. In the record above it is 00952683. Get this wrong and the gateway simply stops talking to you.

When a reply is large, APPC splits it: application data is fragmented at 28,000 bytes, full fragments arriving as Data records and the remainder as a final DataLast. Treating that last record as unexpected truncates every reply bigger than one fragment — a bug that only shows up on real-world payloads, never on a ping.

Layer 3 — RFC_PRO fields

This is the heart of the protocol, and where the archaeology gets fun. The APPC data payload is an RFC_PRO message: a header, then a sequence of self-terminating fields.

The message opens with twelve bytes that, decoded as EBCDIC, spell RFC000000000:

d9 c6 c3 f0 f0 f0 f0 f0 f0 f0 f0 f0
R  F  C  0  0  0  0  0  0  0  0  0

d9 c6 c3 is RFC in code page 500 — a fossil from the protocol's mainframe ancestry, sitting in the middle of an otherwise ASCII/UTF-16 world. And it appears only on the first data record of a conversation; every later record begins directly with a field. So a parser must detect it, never require it — a rule we learned the hard way, because emitting that header on a continuation record makes the server reject the whole message with no diagnostic at all.

Each field is a tag/length/value triple whose tag is repeated after the value:

+--------+--------+--------+--------+= = = = = =+--------+--------+
|    tag (u16 BE)  |  length (u16 BE)|   value   |   tag (u16 BE)  |
+--------+--------+--------+--------+= = = = = =+--------+--------+
                                                 └ must equal the opening tag

That repeated trailing tag is a self-check. If it doesn't match the opening tag, the stream has desynchronised, and the right thing to do is fail hard — not try to resynchronise, because everything after it is already meaningless. The field sequence ends with a 0xffff tag of length zero, and then — this is the part that cost us the most time — an 8-byte trailer whose first four bytes restate the length of the RFC_PRO message. In our record the message is 284 bytes and the trailer begins 00 00 01 1c, which is 284.

Why does that matter? Because a client data record carries no length field of its own anywhere in its payload. You derive the message length from the NI frame, not from a header. Put a stale length in that trailer and the server reads past the end of your message and replies with an empty record — no error, no status, nothing. For a long time, an empty reply looked like rejection. It isn't. An empty reply means your message was malformed. Once we understood that, a whole class of silent failures became diagnosable.

One more field rule, because it bites hard. The length is a u16, so it cannot express a value of 65,535 bytes or more. Such a value declares 0xffff as its length and carries the real length as a u32 immediately after:

tag(u16)   0xffff(u16)   real length(u32)   value …   tag(u16)

Read naively as a plain 65,535-byte value, the closing tag lands in the middle of the data and the stream looks corrupt at an offset far from the real cause. We found this because a single ODP data package came back as a 99,857-byte XML blob — and it spanned four APPC fragments too, just to keep us honest.

Layer 4 — values, and a codepage that changes underfoot

The logon record above is all ASCII. But remember the handshake negotiated codepage 4103 — so in later records, textual values are UTF-16LE. Switch the explorer to the Metadata call tab and you can see it: Function = "RFC_GET_FUNCTION_INTERFACE", but in the bytes it is 52 00 46 00 43 00 … — every character followed by a null. Byte length is twice the character count.

A few values have their own quirks worth knowing:

• Language is a single character. The caller passes EN; the client maps it to SAP's one-character code E before sending. A two-letter language on the wire is a bug.
• The password is scrambled, seventeen bytes, and must never be logged. (In the explorer it is masked to zeros — the one field on this page whose real bytes we do not publish. Everything else is the genuine capture.)
• Packed decimals are BCD, two digits per byte with a sign nibble, and the scale comes from the field's dictionary metadata, not the wire — a detail that turns a correct-looking number into a wrong one if you skip it.

Making a call

With the format in hand, a function call is almost anticlimactic. The client sends the function name, then a list of the parameter names it wants back (RequestedOutput, repeated — the server returns only these), then the input ParameterName/ ParameterValue pairs. Tables get their own tags. The Call reply tab shows the server returning exactly the parameters that were asked for.

There is one subtlety that explains a lot about RFC performance. Before it can make a general call, the client has to know the shape of the function — the width and type of every parameter. So it first calls RFC_GET_FUNCTION_INTERFACE (that's the Metadata call tab), gets back the interface as a table of fixed-width rows, and only then pads each value to the width the metadata declares. A naive client pays that full round trip on every call. Caching the metadata is most of the difference between a fast RFC client and a slow one.

And when a call fails, the failure does not arrive where you'd expect. There is a LogonStatus field, and it is 0x00 — "fine" — even when the logon was rejected. Failure is signalled instead by an ABAP error field (0x0402), carrying a message like Name or password is incorrect. A T100 ABAP message even travels as its structured parts — class, number, and variables as separate fields — so a caller can switch on the stable message identity instead of matching translated text.

The rest of the protocol

The four layers above are the spine, and they carry the everyday traffic. The full specification goes further, and saprfc implements all of it against the live system:

• DDIC structures — structure and table parameters arrive as flat, aligned byte images whose layout comes from the data dictionary. The dictionary reports unaligned offsets, so the wire offsets must be computed; getting the alignment wrong reads every field after the first reference a few bytes early, decoding plausible-but-wrong values rather than failing. Deep BICS structures (SAP BW's analytic layer) took this to its limit.
• BASXML — SAP's "fast serialization" for deep/nested tables, a compact binary XML.
• Compression — replies above ~8 KB arrive LZC/LZH-compressed.
• Transactions — tRFC/qRFC/bgRFC unit lifecycles.
• Server mode — registering at the gateway and serving RFC calls, which is how erpl-rev lets ABAP call out into DuckDB.
• SNC — Secure Network Communications, the GSS-API/CommonCryptoLib encryption layer, with its own framing.
• Message-server load balancing and SAProuter hops.
• DECFLOAT — IEEE 754-2008 decimal floating point, on the wire.

Each of those is its own set of rules, each evidenced by its own capture.

How we know it's right

Reverse-engineering a format is easy to do approximately. The whole point of this project was to do it exactly, and that rests on four kinds of check.

Round-trip. Decode a captured record, re-encode it, and require the result to be the original bytes with zero residue. If our encoder and decoder disagree with the capture, the test fails.

Differential replay. This is the sharp one. After our own client logs in, it replays a captured record verbatim and checks the live server answers correctly — and then diffs our generated record against the captured reference, byte for byte. That turns an open-ended "why does the server treat us differently?" into a single bisection: if the server accepts the captured bytes but not ours, the fault is in our encoder; if it treats both the same, the fault is in our session state. Both real bugs we hit this way showed up as a message-length mismatch with otherwise identical fields.

A descriptor byte-equality gate. For metadata specifically, we built a tiny C program that dumps a function's type descriptor through SAP's SDK, and the same through saprfc, and the build fails unless they are byte-identical. Every structure layout in this post is backed by that gate agreeing with SAP's own library.

The live system, as the final word. None of this runs against mocks. Every change is checked against a real ABAP trial: the erpl SQL suites (RFC, ODP, BICS) have to pass, and a passing suite means our bytes and SAP's produce the same answers from the same server.

We are also honest about the boundary: the parts of the protocol we exercised are proven; the paths a capture never touched are inferred, and we say which is which. A specification that hides that line is telling you a story, not a spec.

A note on where this sits

For readers who know the field: the academic tools for protocol reverse engineering — Discoverer, Netzob, NETPLIER and the surveys around them — automate the inference and measure themselves against a Wireshark dissector as ground truth. That is the right tradeoff when you want breadth across many unknown protocols. Ours is the opposite tradeoff: one protocol, done manually, but validated to byte-exact conformance against the reference implementation and a live server rather than to a statistical accuracy figure. Different goal, stronger guarantee, much narrower scope.

Provenance and legality

This kind of work has a well-worn, lawful path, and we stayed on it deliberately.

The specification is derived from packet captures of the very library we are replacing — libsapnwrfc itself — which carries no third-party provenance. We did not disassemble it and did not read anyone else's source; open-rfc was used only as a black-box oracle, because reading its source would contaminate the clean-room chain. Every rule cites the capture that evidences it, and that citation trail is the record of independent derivation.

This is exactly how Samba was built, and the EFF's framing of it as adversarial interoperability — building a new thing that talks to the incumbent's protocol, from observation of that protocol — is the tradition this belongs to. The goal is interoperability, not appropriation: a customer who has paid SAP for their data should be able to read it with the tool of their choice.

Read it yourself

The bytes in the explorer above are not a mock-up. They are regenerated directly from the capture corpus by the same decoder saprfc uses in production, so this page cannot drift from the evidence: change a rule, rerun the generator, and the bytes on the page change with it. The password is masked; nothing else is altered.

saprfc is what lets erpl speak to your SAP system with no SDK, no ICU, and no post-install download — and it is measurably faster for having dropped them. The protocol was closed. It didn't have to stay that way.