SAP Integration in AWS S3 – ERPL - Connect DuckDB to the SAP ecosystem

Why Amazon S3?

Amazon S3 (Simple Storage Service) is a scalable object storage service offered by Amazon Web Services (AWS) that allows you to store and retrieve any amount of data from anywhere on the web. It is designed to make web-scale computing easier for developers by providing a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the internet. Our SAP connector ERPL is an efficient and flexible connector for integrating SAP data in S3.

Start DuckDB & install and load the ERPL extension

The following examples are executed in the CLI, but are easily replicatable in any other language that supports DuckDB. Let’s start. First, we need to start DuckDB from CLI using the --unsigned flag.

duckdb --unsigned

Install and load the RFC extension and set the SAP login parameters:

SET custom_extension_repository = 'http://get.erpl.io';
FORCE INSTALL erpl;
LOAD erpl;

SET sap_ashost = 'localhost';
SET sap_sysnr = '00';
SET sap_user = 'DEVELOPER';
SET sap_password = 'ABAPtr1909';
SET sap_client = '001';
SET sap_lang = 'EN';

As we want to replicate the SAP data to Amazon S3, a connection needs to be established using your Amazon S3 credentials. Since duckdb 0.10 the credentials can be saved securely in a secret object in DuckDB. Furthermore, through the AWS extension provided by DuckDB credentials can be provided by the [AWS Default Credential Provider Chain] (https://duckdb.org/docs/extensions/aws).

CREATE SECRET (
    TYPE S3,
    KEY_ID '[KEY]',
    SECRET '[SECRET]',
    REGION '[REGION]'
);

If you are working with DuckDB version v0.9.2 and lower, the credentials must be set as follows:

SET s3_region='[REGION]';
SET s3_access_key_id='[SECRET]';
SET s3_secret_access_key='[KEY]';

Please follow the instructions on the DuckDB website if there are any issues with the Amazon S3 connection.

We assume that the Bucket already exists in the desired target regions for your SAP data.

Query your SAP Database using RFC and Copy Data to Amazon S3

Create the query for data you want to replicate to Amazon S3. In this example we will use RFC for extracting data to DuckDB. Have a look at ODP or BW help pages on how to create ODP and BICS queries.

COPY (SELECT * FROM sap_rfc_invoke('BAPI_FLIGHT_GETLIST', path='/FLIGHT_LIST')) TO s3://[YOUR BUCKET]/sflight.parquet

With one line of code your SAP data is placed on S3. Happy analyzing.