SAP Integration in Google Cloud Storage – ERPL - Connect DuckDB to the SAP ecosystem

Why Google Cloud Storage?

Google Cloud Storage is a robust and scalable object storage solution provided by Google Cloud, designed to store and access any amount of data from anywhere on the internet. It aims to simplify cloud storage for developers by offering a straightforward RESTful API, enabling the storage and retrieval of data at any scale and from any location globally. Our SAP connector for Google Cloud Storage enhances this platform’s capabilities by enabling seamless integration of SAP data into Google Cloud Storage. This connector is designed for efficiency and flexibility, making it easier for organizations to leverage their SAP data within the versatile environment of Google Cloud Storage, thereby optimizing data management and accessibility.

Unfortunately, you cannot copy your extracted results directly to Google Cloud Storage. Therefore, you need to use CLI, Python, or R interfaces to copy the final database or parquet file to your cloud storage.

Start DuckDB & install and load the ERPL extension

The following examples are executed in the CLI, but are easily replicatable in any other language that supports DuckDB. Let’s start. First, we need to start DuckDB from CLI using the --unsigned flag.

duckdb --unsigned

Install and load the RFC extension and set the SAP login parameters:

SET custom_extension_repository = 'http://get.erpl.io';
FORCE INSTALL erpl;
LOAD erpl;

SET sap_ashost = 'localhost';
SET sap_sysnr = '00';
SET sap_user = 'DEVELOPER';
SET sap_password = 'ABAPtr1909';
SET sap_client = '001';
SET sap_lang = 'EN';

Query your SAP Database using RFC and Copy Data to Google Cloud Storage

Create the query for data you want to replicate to Google Cloud Storage. In this example we will use RFC for extracting data to DuckDB. Have a look at ODP or BW help pages on how to create ODP and BICS queries.

COPY (SELECT * FROM sap_rfc_invoke('BAPI_FLIGHT_GETLIST', path='/FLIGHT_LIST')) TO sflight.parquet

Now, you may use any tool to copy the generated file to your storage:

CLI

gsutil cp sflight.parquet gs://[BUCKET_NAME]/[DESTINATION_FILE_PATH]

Python

from typing import Text
from google.cloud import storage

def upload_to_blob(bucket_name: Text, source_file_name: Text, destination_blob_name: Text):
    """Uploads a file to the bucket.
    
    Parameters
    ----------
    bucket_name
      The ID of your GCS bucket
    source_file_name
      The path to your file to upload
    destination_blob_name
      The ID to give your GCS blob
    """

    try:
      storage_client = storage.Client()
      bucket = storage_client.bucket(bucket_name)
      blob = bucket.blob(destination_blob_name)

      blob.upload_from_filename(source_file_name)

      print(f"File {source_file_name} uploaded to {destination_blob_name}.")
    except Exception as e:
        print(f"Error uploading file to Google Cloud Storage: {e}")

Happy analyzing.