...

This plugin would allow SAP ECC enterprise users to specify one of the Inventory Management data sources as source in Cloud Data Fusion. Typical pipeline scenario would include specifying one of the data sources as source (under Sources plugins) and BigQuery as sink. For the sake of simplicity, this document will cover one data source (Material Movements from Inventory Management - 2LIS_03_BF).

User Experience and Configuration

• SAP ECC Setup

1. SAP ECC users log into SAP Gateway Service Builder (segw) to create and expose the data source as SAP OData service. SAP has published following articles around this step:

1. OData service introduction 

2. Step-by-step guide to build OData service

2. User creates OData service for the data source (2LIS_03_BF). Output is RESTful OData service that can be accessed by Cloud Data Fusion.

3. User can also use SAP Netweaver Gateway client to test the service (with HTTP request and response)

...

• Data type mismatch - OData output has data types (section 6) different than the data types available in BigQuery. Some sort of data type conversion needs to happen before the extracted data can be ingested into BigQuery. Below is the suggested mapping of OData data types to CDAP schema data types:

...

Types can be mapped as follows, according to the org.apache.olingo.commons.api.edm.EdmPrimitiveType and OData CSDL Primitive Types:

...

Properties of these types are mapped to a record with corresponding fields and each of them is mapped according to it's Edm type: OData V2 Data Types Mapping, OData V4 Data Types Mapping.

OData V4 EnumType and TypeDefinition

OData V4 supports enumeration types and type definitions, that are defined as follows:

<?xml version="1.0" encoding="utf-8"?>
<edmx:Edmx Version="4.0" xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx">
<edmx:DataServices>
  <Schema Namespace="TestService" xmlns="http://docs.oasis-open.org/odata/ns/edm" Alias="self">
    <EnumType Name="PersonGender">
      <Member Name="Male" Value="0"/>
      <Member Name="Female" Value="1"/>
      <Member Name="Unknown" Value="2">
        <Annotation Term="Core.Description" String="Shipped with highest priority" />
      </Member>
    </EnumType>
    <TypeDefinition Name="Length" UnderlyingType="Edm.Int32">
      <Annotation Term="Org.OData.Measures.V1.Unit" String="Centimeters" />
    </TypeDefinition>
    <TypeDefinition Name="Weight" UnderlyingType="Edm.Int32">
      <Annotation Term="Org.OData.Measures.V1.Unit" String="Kilograms" />
    </TypeDefinition>
    ...
    <EntityType Name="AllDataTypesEntity">
      ...
      <Property Name="Gender" Type="self.PersonGender" Nullable="false"/>
      <Property Name="Height" Type="self.Length" />
      <Property Name="Weight" Type="self.Weight" />
    </EntityType>
    ...
    </Schema>
</edmx:DataServices>
</edmx:Edmx>

Properties of the enum types are mapped to string. Properties of the TypeDefinition are mapped according to it's underlying Edm type: OData V2 Data Types Mapping, OData V4 Data Types Mapping.

References

• OData service

• Generic OData overview

• Building New OData services

• SAP Netweaver Gateway

• Overview - https://blogs.sap.com/2013/01/24/a-simple-overview-on-sap-netweaver-gateway/

• SAP Netweaver Gtw and OData Tutorials: https://sapyard.com/tutorials-on-odata-sap-netweaver-gateway/ 

• Microsoft Data Factory - SAP ECC Connector

• Microsoft Data Factory - ECC Connector

• Step-by-step guide to setup SAP ECC as source in Azure Data Factory

Plugin Type

•  Batch Source
•  Batch Sink 
•  Real-time Source
•  Real-time Sink
•  Action
•  Post-Run Action
•  Aggregate
•  Join
•  Spark Model
•  Spark Compute

...