Salesforce Marketing Cloud Batch Source
- Prateek Duble
Salesforce Marketing Cloud is a leading provider of Digital Marketing automation and analytics. It is used to create and manage marketing campaigns. Users can use the platform and the marketing APIs to create mailing lists, send emails, etc. This plugin would allow users to retrieve data from their Salesforce Marketing Cloud instance on objects such as accounts, activities, campaigns etc.
This plugin would support two modes of data retrieval:
- Multi-object data extraction - In this mode, the user will be allowed to select multiple objects and get data associated with those objects in separate files/tables based on the sink choice. This mode can only be used with one of CDAP's Multi-* sinks.
- Single-object data extraction - This mode would user to select a single object and extract the data associated with the specified object.
Use Cases
User would like to specify a Salesforce Marketing Cloud object so that she can get the entire dataset, perform intermediate transformations and load it in data warehouse for reporting and analytics purposes.
- User would like to retrieve SFMC data for a list of objects to perform an extract and load pattern, so that all the needed objects are available in the sink for complex analytical queries.
- User would like retrieve SFMC data so she can connect this data with other data sources to get full understanding of organization's digital marketing initiatives.
- User would like to get SFMC data for a specific object so that it can be transformed and used for specific types of reports such as campaign monitoring, user access and security etc.
Plugin Type
- Batch Source
- Batch Sink
- Real-time Source
- Real-time Sink
- Action
- Post-Run Action
- Aggregate
- Join
- Spark Model
- Spark Compute
User Configurations
| Section | User Configuration Label | Description | Default | User Widget | Notes |
|---|---|---|---|---|---|
| Authentication | Client ID | OAuth client ID for authentication | Text Box | ||
| Client Secret | OAuth client ID for authentication | Password | |||
| Basic | Data retrieval mode | Select Multi-object to retrieve data on multiple objects; Select Single-object for extracting and transforming data for a specific object | Drop-down | ||
| Object | Salesforce Marketing Cloud object name - ex: Campaign, Account, Emaiil etc. | Drop-down | Enabled only when data retrieval mode is set to Single-object | ||
| Objects list | List of Salesforce Marketing Cloud objects to retrieve | Text Box | Enabled only when data retrieval mode is set to Multi-object | ||
| Advanced | Error handling | Possible values: "Skip on error" - ignores any reports about records not inserted. Simply prints an error log.
| Skip on error | Select |
Design / Implementation Tips
- Data retrieve since last batch option: https://developer.salesforce.com/docs/atlas.en-us.noversion.mc-apis.meta/mc-apis/retrieving_all_records_since_last_batch.htm
- API Integration: https://developer.salesforce.com/docs/atlas.en-us.mc-app-development.meta/mc-app-development/api-integration.htm
Design
Approach(s)
Properties
Security
Limitation(s)
Future Work
Test Case(s)
- Test case #1
- Test case #2
Sample Pipeline
Please attach one or more sample pipeline(s) and associated data.
Pipeline #1
Pipeline #2
References
- Salesforce Marketing Cloud objects can be found at https://developer.salesforce.com/docs/atlas.en-us.noversion.mc-apis.meta/mc-apis/account.htm