Introduction
Marketo provide Marketing Automation software that allows companies to streamline, automate, and measure marketing tasks and workflows, so they can increase operational efficiency.
Use case(s)
- Read and replicate all of Marketo's entity data tables, specifically leads, activities, programs and campaigns. This would allow users to extract and load all of their Marketo data into specified multi-table or multi-file sink.
- Retrieve data from one of Marketo's entity data tables, allowing users to transform and enrich the data
User Storie(s)
- As a data pipeline developer, I should be able to import all of Marketo's entity data tables so that I can analyze Marketo data
- As a data pipeline developer, I should be able to specify Marketo entity (e.g. Leads) for which I can retrieve and transform the data
- As a data pipeline developer, I should be able to specify Client ID and Client secret that would allow me to extract the entity datasets from Marketo
- As a data pipeline developer, I should be able to see any errors from Marketo API calls so that I can fix those issues
- As a data pipeline developer, I do not need to specify fields associated with datasets to filter on so that I can get the entire
Plugin Type
- Batch Source
- Batch Sink
- Real-time Source
- Real-time Sink
- Action
- Post-Run Action
- Aggregate
- Join
- Spark Model
- Spark Compute
Configuration
Marketo source will support two different plugins
- Marketo Reporting Plugin - This plugin would get all the Marketo's entities (e.g. Leads, Activities, Campaigns, Companies etc.)
- Marketo Entity Plugin - This plugin would get only the data associated with specified entity (e.g. Leads)
Marketo Reporting plugin
This plugin should be used when the user would like to retrieve all of the Marketo datasets, specifically all the entities noted below:
- Lead Database: Leads, Activities, Campaigns, Companies, Opportunities, Sales Persons, Static Lists, Usage, Custom Objects
- Asset: Channels, Email, Email Templates, Files, Folders, Landing Pages Templates, Programs, Snippets, Tags, Tokens
| User Facing Name | Type | Description | Default value | Notes |
|---|---|---|---|---|
| Client ID | string | Marketo Client ID | ||
| Client Secret | string | Marketo Client secret | ||
| REST API Endpoint | string | REST API endpoint URL | ||
| REST API Identity | string | REST API identity | ||
| Daily API limit | integer | Marketo enforced daily API limit | Additional daily capacity can be purchased and hence user needs to provide this information. https://developers.marketo.com/rest-api/marketo-integration-best-practices/ | |
| Report format | string | Report format | CSV | Options are CSV, TSV or SSV |
| Start date | string | Start date to extract data from | ISO 8601 format | |
| End date | string | End date to extract data upto | ISO 8601 format |
Marketo Entity plugin
This plugin should be used when the user would like to extract the data associated with a single entity (e.g. Leads)
| User Facing Name | Type | Description | Default value | Notes |
|---|---|---|---|---|
| Entity Name | string | Marketo entity name to fetch | Possible entity values include:
| |
| Client ID | string | Marketo Client ID | ||
| Client Secret | string | Marketo Client secret | ||
| REST API Endpoint | string | REST API endpoint URL | ||
| REST API Identity | string | REST API identity | ||
| Daily API limit | integer | Marketo enforced daily API limit | Additional daily capacity can be purchased and hence user needs to provide this information. https://developers.marketo.com/rest-api/marketo-integration-best-practices/ | |
| Report format | string | Report format | CSV | Options are CSV, TSV or SSV |
| Start date | string | Start date to extract data from | ISO 8601 format | |
| End date | string | End date to extract data upto | ISO 8601 format |
Design / Implementation Tips
- Users dont need to specify particular fields to filter on. They will specify the entiry to extract and the plugin will ensure that all the fields are extracted for the specified entity.
Design
Approach(s)
Properties
Security
Limitation(s)
Future Work
Test Case(s)
- Test case #1
- Test case #2
Sample Pipeline
Please attach one or more sample pipeline(s) and associated data.
Pipeline #1
Pipeline #2
References
- Marketo Developers Guide: https://developers.marketo.com/getting-started/
- Bulk Export Leads: https://developers.marketo.com/rest-api/endpoint-reference/lead-database-endpoint-reference/#/Bulk_Export_Leads
- Bulk Export Activities: https://developers.marketo.com/rest-api/endpoint-reference/lead-database-endpoint-reference/#/Bulk_Export_Activities
- Marketo REST Sample code: https://github.com/Marketo/REST-Sample-Code