Versions Compared
Key
- This line was added.
- This line was removed.
- Formatting was changed.
Introduction
Zuora is a subscription billing and finance solution. It is designed to automate recurring billing, collections, quoting, revenue recognition, and subscription metrics.
User Storie(s)
- As a pipeline developer, I would like to pull data from an object in Zuora in a streaming pipeline, so that I can transform, enrich and synthesize it in a data pipeline.
- As a pipeline developer, I would like to post data to an object in Zuora in a batch or streaming pipeline, so that I can update Zuora with my enriched data.
- As a pipeline developer, I would like to pull data from an object in Zuora in a batch pipeline, so that I can transform, enrich and synthesize it in a data pipeline.
- As a pipeline developer, for both batch and streaming sources, I would like the schema to be auto-populated based on the selected objects.
- As a pipeline developer, for all interactions with Zuora, I would like Zuora API limits to be handled by the framework, so that I do not have to worry about them. (https://knowledgecenter.zuora.com/BB_Introducing_Z_Business/Policies/Concurrent_Request_Limits)
Plugin Type
- Batch Source
- Batch Sink
- Real-time Source
- Real-time Sink
- Action
- Post-Run Action
- Aggregate
- Join
- Spark Model
- Spark Compute
Configurables
Batch source - single object/multi object
Should automatically populate schema for the selected object
| Section | User Facing Name | Type | Description | Optional? |
|---|
| Default | Constraints | |||||
|---|---|---|---|---|---|---|
| Authentication | Authentication Type | radio button | The Authentication type to use, either OAuth2 or Basic. Defaults to Basic. | No | Basic | |
| Username | String | The Zuora ID to use to make request. Only required when authentication type is Basic. | Yes | |||
| Password | password | The Zuora password. Only required when authentication type is Basic. | Yes | |||
| Client Id | String | The Client ID of the OAuth2 client. Only required when Authentication Type is OAuth2 | Yes | |||
| Client Secret | password | The Client Secret that was displayed when the OAuth2 client was created. Only required when Authentication Type is OAuth2 | Yes | |||
| REST API Endpoint | String | The one of Zuora API endpoint to use: | No | |||
| Basic | Object(s) to pull | select | The Zuora object to pull from the list at https://www.zuora.com/developer/api-reference/#section/Zuora-Object-Model | No | Account |
Streaming source
Should automatically populate schema for the selected object
| Section | User Facing Name | Type | Description | Optional? | Default | Constraints | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Authentication | Authentication Type | radio button | The Authentication type to use, either OAuth2 or Password or Authorization Cookie. Defaults to OAuth2. | No | OAuth2 | ||||||||||
| Access Key ID | String | The Zuora ID to use to make request. Only required when authentication type is password. | Yes | ||||||||||||
| Secret Key | password | The Zuora password. Only required when authentication type is password. | Yes | ||||||||||||
| Client Id | String | The Client ID of the OAuth2 client. Only required when Authentication Type is OAuth2 | Yes | ||||||||||||
| Client Secret | password | The Client Secret that was displayed when the OAuth2 client was created. Only required when Authentication Type is OAuth2 | Yes | ||||||||||||
| Authorization Cookie | password | The authorization cookie to use for session authentication. Only required when authentication type is Authorization Cookie. | Yes | ||||||||||||
| Basic | Object (s) to pull | multi-select | Select from Contact Lists, Contacts, Email Events, Email Subscription, Recent Campaigns, Analytics, Companies, Deals, Deal Pipelines, Marketing Email, Products, Tickets | Yes | Analytics | Pull frequency | select | 15 min, 30 min, 1 hour, 4 hour, 8 hours | Yes | 30 min | Number of API calls per day | number | The number of API calls to make per day, to avoid hitting HubSpot's rate limits | Yes | 10000 |
| The Zuora object to pull from the list at https://www.zuora.com/developer/api-reference/#section/Zuora-Object-Model | No | Account |
Batch sink
Writes data to the selected object in Zuora
| User Facing Name | Type | Description | Optional? | Default | Constraints | App Id | String | OAuth2 App ID | No | API Key | String | OAuth2 API Key | No | Object to post to | select | Select from Contact Lists, Contacts, Email Events, Email Subscription, Recent Campaigns, Analytics, Companies, Deals, Deal Pipelines, Marketing Email, Products, Tickets | No | Number of API calls per day | number | The number of API calls to make per day, to avoid hitting HubSpot's rate limits | Yes | 10000 |
|---|
Batch Source
Design / Implementation Tips
- Tip #1
- Tip #2
Design
Approach(s)
Properties
Security
Limitation(s)
Future Work
- Some future work – HYDRATOR-99999
- Another future work – HYDRATOR-99999
Test Case(s)
- Test case #1
- Test case #2
Sample Pipeline
Please attach one or more sample pipeline(s) and associated data.
Pipeline #1
Pipeline #2| Authentication Type | radio button | The Authentication type to use, either OAuth2 or Password or Authorization Cookie. Defaults to OAuth2. | No | OAuth2 | |
| Access Key ID | String | The Zuora ID to use to make request. Only required when authentication type is password. | Yes | ||
| Secret Key | password | The Zuora password. Only required when authentication type is password. | Yes | ||
| Client Id | String | The Client ID of the OAuth2 client. Only required when Authentication Type is OAuth2 | Yes | ||
| Client Secret | password | The Client Secret that was displayed when the OAuth2 client was created. Only required when Authentication Type is OAuth2 | Yes | ||
| Authorization Cookie | password | The authorization cookie to use for session authentication. Only required when authentication type is Authorization Cookie. | Yes | ||
| Object to write to | select | The Zuora objects to pull from the list at https://www.zuora.com/developer/api-reference/#section/Zuora-Object-Model | No | Account |
Design / Implementation Tips
As the reference were used Zuora API version 230.0 of swagger definition. The definition could be obtained from the link in the article or be downloaded directly.
zuora/src location contains README.md file with addition information about auto-generated Zuora object definitions from the swagger file
API documentation: https://www.zuora.com/developer/api-reference/
Authorization
Plugin supporting Basic and OAuth2 authorization. OAuth2 method require user to enable such functionality in Zuora configuration or follow the instruction in the documentation:
https://knowledgecenter.zuora.com/CF_Users_and_Administrators/A_Administrator_Settings/Manage_Users; Section "Create an OAuth Client for a User"
Rate limiting
Zuora API have an request rate limiting for the object. The API requester have integrated way to retry over time to fetch wait for penalty time.
Documentation: https://knowledgecenter.zuora.com/BB_Introducing_Z_Business/Policies/Concurrent_Request_Limits
Approach(s)
Create a module zuora plugin in zuora project. Add Zuora objects with their deserialization from API and transformation to structured records.
Test Case(s)
TODO
Sample Pipeline
TODO
Table of Contents
Table of Contents style circle
Checklist
- User stories documented
- User stories reviewed
- Design documented
- Design reviewed
- Feature merged
- Examples and guides
- Integration tests
- Documentation for feature
- Short video demonstrating the feature