The salesforce batch sink is responsible for using Salesforce API to upsert salesforce objects. The sink should handle large batches of data (~10 GB) and should handle all object types - contacts, campaigns, oppurtunities, leads, custom objects. Users should be able to upload subset of fields
...
| Section | User Configuration Label | Description | Default | User Widget | Early Validations |
|---|---|---|---|---|---|
| Authentication | Username | Salesforce username | Text Box | Try a to login to bulk API with given credentials. | |
| Password | Password | ||||
| Consumer Key | Consumer Key from the connected app | Text Box | |||
| Consumer Secret | Consumer Secret from the connected app | Password | |||
| Login Url | For Salesforce sandbox runs login url is different. That's why user needs to have this option. | https://login.salesforce.com/services/oauth2/token | Text Box | ||
| Advanced | SObject | Name of Salesforce sObject - ex: Contact, Campaign, Oppurtunity. | Text Box | Check if sObject with given name exists in Bulk API. | |
| Maximum bytes per batch | If size of batch data is larger than given number of bytes, split the batch. | 10,000,000 [2] | Text Box | If more than 10,000,000 than fail [3] | |
| Maximum records per batch | If there are more than given number of records, split the batch. | 10,000 [2] | Text Box | If more than 10,000 fail [4] | |
| Error handling | Bulk API will return success results per row so this is necessary [1] (unlike for source plugins). Possible values: "Skip on error" - ignores any reports about records not inserted | Select |
...
We will have to convert logical types like date, datetime, time from long to string format accepted by Salesforce. Other types won't require converting.
| Type | format expected by Salesforce |
|---|---|
| date | yyyy-mm-dd |
| datetime | ISO 8601 |
| time | HH:mm:ss,SSS |
Compound fields
For source plugins we don't use nested records for compound fields. Instead we ask user to provide sub-fields for every compound object separately.
...