Compound fields

Support compound field types: address, location.

For example, Account SObject contains two compound fields: BillingAddress, ShippingAddress.

User should be able to execute SOQL which includes such fields:

SELECT ShippingAddress, BillingAddress FROM Account

Implement compound fields processing strategy.

EXCLUDE - compound fields will be excluded from SOQL. Processing type: Bulk API. Default strategy.

FLATTEN - compound fields will be replaced with individual fields. Processing type: Bulk API.

Example: BillingAddress field will be replaced with:

BillingStreet, BillingCity, BillingState, BillingPostalCode, BillingCountry, BillingLatitude, BillingLongitude, BillingGeocodeAccuracy

INCLUDE - compound field will be fetched in JSON format. Processing type: Bulk API + SOAP API. Note: performance may be impacted.

Example:

{
 "city": "San Francisco",
 "country": "US",
 "countryCode": null,
 "geocodeAccuracy": null,
 "latitude": null,
 "longitude": null,
 "postalCode": "94087",
 "state": "CA",
 "stateCode": null,
 "street": "The Landmark @ One Market"
}

More details: https://developer.salesforce.com/docs/atlas.en-us.api.meta/api/compound_fields.htm

User should be able to execute SOQL with offset:

SELECT Name, Id FROM Opportunity OFFSET 30

Adapt existing WideObject query processing to query all SOQL queries with OFFSET clause. Processing type: Bulk API + SOAP API.

Note: performance may be impacted.

Functions: COUNT, SUM

User should be able to execute SOQL with aggregate functions:

SELECT Id,count(Name) cnt,count(Name),sum(Amount),count(CloseDate) FROM Opportunity GROUP BY Id

New SOAP API query() reader should be implemented.

Limitations:

1. Query cannot be processed in batches. All work will be done by one mapper. Performance can be significantly impacted.
2. Wide queries are not supported, i.e. if user provides query that exceeds the 20,000 character limit, switch to WideObject query processing is not possible, pipeline will fail.

Relationships: ROLLUP, GROUP BY CUBE

User should be able to execute SOQL with aggregate relationships:

SELECT Status, LeadSource, COUNT(Name) cnt FROM Lead GROUP BY ROLLUP(Status, LeadSource)

SELECT Type, BillingCountry, GROUPING(Type) grpType, GROUPING(BillingCountry) grpCty, COUNT(id) accts FROM Account GROUP BY CUBE(Type, BillingCountry) ORDER BY GROUPING(Type), GROUPING(BillingCountry)

New SOAP API query() reader should be implemented.

Limitations:

1. Query cannot be processed in batches. All work will be done by one mapper. Performance can be significantly impacted.
2. Wide queries are not supported, i.e. if user provides query that exceeds the 20,000 character limit, switch to WideObject query processing is not possible, pipeline will fail.

User should be able to execute SOQL with nested SOQL query:

SELECT Id,Account.Name, (SELECT Id,Name,Contact.LastName FROM Account.Contacts) FROM Account

New SOAP API query() reader should be implemented.

Limitations:

1. Query cannot be processed in batches. All work will be done by one mapper. Performance can be significantly impacted.
2. Wide queries are not supported, i.e. if user provides query that exceeds the 20,000 character limit, switch to WideObject query processing is not possible, pipeline will fail.

Compound fields processing strategy. Compound fields group together multiple elements of primitive data types, such as numbers or strings, to represent complex data types, such as a location or an address. Can be one of three options:

EXCLUDE - compound fields will be excluded from SOQL. Processing type: Bulk API. Default strategy.

FLATTEN - compound fields will be replaced with individual fields. Processing type: Bulk API.

Example: BillingAddress field will be replaced with:

BillingStreet, BillingCity, BillingState, BillingPostalCode, BillingCountry, BillingLatitude, BillingLongitude, BillingGeocodeAccuracy

INCLUDE - compound field will be fetched in JSON format. Processing type: Bulk API + SOAP API. Note: performance may be impacted.