Goal
The Row Denormalizer plugin allows one to de-normalise data along with ability to convert data types.
Checklist
- User stories documented
- User stories reviewed
- Design documented
- Design reviewed
- Feature merged
- Examples and guides
- Integration tests
- Documentation for feature
- Blog post
Use-case
Imagine that a source database has a table that stores a variable set of custom attributes for an entity like the one defined below. This model allows adding any number of attributes to an entity.
Key Field
Field Name
Field Value
Key Field | Field Name | Field Value |
---|---|---|
joltie | FIRST_NAME | Nitin |
joltie | LAST_NAME | Motgi |
joltie | ADDRESS | 150 Grant Ave, Suite A |
joltie | CITY | Palo Alto |
But once the data is being passed through the Row Denormaliser it should map to the following making it easy to query the data.
Key | FIRST_NAME | LAST_NAME | addr | CITY |
---|---|---|---|---|
joltie | Nitin | Motgi | 150 Grant Ave, Suite A | Palo Alto |
Conditions
In case a field is value is not present then it’s considered as NULL
Options
User is able to specify the Key Field based on the Input Schema (has to be field in Input Schema). This the key of the output row. From the above example it’s “Key Field”
User is able to specify the list of fields that should be considered to form a denormalized record. From the above example it should be ‘FIRST_NAME’, ‘LAST_NAME’, ‘ADDRESS’ & ‘CITY’
Users are able to specify the output field name for each through mapping. From the above example ‘ADDRESS’ in input is mapped to ‘addr’ in output schema.
Similarly simple type conversions should be attempted - {int, long, float, double} -> string
Design