Requirements
- Operations
- Perform single + batch read on single + multiple dataset from script transform
- Perform single + batch read on DistributedCache from script transform
- Supported datasets for lookup
- Key-value table
- ObjectMappedTable
- Optional caching with time-based expiration
Proposed changes
- TransformContext changes
- Rename TransformContext to StageContext (since it is used as the base context in BatchRuntimeContext, not only transform contexts.
- Create TransformContext which extends StageContext
- Transform changes
- Lookup interface with lookup methods that perform read operations on datasets
- DefaultLookup: implementation that uses Transactional to implement Lookup, used in ETLWorker
- Add Lookup field to ScriptContext, so Lookup is accessible via transforms that interpret JavaScript
- Sample usage: ctx.lookup.lookupKVString('purchases', 'key')