Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1.  Lookup interface 

    Code Block
    interface Lookup<T> {
      T lookup(String key);
      Map<String, T> lookup(String... keys);
      Map<String, T> lookup(Set<String> keys);
    }
  2. Implement Lookup in KeyValueTable and ObjectMappedTable
    1. KeyValueTable implements Lookup<String>
    2. ObjectMappedTable implements Lookup<StructuredRecord>
  3. DatasetConfigurer changes
    1. Add method: void useDataset(String datasetName);
  4. ScriptTransform changes
    1. Add configuration property for declaring lookup tables to use, properties for each table (e.g. dataset properties)

      Code Block
      "tables": [
        {
          "name":"purchases",
          "type":"dataset",
          "properties": {
            "dataset":"purchases",
            "properties":{.. dataset properties ..},
            "enableCache":"true",
            "cacheExpiry":1234
          }
        },
        {"name":"ip2geo", "type":"file", "properties":{"file":"/data/ip2geo.csv"}}
      ]
    2. configure(): verify tables (datasets and files) exist by calling DatasetConfigurer.useDataset()
    3. transform(): execute lookup methods in a transaction, provide LookupKV Lookup instance to script
      1. Options for lookup usage: 

        Code Block
        var result = context.getTablegetLookup("purchases").lookup(user);
        // or
        var result = tables["purchases"].lookup(user);
        // or
        var result = purchases.lookup(user);
      2. Options for batch lookup usage:

        Code Block
        var result = context.getLookup("purchases").lookup(["alice", "bob"]);
        // do something with result["alice"]
        // do something with result["bob"]

...