Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Batch source to use Google Cloud Platforms's Bigtable as a Source.

...

SectionUser Configuration LabelLabel DescriptionMandatoryMacro-enabledOptionsDefaultVariableUser Widget
StandardReference NameThis will be used to uniquely identify this source for lineage, annotating metadata, etc++

referenceNameText Box

Table NameDatabase table name++

tableNametableText Box

Instance IDBigtable instance ID++

instanceIdinstanceText Box

Project IDThe ID of the project in Google Cloud
If not specified, will be automatically read from the cluster environment

+

projectIdprojectText Box

Service Account File Path

Path on the local file system of the service account key used for
authorization.

If the plugin is run on a Google Cloud Dataproc cluster, the service account key does not need to be provided and can be set to 'auto-detect'.
Credentials will be automatically read from the cluster environment.

When running on other clusters, the file must be present on every node in the cluster.

See Google's documentation on Service account credentials for details.


+

serviceFilePathText Box

Key Alias

Name of the field to set as the key fieldfor row key.




__key__keyAliasText Box

Scan Row Start

Scan start row.


+

scanRowStartText Box

Scan Row Stop

Scan stop row.


+

scanRowStopText Box

Scan Time Range Start

The starting timestamp used to filter columns with a specific range of versions.


+

scanTimeRangeStartText Box

Scan Time Range Stop

The ending timestamp used to filter columns with a specific range of versions.


+

scanTimeRangeStopText Box

Schema

Specifies the schema that has to be output. If not specified, then by default each item will be emitted as a JSON string. Only columns defined in schema will be included into output record. Field name should be in form "<family>:__<column>".

+


schemaschema
Error HandlingOn Record Error

How to handle error in record processing. Error will be thrown if failed to parse value according to provided schema.

+
  • Skip errorSend to error port
  • Fail pipeline
Skip erroron-errorRadio Button (layout: block)

...