...
Section | Field | Type | Description |
---|---|---|---|
Basic Configuration | Input FileName | String | Full Name of File including path ( URI) |
Compress File | Boolean | True / False | |
Compression Algorithm | String ( List) | Gzip / Snappy. Applicable only if above Compress File is set to true | |
Encrypt File | Boolean | True | |
PGP Public Key Path | String | Location of PGP public key. Path to File | |
PGP Public Key Access Userid | String | Userid to access the public key incase security is enabled | |
PGP Public Key Access password | String | Password to access the key file | |
OutFilePath | String | Path to store the output file from sync. The output filename will follow the format of <InputfileName Suffix>.gz.pgp The file path URI can contain filesystem , Hdfs, gcs - google file system or cloud store. | |
MoveInput | Boolean | True / False - Move the source input file to a different path so the next run of the pipeline the same file will not be processed. | |
MoveFilePath | String | Path to move the input on successful processing of the file. |
Usecase 2
FileDecompressDecrypt Plugin
The plugin support decrypting files using PGP Public key and decompress file.
Plugin Properties
Section | Field | Type | Description |
---|---|---|---|
Basic Configuration |
Path | String |
Full Name of File including path ( URI)
Output
Schema
Path containing file name or directory of files. | |||
Recursive Processing | Boolean | True / False | |
DeCompress File | Boolean | True / False | |
DeCompression Algorithm | String ( List) | Gzip / Snappy. Applicable only if above Compress File is set to true | |
DeEncrypt File | Boolean | True | |
PGP Private Key Path | String | Location of PGP public key. Path to File | |
PGP Private Key Access Userid | String | Userid to access the public key incase security is enabled | |
PGP Private Key Access password | String | Password to access the key file | |
MoveInput | Boolean | True / False - Move the source input file to a different path so the next run of the pipeline the same file will not be processed. | |
MoveFilePath | String | Path to move the input on successful processing of the file. | |
Output Schema | Output | String | Each Row from the file read. |