Google drive plugins will help users move entire files from source to destination. Along the way, users can potentially run transformations on unstructured data such as images, audio and video as well.
User Storie(s)
As a pipeline developer, I want to move all files from a Google drive directory to a different destination
As a pipeline developer, I want to move all files from a Google drive directory that satisfy a filter to a different destination
As a pipeline developer, I want to pull all images from a Google drive directory, so that I can process them using image recognition APIs
As a pipeline developer, I want to pull all audio and video files from a Google drive directory, so that I can process them to extract metadata and/or generate transcripts, or apply other enrichments.
As a pipeline developer, I want to move all files from an FTP source into Google drive.
Plugin Type
Batch Source
Batch Sink
Real-time Source
Real-time Sink
Action
Post-Run Action
Aggregate
Join
Spark Model
Spark Compute
Configurables
This section defines properties that are configurable for this plugin.
Source
User Facing Name
Type
Description
Optional
Constraints
App Id
String
Oauth2 app id
No
Access Token
String
OAuth2 access token
No
Directory identifier
String
ID is the last part of the URL, such as https://drive.google.com/drive/folders/0B2kqcwp2ycGZanhSR3JmREw5VTV
no
Filter
String
A filter that can be applied to the files in the selected directory. Filters follow the Google Drive Filter Syntax
Yes
Modification date range
String
In addition to the filter specified above, also filter files to only pull those that were modified between the date range