...
| name | description | required? | default |
|---|---|---|---|
| name | app name | yes | |
| artifact | scope, name, version of the artifact to create the app with | yes | |
| config | app config (file in the package archive) | no | empty |
create_stream
Results in a call to http://docs.cdap.io/cdap/current/en/reference-manual/http-restful-api/stream.html#creating-a-stream
...
| name | description | required? | default |
|---|---|---|---|
| name | pipeline name | yes | |
| artifact | scope, name, version of the artifact to create the app with | yes | |
| config | pipeline config (file in the package archive) | yes |
Dependencies
Packages will only be able to specify dependencies on the CDAP version, as well as dependencies on the existence of specific CDAP entities. For example, the core-plugins-1.5.0 package requires that there exist system artifacts cdap-data-pipeline-4.0.0 and cdap-data-streams-4.0.0 in the CDAP instance.
...
| Code Block |
|---|
GET /groups/hydrator-plugins/packages/sfdc-lead-dump/versions/1.0.1/spec
{
"name": "sfdc-lead-dump",
"label": "SFDC Lead Dump",
"description": "Reads SFDC data from a CDAP Stream, filters invalid records, and dumps the data to a CDAP Table.",
"author": "Cask",
"org": "Cask Data Inc.",
"version": "1.0.1",
"created": 1234567899,
"changelog": [
"fixed a small parsing bug"
],
"actions": [
{
"type": "create_artifact",
"arguments": [
{
"name": "scope",
"value": "user",
"canModify": false
},
{
"name": "name",
"value": "sfdc-plugins",
"canModify": false
},
{
"name": "version",
"value": "1.0.0",
"canModify": false
},
{
"name": "parents",
"value": "system:cdap-data-pipeline[4.0.0,4.1.0)",
"canModify": false
},
] {
}, { "typename": "create_hydrator_draftjar",
"arguments": [ "value": "sfdc-plugins.jar", // file in the {archive
"namecanModify": "artifact",false
}
"value": { ]
},
"scope": "system",
{
"nametype": "cdap-data-pipelinecreate_hydrator_draft",
"arguments": [
"version": "4.0.0"{
}"name": "artifact",
"canModifyvalue": false{
}, "scope": "system",
{ "name": "namecdap-data-pipeline",
"valueversion": "SFDC Lead Dump"4.0.0"
},
"canModify": truefalse
},
{
"name": "configname",
"value": {"SFDC [hydrator config here] }Lead Dump",
"canModify": falsetrue
},
{
]"name": "config",
} ]
} |
...
"value": "sfdc.json", // file in the archive
"canModify": false
}
]
}
]
} |
The UI also fetches the spec signature and uses the public key to validate the spec:
| Code Block |
|---|
GET /groups/hydrator-plugins/packages/sfdc-lead-dump/versions/1.0.1/spec.asc |
The UI also fetch the package archive and signature. It validates the package, and writes the archive to a local temporary directory so that it can use its resources to create the plugins artifact and create the hydrator draft
| Code Block |
|---|
GET /groups/hydrator-plugins/packages/sfdc-lead-dump/versions/1.0.1/archive.tgz
GET /groups/hydrator-plugins/packages/sfdc-lead-dump/versions/1.0.1/archive.tgz.asc |
Based on the package spec, the UI can setup the relevant wizards and make the relevant CDAP calls to first create the plugin artifact, and next create the Hydrator draft.
Scenario 7: Add MySQL jdbc driver as a Hydrator plugin.
...