Versions Compared

Old Version 20

changes.mady.by.user Sagar Kapare

Saved on

compared with

New Version 21

changes.mady.by.user Chris Lu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

 DescriptionProperties requiredJSON Example
FileCopy/FileMove
Action		To be able to copy files from FTP/SFTP
to unix machine, from unix machine to
the HDFS cluster, from FTP to HDFS.

To be able to move the files from source location A
(HDFS/Unix machine/SFTP) to destination location B
(HDFS/unix machine/SFTP).  
  • Source A (SSH into, see properties)
  • Source B
    • hostname
    • login credentials
    • /PATH to file
{		
    ... 
    "config":{
        "connections":[
         {
            "from": "Copy-File,
            "to": *other end*,
         },
         ...
        ],
        "stages":[
         {
            "name":"Copy-File",
            "plugin":{
               "name":"SCPFile-Manipulation",
               "type":"action",
               "artifact":{
                  "name":"core-action-plugins",
                  "version":"1.4.0-SNAPSHOT",
                  "scope":"SYSTEM"
               },
               "properties":{
                  "type": "SCP"
	          "source": {
                      "host": "hostA.com",
                      "login": "username"
                   },
                  "destination": {
                      "host": "hostB.com",
                      "login": "username",
                      "file-path": "/filepath"
                   } 
               }
            }
         },
         ...
	]	  
    }
}
SSH Script ActionTo be able to execute Perl/Python/R/Shell/Revo R/Hive Query
scripts located on the remote machine
  • hostname
  • login credentials (keyfiles?)
  • /PATH to script
  • arguments for script
{		
    ... 
    "config":{
        "connections":[
         {
            "from": "Copy-File,
            "to": *other end*,
         },
         ...
        ],
        "stages":[
         {
            "name":"Copy-File",
            "plugin":{
               "name":"SSH",
               "type":"action",
               "artifact":{
                  "name":"core-action-plugins",
                  "version":"1.4.0-SNAPSHOT",
                  "scope":"SYSTEM"
               },
               "properties":{
                  "type": "R", //used to figure out what command to execute
	          "host": "scripthost.com",
                  "script-path":"/tmp/myscript",
                  "login": "username",
                  "arguments":[
				{ "name": "timeout", "value": 10 },
				{ "name": "user", "value": "some_user"}
			      ]
               }
            }
         },
         ...
	]	  
    }
}
SQL ActionTo be able to run the SQL stored procedures located on
the remote SQL server, Copy Hive data to SQL Server table 
  • username
  • database name
  • file to push
 
Email actionTo be able to send emails
  • recipient/sender email
  • message
  • subject
  • username/password auth (if needed)
  • protocol (smtp, smtps, tls)
  • smtp host
  • smtp port
{		
    ... 
    "config":{
        "connections":[
         {
            "from": "Copy-File,
            "to": *other end*,
         },
         ...
        ],
        "stages":[
         {
            "name":"Email-Bob",
            "plugin":{
               "name":"Email",
               "type":"action",
               "artifact":{
                  "name":"core-action-plugins",
                  "version":"1.4.0-SNAPSHOT",
                  "scope":"SYSTEM"
               },
               "properties":{
	          "protocol": "smtp",
                  "recipient": "bob@cask.co",
                  "subject": "PR Review",
                  "auth": {
                      "username": "username",
                      "password": "emailpass",
                  "message": "Hey Bob, could you take a look at this PR?"
               }
            }
         },
         ...
	]	  
    }
}

...