Versions Compared

Old Version 22

changes.mady.by.user Chris Lu

Saved on

compared with

New Version 23

changes.mady.by.user Chris Lu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

 DescriptionProperties requiredJSON Example
FileCopy/FileMove
Action		To be able to copy files from FTP/SFTP
to unix machine, from unix machine to
the HDFS cluster, from FTP to HDFS.

To be able to move the files from source location A
(HDFS/Unix machine/SFTP) to destination location B
(HDFS/unix machine/SFTP).  
  • Source A (SSH into, see properties)
  • Source B
    • hostname
    • login credentials
    • /PATH to file
{		
    ... 
    "config":{
        "connections":[
         {
            "from": "Copy-File,
            "to": *other end*,
         },
         ...
        ],
        "stages":[
         {
            "name":"SCP-Copy-File",
            "plugin":{
               "name":"File-Manipulation",
               "type":"action",
               "artifact":{
                  "name":"core-action-plugins",
                  "version":"1.4.0-SNAPSHOT",
                  "scope":"SYSTEM"
               },
               "properties":{                  "type": "SCP"

	          "source": {
                      "-host": "hostA.com",
                      "source-login": "username"
                   },
                  "destination": {
                      "-host": "hostB.com",
                      "destination-login": "username",
                      "destination-file-path": "/filepath"
                   } 
               }
            }
         },
         ...
	]	  
    }
}
SSH Script ActionTo be able to execute Perl/Python/R/Shell/Revo R/Hive Query
scripts located on the remote machine
  • hostname
  • login credentials (keyfiles?)
  • /PATH to script
  • arguments for script
{		
    ... 
    "config":{
        "connections":[
         {
            "from": "Copy-File,
            "to": *other end*,
         },
         ...
        ],
        "stages":[
         {
            "name":"CopyRun-Remote-FileScript",
            "plugin":{
               "name":"SSHShell",
               "type":"action",
               "artifact":{
                  "name":"core-action-plugins",
                  "version":"1.4.0-SNAPSHOT",
                  "scope":"SYSTEM"
               },
               "properties":{
	          "host": "scripthost.com",
                  "script-path":"/tmp/myscript",
                  "login": "username",
                  "arguments":[				{  "{\"name"\": \"timeout\", \"value\:\": 10 \"},
				{ \"name\": \"user\", \"value\": \"some_user\"}
"			      ]
               }
            }
         },
         ...
	]	  
    }
}
SQL ActionTo be able to run the SQL stored procedures located on
the remote SQL server, Copy Hive data to SQL Server table
  • username
  • database name
  • file to push
 similar to SSH properties
Email actionTo be able to send emails
  • recipient/sender email
  • message
  • subject
  • username/password auth (if needed)
  • protocol (smtp, smtps, tls)
  • smtp host
  • smtp port
{		
    ... 
    "config":{
        "connections":[
         {
            "from": "Copy-File,
            "to": *other end*,
         },
         ...
        ],
        "stages":[
         {
            "name":"Email-Bob",
            "plugin":{
               "name":"Email",
               "type":"action",
               "artifact":{
                  "name":"core-action-plugins",
                  "version":"1.4.0-SNAPSHOT",
                  "scope":"SYSTEM"
               },
               "properties":{
	          "protocol": "smtp",
                  "recipient": "bob@cask.co",
                  "subject": "PR Review",
                  "auth": {
                      "username": "username",
                      "password": "emailpass",
                  "message": "Hey Bob, could you take a look at this PR?"
               }
            }
         },
         ...
	]	  
    }
}

...