Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Introduction

An option of IAM role based authentication in the existing S3 source and sink plugins.

Use case(s)

  • In the S3 source and S3 sink(Avro and Parquet) plugins,there should be a provision for user to select authentication mechanism for S3.User should have an option to select IAM role based authentication in the plugins.

User Storie(s)

  • As a pipeline user,i want to have an option of IAM role based authentication in the S3 source and sink plugins in Hydrator.

Plugin Type

  • Batch Source
  • Batch Sink 
  • Real-time Source
  • Real-time Sink
  • Action
  • Post-Run Action
  • Aggregate
  • Join
  • Spark Model
  • Spark Compute

Configurables

New Configuration would be added in the S3 plugin

User Facing NameTypeDescriptionConstraints
Authentication MethodSelect
Authentication method to access S3.
Defaults to Access Credentials
Since this is an EC2+IAM capability,user will need to have AWS environment with roles assigned to all the members.
    

Design / Implementation Tips

  • Tip #1
  • Tip #2

Design

Approach(s)

Properties

Security

Limitation(s)

Future Work

  • Some future work – HYDRATOR-99999
  • Another future work – HYDRATOR-99999

Test Case(s)

  • Test case #1
  • Test case #2

Sample Pipeline

Please attach one or more sample pipeline(s) and associated data. 

Pipeline #1

Pipeline #2

 

 

Table of Contents

Checklist

  • User stories documented 
  • User stories reviewed 
  • Design documented 
  • Design reviewed 
  • Feature merged 
  • Examples and guides 
  • Integration tests 
  • Documentation for feature 
  • Short video demonstrating the feature
  • No labels