Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. As a CDAP/Hydrator security admin, I want all sensitive information like passwords not be stored in plaintext.

 

Scenarios

 

Brief introduction to Hadoop KMS

Hadoop KMS is a cryptographic key management server based on Hadoop’s KeyProvider API.

...

The KMS is a proxy that interfaces with a backing key store on behalf of HDFS daemons and clients. Both the backing key store and the KMS implement the Hadoop KeyProvider API. A default Java Key store is provided for testing but is not recommended for production use. Cloudera provides Navigator Key Trustee for production clusters. Hortonworks recommends using Ranger KMS.


Image Added


*Image taken from Cloudera engineering Blog


Design

 

The entity stored will be composed of three parts

  1. Alias: This will be the identifier, provided by the user, that will be used to retrieve the object.
  2. Properties: A key value map containing the properties of the object being stored.
  3. Data: The data being stored. Passed in as a byte array.

 

Design decisions:

  1. Hadoop KMS supports versioning for the keys it stores. This is used mainly for key rollovers. In this version, we won't support versioning.

 

Following operations will supported by the store

...

Code Block
languagejava
titleJavaSecureStoreProvider
//Implementation needs to be thread safe
public class JavaSecureStoreProvider extends KeyProvider {
  //Implementation needs to thread safe
  private JavaSecureStoreProvider(URI uri, Configuration conf) throws IOException {
    //Get the file path for local storage
    //Get the password for the secure store
    //Load or create the store
  }
 
  //Since we are not supporting versioning, the KeyVersion will always be current
  public KeyVersion getKeyVersion(String versionName) throws IOException {
  }
 
  //Lists all the keys that is accessible to this user.
  public List<String> getKeys() throws IOException {
  }
 
  //Since we are not supporting versioning, the will only have on item
  public List<KeyVersion> getKeyVersions(String name) throws IOException{
  }
 
  public Metadata getMetadata(String name) throws IOException {
  }
 
  public KeyVersion createKey(String name, byte[] material,  Options options) throws IOException {
  }
  
  public void deleteKey(String name) throws IOException {
  }
 
  //No-op for this version
  public abstract KeyVersion rollNewVersion(String name, byte[] material) throws IOException {
  }
 
  public abstract void flush() throws IOException{
  }
  public static class Factory extends KeyProviderFactory {
	@Override
    public KeyProvider createProvider(URI providerName,
                                      Configuration conf) throws IOException {
	}
  }
}

...