Page Comparison

...

1. Hadoop KMS supports versioning for the keys it stores. This is used mainly for key rollovers. In this versionrelease, we won't support versioning.

...

Following operations will supported by the store

• Store
• Get data
• Get metadata
• Get metadata list
• List
• Delete

The system will expose APIs to clients

...

// Represents the meta datametadata about the data
interface SecureStoreMetaData {
  String getName();
  String getDescription();
  long getLastModifiedTime();
  Map<String, String> getProperties();
}
 
// Represents the secure data
interface SecureStoreData {
  // Returns the meta data about the secure data
  SecureStoreMetaData getMetaData();
 
  // Returns the secure data
  byte[] get();
}
 
// Provides read-only access to secure store
interface SecureStore {
  // Returns a list of available secure data in the secure store.
  List<String> list();
  // Returns a list of metadata objects for the list of data items
  List<SecureStoreMetaData> getMetadata(List<String> data);
 
  // Gets the secure data
  SecureStoreData get(String name);
}
 
// Manager interface for managing secure data
interface SecureStoreManager {
  // Stores the secure data
  void put(String name, byte[] data, Map<String, String> properties);
 
  // Remove the secure data
  void delete(String name);
}

...

All access to the secure store will be logged. 

Audit logs are aggregated by KMS for API accesses to the GET_KEY_VERSION, GET_CURRENT_KEY, DECRYPT_EEK, GENERATE_EEK operations.

Entries are grouped by the (user,key,operation) combined key for a configurable aggregation interval after which the number of accesses to the specified end-point by the user for a given key is flushed to the audit log.

Implementation

Following two implementations will be provided

...

An implementation using standard Java tools (JKS or JCEKS) will be provided. The key secure store will be stored kept in a an encrypted file on the local filesystem.

Distributed mode

The cluster has KMS running

If the cluster has KMS running, we will utilize that for securely storing sensitive information. To do that we will implement the Hadoop KeyProvider API and forward user calls to KMS. The API with the methods that need to be implemented are listed below.

The cluster does not have KMS running

This mode will not be supported in this release.

//Implementation needs to be thread safe
public class JavaSecureStoreProvider extends KeyProvider {
  private JavaSecureStoreProvider(URI uri, Configuration conf) throws IOException {
    //Get the file path for local storage
    //Get the password for the secure store
    //Load or create the store
  }
 
  //Since we are not supporting versioning, the KeyVersion will always be current
  public KeyVersion getKeyVersion(String versionName) throws IOException {
  }
 
  //Lists all the keys that is accessible to this user.
  public List<String> getKeys() throws IOException {
  }
 
  //Since we are not supporting versioning, the will only have on item
  public List<KeyVersion> getKeyVersions(String name) throws IOException{
  }
 
  public Metadata getMetadata(String name) throws IOException {
  }
 
  public KeyVersion createKey(String name, byte[] material,  Options options) throws IOException {
  }
  
  public void deleteKey(String name) throws IOException {
  }
 
  //No-op for this version
  public KeyVersion rollNewVersion(String name, byte[] material) throws IOException {
  }
 
  public void flush() throws IOException{
  }
  public static class Factory extends KeyProviderFactory {
	@Override
    public KeyProvider createProvider(URI providerName,
                                      Configuration conf) throws IOException {
	}
  }
}

Distributed mode

The cluster has KMS running

If the cluster has KMS running, we will utilize that for securely storing sensitive information. To do that we will implement the Hadoop KeyProvider API and forward user calls to KMS.

The cluster does not have KMS running

...

Questions:

1. CDAP master key store

...