Page Comparison

...

notificationService

schedulerService

applicationLifecycleService

systemArtifactLoader

programRuntimeService

streamCoordinatorClient

programLifecycleService

pluginService

httpService

HybridDatasetFramework

Requirements:

1) Pipeline want's to read from a dataset source (or) pipeline wants to write to a dataset sink (or) transform uses a lookup table. These datasets are in CDAP Standalone space.

2) Pipeline run's records, Pipeline run metrics, program status, etc are stored in System datasets in Preview space.

3) Error dataset : Its not clear if using error dataset should cause creating an error dataset in CDAP standalone space. I feel it might not be required to created in Standalone space. In which case if its a dataset then it's the only user level dataset that has to be created in Preview space, we can say we would have an in-memory implementation for maintaining error records.

Assumptions :

1) All Datasets in System Namespace will be using the "PreviewDatasetFramework"

2) All Datasets in User's Namespaces will be using the "StandaloneDatasetFramework"

... snippet
@Nullable
@Override
public <T extends Dataset> T getDataset(Id.DatasetInstance datasetInstanceId,
                                        @Nullable Map<String, String> arguments,
                                        @Nullable ClassLoader classLoader)
  throws DatasetManagementException, IOException {
  if (datasetInstanceId.getNamespace().equals(Id.Namespace.SYSTEM)) {
    return previewDatasetFramework.getDataset(datasetInstanceId, arguments, classLoader);
  } else {
    return standaloneDatasetFramework.getDataset(datasetInstanceId, arguments, classLoader);
  }
}