CDAP - Remove Deprecated Code

Owned by Shankar Selvam
Last updated: Jan 20, 2018
3 min read

Goal 

Remove deprecated code (class, method) usage from the CDAP Repository. 

Relevant JIRA : 

Error rendering macro 'jira' : Unable to locate Jira server for this macro. It may be due to Application Link configuration.

CDAP Deprecation Checklist

Things to note while deprecating API/Method:

1) When annotation @Deprecated is added, Make sure the description it has information on which method/class to use instead of deprecated method/class.

2) Remove deprecated usage in the codebase and replace them with updated API method/class. 

3) If the deprecated method/class is used in any of the dependent repositories, make sure they are updated to use new method/class there.

4) When the deprecated method/class is removed, does it make any other class/method unused. It makes sense to deprecate those class/method as well with description on why.

5) If you are deprecating a property (cdap property), add the old deprecated property and the new property to deprecatedKeyMap and reverseDeprecatedKeyMap in co.cask.cdap.common.conf.Configuration class and only use the new property in code usages.

6) Maven build can show deprecation using the flag -Dmaven.compiler.showDeprecation=true and -Dmaven.compiler.showWarnings=true, This will show the deprecation usages as warnings.

CDAP-API Deprecation Removal

 

ClassMethodDescription
AbstractApplication
scheduleWorkflow(Schedule schedule, String workflowName)
use ScheduleCreationSpec
AbstractApplication
scheduleWorkflow(Schedule schedule, String workflowName, Map<String, String> properties)
use ScheduleCreationSpec
ApplicationConfigurer
addSchedule(Schedule schedule, SchedulableProgramType programType, String programName,
            Map<String, String> properties)
use ScheduleCreationSpec
MapReduce, Spark
beforeSubmit(MapReduceContext context)
ProgramLifecycle#initialize
MapReduce, Spark
onFinish(boolean succeeded, MapReduceContext context)
ProgramLifecycle#destroy
MapReduceTaskContext
getInputName()
getInputContext()
Plugin
Plugin(ArtifactId artifactId, PluginClass pluginClass, PluginProperties properties)
Constructor without any parent artifacts
RunConstraints
Entire class
ScheduleBuilder#withConcurrency(int)
ScheduleSpecification
Entire class
ScheduleCreationSpec
AbstractWorkflow
addAction(WorkflowAction action)
addAction(CustomAction)
AbstractWorkflowAction
Entire class
CustomAction
WorkflowActionConfigurer
Entire class
CustomActionConfigurer
WorkflowActionNode
getActionSpecification
getCustomActionSpecification
WorkflowActionSpecification
Entire class
CustomActionSpecification
WorkflowConditionConfigurer
addAction(WorkflowAction action)
addAction(CustomAction)
WorkflowConfigurer
addAction(WorkflowAction action)
addAction(CustomAction action)
WorkflowForkConfigurer
addAction(WorkflowAction action)
addAction(CustomAction action)

 

CDAP REST API Removal :

LogHandler

/namespaces/{namespace-id}/apps/{app-id}/{program-type}/{program-id}/logs


 

Overview of Deprecated usage in repository.

Package#Usage
cdap-api31
cdap-app-fabric14
cdap-app-templates16
cdap-client151
cdap-cli10
cdap-proto35
cdap-test23
cdap-unit-test20
cdap-common5

 

Details 


CDAP-Client (151)
  • Straight forward to remove most of the client api deprecations, except for schedule client and metrics client.
  • Metrics Client - Runtime metrics usage have to be updated before removal. 
  • Schedule Client - We have methods that are marked deprecated, but ScheduleInstanceConfiguration is not deprecated yet. No information on how to convert ScheduleInstanceConfiguration to ScheduleDetail ? 

CDAP-API 
  • Schedule API related deprecation in AbstractApplication and ApplicationConfigurer - Related JIRA https://issues.cask.co/browse/CDAP-12692
  • Use ProgramLifecycle Initialize and Destroy(deprecated) instead of Map-reduce beforeSubmit and onFinish. 
  • Use ProgramLifecycle Initialize and Destroy (deprecated) instead of Spark beforeSubmit and onFinish. 
  • Use ScheduleBuilder#withConcurrency instead of RunConstraints
  • Use ScheduleCreationSpec instead of ScheduleSpecification  Related JIRA : https://issues.cask.co/browse/CDAP-11575
  • Workflow Action and Related deprecation
    • Note : Workflow program is run as workflow action(wrapped) this needs to be handled appropriately. 
Questions :
  • Why do we have unused variable in IndexedTableDefinition which are deprecated ?
  • there is a TODO to remove methods “getStreams, getDatasetModules, getDatasetSpecs", However there is no information on what to use instead for their usages. 

CDAP-App-Fabric
  • ForwardingTwillPreparer deprecations can be ignored, those methods will be removed in the future from TwillPreparer API.
  • Deprecation in CustomActionExecutor, DefaultWorkflowActionConfigurer, WorkflowNodeCreator related to Workflow Action deprecation.
  • AppFabricTestbase, AppFabricClient deprecations for methods about Schedule Specification (CDAP-11575)

CDAP-App-Templates
  • use settableArguments method instead of get/set arguments
  • use addOutput() for batch sink methods
  • remove deprecated Engine class
  • constructors deprecated in ETLBatchConfig, ETLConfig (v1)

CDAP-CLI
  • Methods in classes related to Preferences (get, set, load, delete) preferences.

CDAP-Proto
  • ApplicationDetail - use getArifact deprecated getArtifactVersion

  • MetadataRecord deprecated without info on what to use instead - used at multiple places.

  • remove getId method in ApplicationRecord, DatasetRecord, ProgramLiveInfo, ProgramRecord,
  • Schedule related deprecation ScheduleDetail (toScheduleSpecs), ScheduleType, ScheduleUpdateDetail (CDAP-11575)
  • Id class has a lot of deprecations, since its used widely, will be prudent to do those deprecations later.

CDAP-Test
  • Id related deprecations
  • waitForFinish in program manager
  • schedules in workflow

CDAP-unit-test
  • Implementation of test interfaces, so similar deprecations as above. 

CDAP-common

  • Constants, KafkaConstants (simple except for one or two properties which were not deprecated properly)
  • UnauthorizedException 

CDAP-Data-fabric
  • encode/decode obsoleteAttributes methods in AbstractSnapshotCodec (note about breaking backward compatibility)
  • TransactionEdit, TransactionEditCodecs (note about will be removed in future, here for backward compat)

 

Dependent Changes

  • Other repositories have to be updated, hydrator-plugins, integration-tests, wrangler, need to be updated if they are using the deprecated API. 
  • some of the apps might need an update if they are still using the deprecated API's for workflow etc
  • cdap-docs might have to be updated as they use old API in some examples