Introduction 

CDAP pipeline is composed of various CDAP plugins. These plugins handle error situations in case of invalid inputs or configurations. While developing CDAP pipelines, pipeline developer can provide invalid plugin configurations. For example, the BigQuery sink plugin can have schema which does not match with an underlying BigQuery table. In such situations, providing clear error message is helpful to guide user in right direction. Also, when such situation happens, providing all the error messages at once is important for better user experience.

Wrangler provides interactive way for users to apply directives to the data. However, while applying these directives, user may run into error situations. For example, the input json file may be corrupted which can fail parse-as-json directive. In such error situations, user should be provided clear error message so that further actions can be taken.

Goals

Contextual error messages provides meaningful information to CDAP users which can help them move forward in error situations. However, sometimes error messages can be vague and without context which can lead to an unpleasant user experience. Providing meaningful error messages to the users is a challenge. In many error situations, for the same type of error, the error message format is different. This inconsistency can confuse the users. Having a central repository of errors would help developers reuse the same error messages as well as formulate error messages in a consistent way.

When user is encountered with an error situation, error message can point users to appropriate actions. Sometimes the remedy would require users to follow a certain steps. For example, in wrangler, when a deployed database driver does not contain the driver class, users would need to repackage the jar and redeploy the jar. In this case, having a catalog of errors would help users to quickly find the steps to follow to fix the underlying issue. One approach to solve this challenge is to surface error codes to users along with error messages. Error codes would help users search the catalog with an error code and find the remedial steps.

Goals

There are three goals which needs to be achieved to improve error handling:

Have
• Provide a guideline
on how an error message should be formulated that makes it easier for end user to interpret the error situation.
• Improve error messages in Wrangler directives.

• To fail fast, validate fields and plugin schema at the deploy time.

• Improve error messages in CDAP plugins.

User Stories

1. As a CDAP pipeline developer, if a pipeline contains plugin configurations which are invalid, I will like it to fail early with appropriate error message.

2. As an ETL engineer, if I run into error situation while applying directives, I will like to see appropriate error message which clearly indicates the error.

Guidelines for Error Messages

An error message is the text used to provide information about error situations. Poorly written error messages can increase support costs and can be a source of frustration for users. Well-written error messages are very important for better user experience. Below is the guideline on writing better error messages. 

1. Error messages should be contextual.

Contextual error messages provide specific information particular to error situation. Error messages without any context are very hard to interpret by users. Contextual information may include information such as why the error happened, what is the error value, what is the expected value, how user can fix the error etc.

For example, if there is a mismatch in data type of a field, providing more contextual information to user in error message would help user understand the problem and fix it if needed. 

Error Message:
Data type mismatch for field 'X'. 

Better Error Message:
Field 'X' is expected to be of data type 'int'. However, provided data type is 'string'.

2. Do not include implementation details in user facing error messages.

Exposing implementation details to end users can be confusing and users may not be able to take any action to solve error situation. Thats why user facing error messages should not include implementation details. Below are some of the cases where we can avoid exposing implementation details:

• Avoid using class hashes in error messages. For example:

  No Format
  Error Message: co.cask.directives.language.SetCharset@781ecbac : Invalid type 'java.lang.String' of column 'body'. Should be of type String. Better Error Message: Error executing 'set-charset' directive: The 'body' column was expected to be a byte array or ByteBuffer, but is of type 'String'.

• Avoid using exception class names in user facing error messages. For example:

  No Format
  Error Message: java.lang.IllegalArgumentException: Database driver 'cloudsql-postgresql' not found. Better Error Message: Database driver 'cloudsql-postgresql' not found. Please make sure correct database driver is deployed.

• Avoid using technical implementation details in user facing error messages. For example:

  No Format
  Error Message: Failed to configure pipeline: valueOf operation on abc failed. Better Error Message: Failed to configure pipeline: Expected type of field 'X' is either int/double but found 'abc'.

3. Error message should provide direction to user if action is needed from user.

An error message has 3 parts, problem identification, cause details if helpful, and a solution if possible. Whenever error situation occurs, users would like to fix it immediately. The error message should have enough information to guide the user in right direction.

4. Provide complete concise error message to user and avoid ambiguity.

An error message should be a complete sentence which provides clear message. User should be able to understand the problem by reading the error message. For example:

Error Message:
io.cdap.directives.transformation.Decode@c2e00f5 : Failed to decode hex value.

Better Error Message:
Error while decoding field 'X' as hex value. Please make sure the provided field is encoded as hex.

5. Always prefer error message specific to the error situation instead of generic error messages

Whenever possible, use specific error message instead of generic error message. For example:

Error Message:
Failed to decode hex value.

Better Error Message:
Error while decoding field 'X' as hex value. Please make sure the provided field is encoded as hex.

Scope

Dataprep

• Improve error messages in all Directives
  • Remove usages of object hashes in the error messages
  • Update error messages to be more contextual
  • In Http Handlers, add cause to the thrown exception for debugging so that the cause is not lost

Plugins

• Plugin Validation
  • Instrument plugins so that all the invalid config and schema fields are reported to the user at once when a plugin is validated
  • Deploy time validation for reference name
• Improve error messages in plugins (core-plugins, google-cloud plugins to start with)

Platform

• Improve user facing error messages from HttpHandlers
• When a mapreduce job/pipeline fails actual cause of error are logged under WARN level instead of ERROR log level.

• to write better error messages: General best practices on formulating a meaningful Error Message
• Add a framework to add error codes to the error messages so that users can browse the catalog to figure out remedial steps.
• Add a framework to standardized error messages and put them at central location for better visibility.

Scenarios

• Scenario 1: Errors in Wrangler

  • Scenario 1.1

Alice wants to wrangle data using CDAP's Wrangler tool. As part of that, Alice wants to connect to Database Source using Wrangler Connection. While attempting to do that, Alice is seeing a cryptic error message while testing the connection that she does not know how to resolve just by looking at the error message. In order figure out cause of the issue and recommended action, Alice will like to browse error code catalog with the error code displayed along with the error message.

• • Scenario 1.2

Alice is applying transformations to the data on the fly using CDAP's Wrangler tool. However, while applying transformations, Alice sees an error message that does not suggest recommended action to fix the issue. She wants to browse the error code catalog to figure out recommended actions to resolve the issue.

• • Scenario 1.3

Alice wants to wrangle data using CDAP's Wrangler tool. While applying transformations to the connected source data on the fly, Alice tries to parse boolean column as csv. However, Alice observes that built-in directives to parse boolean data as csv and avro returns different error messages. Alice would like to see standard error message from both the directives when column type is incompatible.

• • Scenario 1.4

Alice wants to wrangle data using CDAP's Wrangler tool. While connecting to the source Alice uses user name and password stored in secure store. However, the secure store keys are not resolved. Alice wants to look at the error code and share it with the CDAP developer to figure out the underlying root cause.

• Scenario 2: Errors in Pipeline

  • Scenario 2.1

Alice is a pipeline developer who has built a pipeline using CDAP data pipeline studio. While running the pipeline, it failed with no meaningful error message in logs.To debug this issue further, Alice wants to share error code from the logs with Bob, the data pipeline app developer to figure out what is the cause of the error.

• • Scenario 2.2

As a data pipeline developer, Alice is trying to get schema from the database source. While attempting to do that from CDAP data pipeline studio, Alice sees an error message. In order to figure out the cause and action for the error message, Alice will like to search the error code in the error catalog.

• • Scenario 2.3

As a data pipeline developer, Alice is trying to validate the pipeline. While attempting to validate it, Alice saw an error message that requires fixing the plugin property. Alice want to have an error code along with the error message so that she can search the probable cause on error catalog.

• • Scenario 2.4

As a data pipeline developer, Alice is using secure macros in the pipeline, however, the secure macros are not resolved. Alice wants to use the error code and share it with CDAP developer to figure out the root cause.

• Scenario 3: Errors in CDAP

  • Scenario 3.1

As a CDAP user, Alice wants to upload a plugin to CDAP. However, Alice is not deploy a plugin and gets an error. Alice wants to use the error code to browse the catalog to figure out the cause and recommended action.

• • Scenario 3.2

As a CDAP user, Alice wants to enable Wrangler to wrangle data. However, Wrangler service is not starting up.Alice wants to use the error code so that it can be shared with the support engineer for further debugging.

• • Scenario 3.3

As a CDAP user, Alice wants to enable Wrangler to wrangle data. However, Wrangler service is not starting up.Alice wants to use the error code so that it can be shared with CDAP developer to figure out the root cause.

Design Discussions

/**
 * Represents error codes.
 */
@Beta
public enum ErrorCode {
  // 11 prefix for directive errors
  UNSUPPORTED_ENCODING_TYPE("WRA-111001"),

  // 12 prefix for handler errors
  DRIVER_CLASS_NOT_FOUND("WRA-121001"),
  INVALID_MACRO_ERROR("WRA-121102");
  .....


  // 13 prefix for storage
  ....

  private String code;

  ErrorCode(String code) {
    this.code = code;
  }

  /**
   * Returns the code for this error.
   */
  public String getCode() {
    return this.code;
  }
}

/**
 * Error information provider.
 */
@Beta
public interface ErrorInfoProvider {

  /**
   * Returns error code associated with the error.
   * NOTE: this interface returns special error code for backwards compatibility.
   */
  default ErrorCode getErrorCode() {
    return ErrorCode.UNDEFINED_CODE;
  }

  /**
   * Returns an array of arguments to formulate error messages.
   */
  default Object[] getErrorArgs() {
    return null;
  }
}

Look up in the exception vs highest level (Handler and Wrangler Transform)

• should be done in the exception - What if for some reason resolution fails?

/**
 * Class description here.
 */
public class DirectiveLoadException extends Exception implements ErrorInfoProvider {
  private static final MessageLookup LOOKUP = new MessageLookup();
  private ErrorCode errorCode;
  private Object[] errorArgs;

  ...

  public DirectiveLoadException(ErrorCode errorCode, Object... args) {
    super(LOOKUP.get(errorCode.name(), args));
    this.errorCode = errorCode;
    this.errorArgs = args;
  }

  @Override
  public ErrorCode getErrorCode() {
    return errorCode;
  }

  @Override
  public Object[] getErrorArgs() {
    return errorArgs;
  }
}

Impact on UI

Test Scenarios

Bug Fixes 

• Jira Legacy
  server Cask Community Issue Tracker columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 45b48dee-c8d6-34f0-9990-e6367dc2fe4b key CDAP-14378

  Jira Legacy
  server Cask Community Issue Tracker columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 45b48dee-c8d6-34f0-9990-e6367dc2fe4b key CDAP-

  15499

  15040

• Jira Legacy
  server Cask Community Issue Tracker columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 45b48dee-c8d6-34f0-9990-e6367dc2fe4b key CDAP-15507

• Jira Legacy
  server Cask Community Issue Tracker columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 45b48dee-c8d6-34f0-9990-e6367dc2fe4b key CDAP-15040

• 15499

• Jira Legacy
  server Cask Community Issue Tracker columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 45b48dee-c8d6-34f0-9990-e6367dc2fe4b key CDAP-15593

• Jira Legacy
  server Cask Community Issue Tracker columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 45b48dee-c8d6-34f0-9990-e6367dc2fe4b key CDAP-14797
  Jira Legacy
  serverId server Cask Community Issue Tracker columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution 45b48dee-c8d6-34f0-9990-e6367dc2fe4b key CDAP-15563

• Jira Legacy
  server Cask Community Issue Tracker columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 45b48dee-c8d6-34f0-9990-e6367dc2fe4b key CDAP-15560

• Jira Legacy
  server Cask Community Issue Tracker columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 45b48dee-c8d6-34f0-9990-e6367dc2fe4b key CDAP-11767

• Jira Legacy
  server Cask Community Issue Tracker columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 45b48dee-c8d6-34f0-9990-e6367dc2fe4b key CDAP-15426

• Jira Legacy
  server Cask Community Issue Tracker columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 45b48dee-c8d6-34f0-9990-e6367dc2fe4b key CDAP-15581

API Changes

Plugin Validation

Plugin validation endpoint is used to surface all the stage level errors at once. To collect multiple stage validation errors from the stage, StageConfigurer, MultiInputStageConfigurer and MultiOutputStageConfigurer can be modified as below. If there are one or more errors added to stage configurer, the pipeline deployment will fail.

public interface StageConfigurer {

  /**
   * get the input schema for this stage, or null if its unknown
   *
   * @return input schema
   */
  @Nullable
  Schema getInputSchema();

  /**
   * set the output schema for this stage, or null if its unknown
   *
   * @param outputSchema output schema for this stage
   */
  void setOutputSchema(@Nullable Schema outputSchema);

  /**
   * set the error schema for this stage, or null if its unknown.
   * If no error schema is set, it will default to the input schema for the stage. Note that since source
   * plugins do not have an input schema, it will default to null for sources.
   *
   * @param errorSchema error schema for this stage
   */
  void setErrorSchema(@Nullable Schema errorSchema);

  /**
   * add errors for this stage to the configurer if pipeline stage is invalid. 
   *
   * @param error {@link InvalidStageException} when a pipeline stage is invalid for any reason.
   */
  void addStageError(InvalidStageException error);
}

Plugins can use this api as below:

@Override
public void configurePipeline(PipelineConfigurer pipelineConfigurer) {
  pipelineConfigurer.createDataset(conf.destinationFileset, FileSet.class);
  StageConfigurer stageConfigurer = pipelineConfigurer.getStageConfigurer();
  try {
    Pattern.compile(conf.filterRegex);
  } catch (Exception e) {  
    stageConfigurer.addStageError(new InvalidConfigPropertyException(e.getMessage(), "filterRegex"));
  }
  if (conf.sourceFileset.equals(conf.destinationFileset)) {
    stageConfigurer.addStageError(new InvalidStageException("source and destination filesets must be different"));
  }
}

Sources and sinks can have schema mismatch with underlying storage. New type of exception can be introduced so that invalid schema fields can be highlighted when schema mismatch occurs:

public class InvalidSchemaFieldException extends InvalidStageException {
  private final String field;

  public InvalidSchemaFieldException(String message, String field) {
    super(message);
    this.field = field;
  }

  public InvalidSchemaFieldException(String message, Throwable cause, String field) {
    super(message, cause);
    this.field = field;
  }

  public String getField() {
    return field;
  }
}

Validation error will have corresponding INVALID_SCHEMA type for UI to identify schema field errors.

public class ValidationError {
  protected final Type type;
  protected final String message;

  /**
   * Types of validation errors
   */
  public enum Type {
    ERROR,
    STAGE_ERROR,
    INVALID_FIELD,
    PLUGIN_NOT_FOUND,
    INVALID_SCHEMA
  }

  ...
}

/**
 * An error that occurred due to field schema mismatch in a specific pipeline stage.
 */
public class InvalidSchemaFieldError extends StageValidationError {
  private final String field;

  public InvalidSchemaFieldError(String stage, InvalidSchemaFieldException cause) {
    this(cause.getMessage(), stage, cause.getField());
  }

  public InvalidConfigPropertyError(String message, String stage, String field) {
    super(Type.INVALID_SCHEMA, message, stage);
    this.field = field;
  }

  public String getField() {
    return field;
  }

  @Override
  public boolean equals(Object o) {
    ....
  }

  @Override
  public int hashCode() {
    ...
  }
}

Impact on UI

UI changes will be needed for invalid schema type errors returned from validation endpoint.

Test Scenarios

Test ID

Test Description

Expected Results

Releases

Release 6.1.0

Related Work

Future work