Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Current »

This topic is now available in the CDAP docs. It’ll be updated here.

In some cases, you might want to disable Spark auto-caching at the Engine config level to improve pipeline performance.

For example, it is highly recommended that you disable Spark auto-caching at the Engine config level before using the Data Cacher plugin in pipelines.

To disable Spark auto-caching:

  1. In Pipeline Studio, click Configure.

  2. Click Engine config > Show Custom Config.

  3. In the Name field, enter spark.cdap.pipeline.autocache.enable and enter false as the Value.

     

  4. Click Save.

  • No labels