It In some cases, you might want to disable Spark auto-caching at the Engine config level to improve pipeline performance.
For example, it is highly recommended that you disable Spark auto-caching at the Engine config level before using the Data Cacher plugin in pipelines.
...