...
Setting custom cluster properties
In the Cloud Data Fusion UIPipeline Studio, navigate to the detail page of the pipeline you would like to configure.
Click on the drop-down next to the Run button.
Set your desired cluster properties, prefixing all property names with system.profile.properties. For example, if you would like to set yarn:yarn.nodemanager.pmem-check-enabled to false, enter system.profile.properties.yarn:yarn.nodemanager.pmem-check-enabled as the Name and false as the Value.
...
spark:spark.task.maxFailures
mapred:mapreduce.map.maxattempts
mapred:mapreduce.reduce.maxattempts
mapred:mapreduce.task.io.sort.mb
yarn:yarn.scheduler.minimum-allocation-mb
...
| Page Properties | ||
|---|---|---|
| ||
|