Recovering default Dataproc compute profile in Developer Edition

Owned by Edwin Elia
Jan 15, 2021
2 min read

This document describes the steps required to recover when the default Dataproc compute profile is deleted. This scenario can happen in Developer edition when the user deletes the system preference that sets the default Dataproc profile, disable the profile, then delete the profile.

Steps

  1. Update cdap-site property to enable updating compute profile. Follow the procedure in Updating cdap-site property.

    {"options":{"profile.update.allowed": "true"}, "type": "DEVELOPER"}

  2. After restart is complete, go the CDF instance > System Admin > Configuration > Make HTTP Calls

  3. Change the method to PUT

  4. Set the path to

    profiles/dataproc

  5. Set the body to

    { "name": "dataproc", "label": "Dataproc", "description": "Creates Dataproc clusters for program runs", "provisioner": { "name": "gcp-dataproc", "properties": [ { "name": "masterNumNodes", "value": "1", "isEditable": true }, { "name": "masterCPUs", "value": "1", "isEditable": true }, { "name": "masterMemoryMB", "value": "4096", "isEditable": true }, { "name": "masterDiskGB", "value": "1000", "isEditable": true }, { "name": "workerNumNodes", "value": "2", "isEditable": true }, { "name": "workerCPUs", "value": "2", "isEditable": true }, { "name": "workerMemoryMB", "value": "8192", "isEditable": true }, { "name": "workerDiskGB", "value": "1000", "isEditable": true } ] } }

  6. Go to System Admin > Configuration > System Preferences > Edit System Preferences

  7. Add the following key and value

  8. Click Save & Close

Â