This page describes how to configure a pipeline to run on a Dataproc cluster in a specific network. This can be done by setting the network using runtime arguments, or creating a custom profile, or by setting the network using arguments..
Setting network using runtime arguments
In the Cloud Data Fusion UI, navigate to the detail page for the pipeline you would like to run.
Click on the drop-down next to the Run button.
Enter the system.profile.properties.network as the Name, and the name of your desired network as the Value.
...
Creating a custom profile
In the Cloud Data Fusion UI, navigate to the System Admin page using the top nav bar.
Click on the Configuration tab, then click on Create New Profile in the System Compute Profiles section.
Click on Google Cloud Dataproc.
Enter a Profile label and a Description.
Enter the Network that you would like your pipelines to run in.
Click on the Create at the bottom of the page. The other settings will be automatically detected.
Navigate to the pipeline detail page for the pipeline you would like to run.
Click on the Configure button, select the profile you just created, then save.
Setting network using arguments
In the Cloud Data Fusion UI, navigate to the detail page for the pipeline you would like to run.
Click on the drop-down next to the Run button.
Enter the system.profile.properties.network as the Name, and the name of your desired network as the Value.
...
Related articles
| Page Properties | ||
|---|---|---|
| ||
|