Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Setting up permissions for Datastore or BigQuery

Data Fusion uses a service account to access GCP resources in Wrangler, preview, and for data pipelines running on Dataproc. The service account used for running services in the tenant project such as preview and Wrangler is in the following format service-<customer-project-number>@gcp-sa-datafusion.iam.gserviceaccount.com. This service account is already created when Cloud Data Fusion API is enabled on the project. Actual data pipeline execution on the Dataproc cluster happens using the compute engine default service account. Any additional GCP resources that Data Fusion needs access to should have appropriate permissions for both of these service accounts.

...

For example, to add access to Datastore from preview and Wrangler, follow the steps below.:

  1. In the GCP Console, open the IAM & Admin page.

  2. In the left bar click IAM.

  3. Edit roles for service-<some_number>@gcp-sa-datafusion.iam.gserviceaccount.com.

  4. In the Edit permissions page, add the role Cloud Datastore Owner and click on Save.

5. Perform similar steps (i.e. add the same roles) for the compute engine default service account to allow the data pipeline to access Datastore during its execution on Dataproc.

...