...
Every Cloud Data Fusion pipeline run is assigned a unique RunID. After you deploy and run your pipeline, find the RunID of the pipeline run you would like to view logs for. In Stackdriver, view the logs for the pipeline run with that RunID.
...
Getting your pipeline RunID
After your pipeline has successfully run, click Summary.
In the Summary page, click the Table link.
Click on the
RunIdlink to view and copy to clipboard.
Viewing
...
logs
In the Stackdriver UI, in the Logs View page, select Cloud Dataproc Cluster → cdap-<pipeline-name>-<runId>.
...
Filtering logs
You can use the filter options filter to filter the logs you see. You can filter by component such as datafusion-pipeline-logs or yarn-resourcemanager logs, or you can filter by various log levels. Use the dropdown menu to choose a filter.
...
Downloading logs
You can download logs from Stackdriver. Click the Download logs option at the top.
...