...
Set batch interval to a value that’s greater than the batch processing time. You can see the batch processing time in the Spark Streaming UI, as explained above.
...
Info |
---|
The pipeline should cloned before making configuration changes. Configuration changes will not take effect if you edit the configuration of a deployed pipeline. Upon restart, spark will read its configuration from checkpoint, so any config changes made in the UI will not take effect. |
Use Pub/Sub instead of GCS sink
...