Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. User should be able to run a workflow to process new partitions of a PFS.
  2. User should be able to limit the number of partitions I process in each run of the workflow.
  3. User should be able to apply a PartitionFilter over the partitions to be processed by a workflow.
  4. User should be able to run multiple instances of the workflow, each processing its own set of partitions.
  5. User should be able to retry the processing of a partition a configurable number of times, if there is a failure to process it.
  6. User should be able to start or end the incremental processing at a particular start/end timestamp.
  7. User should be able to execute queries to query for new partitions outside of a workflow/MapReduce (for instance, from a Worker).

...