...
An aggregator plugin will operate over windows in the stream, and can be implemented by a call to flatMapToPair(), then reduceByKey().
Reducer
A reducer plugin will reduce all the records in a window into a single record.
| Code Block |
|---|
public interface ReduceFunction<T> {
public T reduce(T record1, T record2);
} |
This can be used to do things like calculate the min, max, average, count, etc of records in a window.
This will translate directly into a reduce(Function2<T, T, T>) call on a JavaDStream<T>.
StreamingTransform
Windowing can be implemented through a general StreamingTransform API:
...