Please read HTTP Batch Source first to grab the core principals of pagination, formats parsing etc.
...
| Code Block |
|---|
class HttpInputDStream extends ... {
private String lastProcessedPageUrl;
private String lastProcessedPageHash; #// needed when config.trackLastPageChanges is true
} |
...