Page Comparison

...

1. Preview entire pipeline. Read from File source and write to the database.

   Code Block
   language java
   a. "appPreviewConfig": { "realLookups": ["File"], "realExternalWrites": ["DBSink"] }   b. Same can be achieved by "appPreviewConfig": { "endStages": ["DBSink"], "realLookups": ["FTP"], "realExternalWrites": ["DBSink"] }

2. Preview pipeline with test records (in JSON format) instead of reading it from File.

   Code Block
   language java
   "appPreviewConfig": { "endStages": ["DBSink"], "realLookups": ["File"], "realExternalWrites": ["DBSink"], "outputs": { "File": { "data": [ {"offset": 1, "body": "100,bob"}, {"offset": 2, "body": "200,rob"}, {"offset": 3, "body": "300,tom"} ] } } }   Above pipeline uses test records provided in the outputs section instead of reading from the actual Files.

3. Preview pipeline without writing to the Database.

   Code Block
   language java
   "appPreviewConfig": { "endStages": ["DBSink"], "realLookups": ["File"], "outputs": { "File": { "data": {[ {"offset": 1, "body": "100,bob"}, {"offset": 2, "body": "200,rob"}, {"offset": 3, "body": "300,tom"} }] } } }   Since the realExternalWrites is not provided, app will be reconfigured to write to /dev/null

4. Preview single stage CSVParser

   Code Block
   language java
   "appPreviewConfig": { "endStages": ["CSVParser"], // Note that endStages is updated to CSVParser to just preview the single stage. "realLookups": ["File"], "outputs": { "File": { "data": [ {"offset": 1, "body": "100,bob"}, {"offset": 2, "body": "200,rob"}, {"offset": 3, "body": "300,tom"} ] } } }

...