Goals:

1. Improve operability in the Hydrator Studio (Improvements to logs, metrics, debuggability)
2. Improve usability in the Hydrator Studio (Redesign of bottom panel, etc)

...

• As a Hydrator/CDAP user, I want to be able to view my pipeline logs from both currently running pipelines as well as past pipelines to effectively debug the pipeline during failures
• As a Hydrator/CDAP user, I want to clearly know if the logs I am viewing are being updated live or are from a past run
• As a Hydrator/CDAP user, I want greater emphasis on the most important part of logs - the messages
• As a Hydrator user, I do not want logs to be flooded with stack traces. I want the ability to suppress them individually and as a whole
• As a Hydrator/CDAP user, I want the ability to download complete log files
  
• As a Hydrator/CDAP user, I want to view a summary of the logs I'm viewing (the number of messages, the number of errors, the number of warnings)
• As a Hydrator/CDAP user, I want to be able to filter logs by the lowest log level
• As a Hydrator/CDAP user, I want to be able to filter logs by keywords
• As a Hydrator/CDAP user, I want to be able to view a larger number of log events with a single-line summary for each, with the capability to drill down into particular events as desired
• As a Hydrator/CDAP user, I want to be able to view logs in the selected time range. I want to be able to dynamically change the time range for which I want to view logs, with context about how that time range maps to the duration of the program/service run.
• As a Hydrator/CDAP user, I want to be able to be able to maximize the log viewer to full screen size and restore it to original size as required.

   Possible solutions

• Tabular view: Columns for date, Class Name/Thread Name, Log Level, Log Line
• Alternate row background colors
• Vertically expandable with scrolling
• Searchable (Filter-able) columns
• Clear demarcation of rows
• Snippet with expand - especially for stack traces
• Picking only 1 or more log level -INFO, DEBUG, WARN, ERROR, ALL
• Ability to view and download raw logs if required
• Ability to view and expand only the "content" column of a log line

Use Case 2: Bottom Panel

Problems with current Bottom Panel

...

Design:

Proposed Log Viewer:

Composed of two main views:

1. Timeline:
   1. Starts at the program/service start time. Ends at the program/service end time (past) or current.
   2. Time range indicated by two sliders on each side. Time range can be selected by sliding these sliders.
   3. Updating slider position causes a refresh of the log viewer to show logs in the selected range with the selected filters
   4. If program/service is still running, the right/bottom end of the slider indicates current time, and if the slider is at this position, logs are updated live. The timeline keeps updating to reflect that.
   5. Sliders must not cross each other
   6. Label on the selected time range indicates the selected time range
   7. The timeline is marked with time range with granularity that depends on the duration of the log (which is the duration of the program run). 
2. Filters:
   1. Filter by lowest log level: 
      1. If ERROR is selected, then we show only ERROR
      2. If WARN is selected, then we show ERROR and WARN
      3. If INFO is selected, then we show ERROR, WARN and INFO
      4. If DEBUG is selected, then we show ERROR, WARN, INFO and DEBUG
      5. If TRACE is selected, then we show ERROR, WARN, INFO, DEBUG and TRACE
         
   2. Filter by search keywords:
      1. Search box that filters logs by the search text.
      2. This is a simple filter that applies on the message column
3. Viewing current logs along with monitoring (Live)
   1. Similar to tail -f
   2. Starts off with 50 lines
   3. Shows newer logs as they become available towards the end
   4. Users can see newer logs if they are 'scroll-positioned' at the last log line
   5. Scroll position is retained if users are at any position other than the last log line
   6. Previous button
4. Viewing logs within a specified time range (Not Live)
   1. Similar in behavior to less, so its not live, but allows the following capabilities
   2. Time range selector
   3. Previous/Next buttons
   4. Download button

...

New features

• Preview
• Log Viewer

 

Scratch Pad:

   Possible solutions

• Tabular view: Columns for date, Class Name/Thread Name, Log Level, Log Line
• Alternate row background colors
• Vertically expandable with scrolling
• Searchable (Filter-able) columns
• Clear demarcation of rows
• Snippet with expand - especially for stack traces
• Picking only 1 or more log level -INFO, DEBUG, WARN, ERROR, ALL
• Ability to view and download raw logs if required
• Ability to view and expand only the "content" column of a log line