Overview

...

• Use a Workflow Custom Action to run periodic RESTful calls to  APIs
• A spark job can read the data from filesystem and update the cube
• In order to allow different scheduling of different calls, each call will have its own workflow

UI

• UI could be deployed on a thin coopr node
  
• Probable stack for UI will be Jquery embedded in a bootsrap dashboard
  
• ChartJS and c3JS would be used to render charts
  

Trends

• UI should allow refining all metrics to different time granularities (Hourly, Daily, Weekly, Monthly, Every three months, Every year)

• Visualize metrics in the form of Dashboard (Widgets - Line, Bar, Pie, Scatter, ...)

• Dashboard and backend should support overlaying week-over-week, month-over-month or year-over-year for any metric

• Backend should  allow for raw querying of data through SQL commands

Reports

• Generate daily and weekly digest report

• Export data into PDF/Excel available for download in UI

...