Introduction

HubSpot is a developer and marketer of software products for inbound marketing and sales. HubSpot is an inbound marketing and sales platform that helps companies attract visitors, convert leads, and close customers. Its products and services aim to provide tools for social media marketing, content management, web analytics, CRM and search engine optimization. They expose an API that allows you to build an integration to their CRM software. The plugins will allow users to pull data from HubSpot CRM, and use it in CDAP pipelines to enrich and synthesize with other data.

User Storie(s)

As a pipeline developer, I would like to pull data from an object in HubSpot in a streaming pipeline, so that I can transform, enrich and synthesize it in a data pipeline.
As a pipeline developer, I would like to post data to an object in HubSpot in a batch or streaming pipeline, so that I can update HubSpot with my enriched data.
As a pipeline developer, I would like to pull data from an object in HubSpot in a batch pipeline, so that I can transform, enrich and synthesize it in a data pipeline.
As a pipeline developer, for both batch and streaming sources, I would like the schema to be auto-populated based on the selected objects.
As a pipeline developer, for all interactions with HubSpot, I would like HubSpot's API limits to be handled by the framework, so that I do not have to worry about them. (https://developers.hubspot.com/apps/api_guidelines)

Plugin Type

Batch Source
Batch Sink
Real-time Source
Real-time Sink
Action
Post-Run Action
Aggregate
Join
Spark Model
Spark Compute

Configurables

Streaming Source

User Facing Name	Type	Description	Optional?	Default
App Id	String	OAuth2 App ID	No
API Key	String	OAuth2 API Key	No
Object(s) to pull	select	Select from Contact Lists, Contacts, Email Events, Email Subscription, Recent Campaigns, Analytics, Companies, Deals, Deal Pipelines, Marketing Email, Products, Tickets	Yes	Analytics
Pull frequency	select	15 min, 30 min, 1 hour, 4 hour, 8 hours	Yes	30 min
Number of API calls per day	number	The number of API calls to make per day, to avoid hitting HubSpot's rate limits	Yes	10000

Batch sink

User Facing Name	Type	Description	Optional?	Default
App Id	String	OAuth2 App ID	No
API Key	String	OAuth2 API Key	No
Object to post to	select	Select from Contact Lists, Contacts, Email Events, Email Subscription, Recent Campaigns, Analytics, Companies, Deals, Deal Pipelines, Marketing Email, Products, Tickets	No
Number of API calls per day	number	The number of API calls to make per day, to avoid hitting HubSpot's rate limits	Yes	10000

Batch Source

User Facing Name	Type	Description	Optional?	Default
API Key	String	OAuth2 API Key	No
Object(s) to pull	multi-select	Select from Contact Lists, Contacts, Email Events, Email Subscription, Recent Campaigns, Analytics, Companies, Deals, Deal Pipelines, Marketing Email, Products, Tickets	Yes	Analytics
Report Type	select	Analytics report target to get data for.	Yes	totals
Report Content	select	The analytics report type of content that you want to get data for. Must be one of: landing-pages - Pull data for landing pages. standard-pages - Pull data for website pages. blog-posts - Pull data for individual blog posts. listing-pages - Pull data for blog listing pages. knowledge-articles - Pull data for knowledge base articles.	Yes
Report Category	select	The analytics report category used to break down the analytics data. Must be one of: totals - Data will be the totals rolled up from sessions - Data broken down by session details sources - Data broken down by traffic source geolocation - Data broken down by geographic location utm-:utm_type - Data broken down by the standard UTM parameters. :utm_type must be one of campaigns, contents, mediums, sources, or terms (i.e. utm-campaigns).	Yes
Report Type of object:	select	The analytics report type of object that you want the analytics data for. Must be one of: event-completions - Get data for analytics events. The results are broken down by the event ID. You can get the details for the events using this endpoint. forms - Get data for your HubSpot forms. The results are broken down by form guids. You can get the details for the forms through the Forms API. pages - Get data for all URLs with data collected by HubSpot tracking code. The results are broken down by URL. social-assists - Get data for messages published through the social publishing tools. The results are broken down by the broadcastGuid of the messages. You can get the details of those messages through the Social Media API.	Yes
Time Period	select	The time period used to group the data. Must be one of: total - Data is rolled up to totals covering the specified time. daily - Grouped by day weekly - Grouped by week monthly - Grouped by month summarize/daily - Grouped by day, data is rolled up across all breakdowns summarize/weekly - Grouped by week, data is rolled up across all breakdowns summarize/monthly - Grouped by month, data is rolled up across all breakdowns NOTE: When using daily, weekly, or monthly for the time_period, you must include at least one filter	Yes	total
Start Date	String	Start date for the report data. YYYYMMDD format.	Yes
End Date	String	End date for the report data. YYYYMMDD format.	Yes
Filters	dsv	Filter the analytics report data to include only the specified breakdowns.	Yes

Design / Implementation Tips

Tip #1
Tip #2

Design

Approach(s)

Properties

Security

Limitation(s)

Future Work

Some future work – HYDRATOR-99999
Another future work – HYDRATOR-99999

Test Case(s)

Test case #1
Test case #2

Sample Pipeline

Please attach one or more sample pipeline(s) and associated data.

Pipeline #1

Pipeline #2

Table of Contents

Checklist

User stories documented
User stories reviewed
Design documented
Design reviewed
Feature merged
Examples and guides
Integration tests
Documentation for feature
Short video demonstrating the feature