ServiceNow Batch Source
- Prateek Duble
- Shiva Gokaram
- Akhilesh Trivedi
Introduction
ServiceNow provides service management software as a service. It specializes in IT services management, IT operations management and IT business management.
Company offerings are centered around the creation of a “service model” that corrects the root cause of service issues and enables self-service. ServiceNow's tasks, activities and processes occur as cloud services, overseen as part of a comprehensive managed workflow that supports real-time communication, collaboration and resource sharing. ServiceNow has service management offerings for IT, human resources, security, customer service, software development, facilities, field service, marketing, finance and legal enterprise needs.
Use case(s)
- Read and replicate all of ServiceNow's data tables (e.g. Contract Management, Product Catalog, Procurement etc.). This would allow users to extract and load all of their ServiceNow data into specified multi-table or multi-file sink.
- Retrieve data from one of ServiceNow's data tables, allowing users to transform and enrich the data before potentially joining it with other datasets.
User Storie(s)
- As a data pipeline developer, I should be able to import all of ServiceNow data tables so that I can analyze ServiceNow data
- As a data pipeline developer, I should be able to specify ServiceNow table name (e.g. ast_service, proc_po, proc_po_lineitem) for which I can retrieve and transform the data
- As a data pipeline developer, I should be able to specify Client ID and Client secret that would allow me to extract the table records from ServiceNow
- As a data pipeline developer, I should be able to see any errors from ServiceNow REST API calls so that I can resolve those issues
Plugin Type
- Batch Source
- Batch Sink
- Real-time Source
- Real-time Sink
- Action
- Post-Run Action
- Aggregate
- Join
- Spark Model
- Spark Compute
Configuration
ServiceNow batch source plugin will support two different modes:
- ServiceNow Reporting mode - This mode would get all the ServiceNow's data tables for a specific process/application (e.g. Contract Management, Product Catalog, Procurement etc.)
- ServiceNow Table mode - This plugin would get only the data associated with specified entity (e.g. ast_service, proc_po, proc_po_lineitem)
ServiceNow Reporting plugin
This plugin should be used when the user would like to retrieve all of the ServiceNow data tables associated with Leads and Activities entities:
| User Facing Name | Type | Description | Default value | Notes |
|---|---|---|---|---|
| Client ID | string | ServiceNow Client ID | ||
| Client Secret | string | ServiceNow Client secret | ||
| Application Name | drop-down | ServiceNow Application Name | None | Values: Contract Management, Product Catalog, Procurement etc. |
| REST API Endpoint | string | REST API endpoint URL | ||
| User name | string | ServiceNow user name | ||
| Password | string | ServiceNow password | ||
| Start date | string | Start date for reports data | ||
| End date | string | End date for reports data | ||
| Type of Value | drop-down | The type of values to be returned. The type can be one of two values: | Actual | Values: Actual , Display |
ServiceNow Table plugin
This plugin should be used when the user would like to extract the data associated with a single ServiceNow table (e.g. ast_service)
| User Facing Name | Type | Description | Default value | Notes |
|---|---|---|---|---|
| Table Name | string | ServiceNow table name to fetch | Tables from Contract Management, Product Catalog, Procurement, Financial Management etc. | |
| Client ID | string | ServiceNow Client ID | ||
| Client Secret | string | ServiceNow Client secret | ||
| REST API Endpoint | string | REST API endpoint URL | ||
| User name | string | ServiceNow user name | ||
| Password | string | ServiceNow password | ||
| Start date | string | Start date for the report | ||
| End date | string | End date for the report | ||
| Type of Value | drop-down | The type of values to be returned. The type can be one of two values: | Actual | Values: Actual , Display |
Design / Implementation Tips
- None yet
Approach(s)
Properties
Security
Limitation(s)
Future Work
Test Case(s)
- Test case #1 - TBD
- Test case #2 - TBD
References
- ServiceNow Developer site: https://developer.servicenow.com/app.do#!/home
- Table REST API: https://developer.servicenow.com/app.do#!/rest_api_doc?v=newyork&id=c_TableAPI
- REST API Explorer: https://docs.servicenow.com/bundle/newyork-application-development/page/integrate/inbound-rest/concept/use-REST-API-Explorer.html#use-REST-API-Explorer