Introduction
...
The following example uses '{ status: { $in: [ "A", "D" ] } }' query filter document to retrieve all documents from the 'inventory' collection where 'status' equals either "A" or "D":
...
The following MongoDB data types are missing: Undefined, Regular Expression, DBPointer, JavaScript, Symbol, JavaScript (with scope), Timestamp, Min key, Max key.
| CDAP Schema Data Type | MongoDB Data Types | ||
|---|---|---|---|
| boolean | Boolean | ||
| bytes | Binary data, ObjectId(if 'ID Field' specified) | ||
| date | Date | ||
| double | Double | ||
| decimal | Decimal128 | ||
| float | Double | ||
| int | 32-bit integer | ||
| long | 64-bit integer | ||
| string | String, ObjectId(if 'ID Field' specified) | ||
| time | String | ||
| timestamp | Date | ||
| array | Array | ||
| record | Object | ||
| enum | Array | ||
| map | Object | ||
| union | Depends on the actual value. For example, if it's a union:
and the value is actually a long, the mongo document will have the field as a 64-bit integer. If a different record comes in with the value as a string, the mongo document will end up with a String for that field. |
...
| User Facing Name | Widget Type | Description | Constraints |
|---|---|---|---|
| Label | textbox | Label for UI. | |
| Reference Name | textbox | Uniquely identified name for lineage. | |
| Host | textbox | Host that MongoDB is running on. | Required (defaults to localhost on UI) |
| Port | number | Port that MongoDB is listening to. | Optional (default 27017) |
| Database | textbox | MongoDB database name. | Required |
| Collection | textbox | Name of the database collection to write to. | Required |
| Output Schema | schema | Specifies the schema of the documents. | Required |
| On Record Error | selectradio-group | Specifies how to handle error in record processing. An error will be thrown if failed to parse value according to a provided schema. | Possible values are:
Default: 'Fail pipeline' |
| Input Query | json-editor | Optionally filter the input collection with a query. This query must be represented in JSON format and use the MongoDB extended JSON format to represent non-native JSON data types. | |
| Username | textbox | User identity for connecting to the specified database. | |
| Password | password | Password to use to connect to the specified database. | |
| Authentication Connection String | textbox | Auxiliary MongoDB connection string to authenticate against when constructing splits. | |
| Connection Arguments | keyvalue | A list of arbitrary string key/value pairs as connection arguments. See Connection String Options for a full description of these arguments. |
...
The source requires Output Schema to be set. Based on the schema source will expect a field in each document to be of a specific Mongo data type.
On Record Error error handling property allows the user to decide whether the pipeline should fail, the record should be skipped, or the record should be sent to the error dataset.
...