| Table of Contents | ||
|---|---|---|
|
...
Result of this query will be next:
| Text view | ||
|---|---|---|
| name | roles | title |
| "Meg Ryan" | ["DeDe", "Angelica Graynamore", "Patricia Graynamore"] | "Joe Versus the Volcano" |
| "Meg Ryan" | ["Sally Albright"] | "When Harry Met Sally" |
| "Meg Ryan" | ["Kathleen Kelly"] | "You've Got Mail" |
| "Meg Ryan" | ["Carole"] | "Top Gun" |
| "Meg Ryan" | ["Annie Reed"] | "Sleepless in Seattle" |
Source Splitter
The proposal is to add "Splits Number" Source configuration property, which allows specifying the desired number of splits to divide the query into when reading from Neo4j.
Fewer splits may be created if the query cannot be divided into the desired number of splits.
Also, we can use '0' as the default value for this configuration property and determine the number of splits according to the number of map tasks (controlled by the "mapreduce.job.maps" property):
...
| Section | User Facing Name | Widget Type | Description | Constraints |
|---|---|---|---|---|
| General | Label | textbox | Label for UI. | |
| Reference Name | textbox | Uniquely identified name for lineage. | Required | |
| Neo4j Host | textbox | Neo4j database host. | Required | |
| Neo4j Port | number | Neo4j database port. | Required | |
| Input Query | textbox | The query to use to import data from the Neo4j database. | Required | |
| Credentials | Username | textbox | User identity for connecting to the Neo4j. | Required |
| Password | password | Password to use to connect to the Neo4j. | Required | |
| Advanced | Splits Number | number | The number of splits to generate. If set to one, the orderBy is not needed. | |
| Order By | textbox | Field Name which will be used for ordering during splits generation. This is required unless numSplits is set to one. |
...
| Section | User Facing Name | Widget Type | Description | Constraints |
|---|---|---|---|---|
| General | Label | textbox | Label for UI. | |
| Reference Name | textbox | Uniquely identified name for lineage. | Required | |
| Neo4j Host | textbox | Neo4j database host. | Required | |
| Neo4j Port | number | Neo4j database port. | Required | |
| Output Query | textbox | The query to use to export data to the Neo4j database. | Required | |
| Credentials | Username | textbox | User identity for connecting to the Neo4j. | Required |
| Password | password | Password to use to connect to the Neo4j. | Required |
...
To solve these problems, the following solution was proposed:
Using next structure $(...) for identify place where properties will be inserted.
Example of using $(...):
List of output fields: ["name", "age", "profesion", "company", "rating", "position"]
| Output query | Expected results |
|---|---|
| CREATE (n:Node $(*)) | Will be created node with label Node and properties ["name", "age", "profesion", "company", "rating", "position"] |
| CREATE (p:Person $(name, age, profesion)), (c:Company $(company, rating)) | Will be created node with label Person and properties ["name", "age", "profesion"] Will be created node with label Companyand properties ["company", "rating"] |
| CREATE (p:Person $(name, profesion))-[r:WorkOn $(position)]->(c:Company $(company)) | Will be created node with label Person and properties ["name", "profesion"] Will be created relation with type WorkOn and properties ["position"] Will be created node with label Companyand properties ["company"] |
Sink Data Types Mapping
| CDAP Schema Data Types | Neo4j Data Types |
|---|---|
| null | null |
| array | List |
| boolean | Boolean |
| long | Integer |
| double | Float |
| string | String |
| bytes | ByteArray |
| date | Date |
| time-micros | Time |
| timestamp-micros | DateTime |
| Duration | |
| Point |
...