Run Data Hub-4 Harmonize Flow
This demo template demonstrates how to use EvaluateCollector Processor with DHF to call the DHF collector module. This template can be used to call the DHF Harmonization flow independently.
The example DHF Tutorial can be found at: Data Hub Framework Tutorial 4.X This template will follow the Product Entity example from DHF 4.X tutorial.
You can download the NiFi template here.
DHF version : 4.3.1
NAR version : MarkLogic NiFi Incubator - TBD
Apache NiFi version : 1.8.0
Processors
EvaluateCollectorMarkLogic
Evaluate collector module of DGF Harmonize flow
Properties
- Database Client Service
- (MLStagingService, pointing to data-hub-STAGING database and the corresponding HTTP port)
- Entity Name
- Product
- Flow Name
- Harmonize Products
- Options
- {"entity":"Product", "flow":"Harmonize Products","flowType":"harmonize"}
- Batch Size
- 100
- Identifiers Attribute Name
- dhf.collector.identifiers
Relationships
- batches
- FlowFiles created from each batch of collected identifiers are routed her
ExtensionCallMarkLogic
Properties
- Database Client Service
- (MLStagingService, pointing to data-hub-STAGING database and the corresponding HTTP port)
- Extension Name
- ml:sjsFlow
- Requires Input
- true
- param:entity-name
- Product
- param:flow-name
- Harmonize Products
- param:identifiers
- ${dhf.collector.identifiers}
- param:job-id
- 12345678
- param:target-database
- data-hub-FINAL
- separator:param:identifiers
- ,
Run Processors
Select each of the processors and start them.
Template
