2-7-1 Create View, use GenerateTableFetch
In this example, I start by creating a view called employee_detail (using the SQL query on the parent page) because GenerateTableFetch can only read from a single table or view.
Download TemplateProcessors:
- GenerateTableFetch – queries for row count, then generates SQL statements with paging
- Properties
- Database Connection Pooling Service: DBCPConnectionPool
- Database Type: Generic
- Table Name: employee_detail
- Columns to Return: *
- Settings
- Automatically Terminate Relationships: failure
- Scheduling
- Run Schedule: 1 day (prevents infinitely looping)
- ExecuteSQL
- Properties
- Database Connection Pooling Service: DBCPConnectionPool
- SQL select query: (leave this blank)
- SplitAvro
- Properties
- (all default)
- Settings
- Automatically Terminate Relationships: failure, original
- ConvertAvroToJson
- Properties
- (all default)
- Settings
- Automatically Terminate Relationships: failure
- EvaluateJsonPath - Store values from JSON in FlowFile properties
- Properties
- Destination: flowfile-attribute
- emp.no: $.emp_no (custom property)
- Settings
- Automatically Terminate Relationships: failure, unmatched
- UpdateAttribute
- Properties
- marklogic.uri: /employees/${emp.no}.json
- InvokeHTTP – HTTP PUT to MarkLogic REST API /LATEST/documents
- Properties
- HTTP Method: PUT
- Remote URL: http://localhost:8000/LATEST/documents?uri=${marklogic.uri}
- Basic Authentication Username: youruser
- Basic Authentication Password: yourpassword
- Settings
- Check all five checkboxes under "Automatically Terminate Relationships"
