Navigation

3-1 Transform JSON

This example demonstrates how to transform JSON with the built-in JoltTransformJSON processor. The example flow is based on the "Ingest Line-Delimited JSON" example.

Jolt supports a rich domain-specific transformation language for JSON. The simple Jolt specification in the example below removes a single property, "ip_address", from the JSON object.

Jolt links:
Download Template
Sample Data (generated by Mockaroo)
Processors:
  • GetFile – reads files from a watched directory
    • Properties
      • Input Directory: /some/path
  • SplitText – splits the line-delimited file into one file per line
    • Properties
      • Line Split Count: 1
  • JoltTransformJSON
    • Properties
      • Jolt Specification:
        [ { "operation": "remove", "spec": { "ip_address": "" } }]
    • Settings
      • Automatically Terminate Relationships: failure
  • EvaluateJsonPath - Store values from JSON in FlowFile properties
    • Properties
      • Destination: flowfile-attribute
      • ml.json.id: $.id (custom property)
    • Settings
      • Automatically Terminate Relationships: failure, unmatched
  • InvokeHTTP – HTTP PUT to MarkLogic REST API /LATEST/documents
    • Properties
      • HTTP Method: PUT
      • Remote URL: http://localhost:8000/LATEST/documents?uri=/json/${ml.json.id}.json
      • Basic Authentication Username: youruser
      • Basic Authentication Password: yourpassword
    • Settings
      • Check all five checkboxes under "Automatically Terminate Relationships"