2-5 Get Files by FTP
This example uses the GetFTP processor to get a single file from an anonymous ftp server. GetFTP has options to get all of the files in a single directory matching a file pattern, or from multiple directories matching a pattern.
Note that the default properties include Delete Original: true, which will fail if you do not have permissions to delete from the ftp site. If you set Delete Original to false, you must change the Run Schedule from the default "0 sec" to some longer duration (e.g. "1 day" as below) or GetFTP will continually pull the same file(s).
- Download Template
- Processors:
- GetFTP – gets file(s) from FTP site
- Properties
- Hostname: dbprftp.state.fl.us
- Username: anonymous
- Password: (a valid email address)
- Remote Path: /pub/llweb
- File Filter Regex: HR526-DET_Violations-D4\.xml
- Delete Original: false
- Scheduling
- Run Schedule: 1 day
- Properties
- InvokeHTTP – HTTP PUT to MarkLogic REST API /LATEST/documents
- Properties
- HTTP Method: PUT
- Remote URL: http://localhost:8000/LATEST/documents?uri=/fl/ftp/${filename}
- Basic Authentication Username: youruser
- Basic Authentication Password: yourpassword
- Settings
- Check all five checkboxes under "Automatically Terminate Relationships"
- Properties
- GetFTP – gets file(s) from FTP site
