GetFTP
Description
Fetches files from an FTP Server and creates FlowFiles from them
Tags
FTP, fetch, files, get, ingest, input, remote, retrieve, source
Properties
In the list below required Properties are shown with an asterisk (*). Other properties are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
| Display Name | API Name | Default Value | Allowable Values | Description |
|---|---|---|---|---|
| Hostname * | Hostname | The fully qualified hostname or IP address of the remote system Supports Expression Language, using FlowFile attributes and Environment variables. | ||
| Port * | Port | 21 | The port that the remote system is listening on for file transfers Supports Expression Language, using FlowFile attributes and Environment variables. | |
| Username * | Username | Username Supports Expression Language, using FlowFile attributes and Environment variables. | ||
| Password | Password | Password for the user account Supports Expression Language, using FlowFile attributes and Environment variables. | ||
| Connection Mode | Connection Mode | Passive |
| The FTP Connection Mode |
| Transfer Mode | Transfer Mode | Binary |
| The FTP Transfer Mode |
| Remote Path | Remote Path | The path on the remote system from which to pull or push files Supports Expression Language, using FlowFile attributes and Environment variables. | ||
| File Filter Regex | File Filter Regex | Provides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetched | ||
| Path Filter Regex | Path Filter Regex | When Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scanned | ||
| Polling Interval * | Polling Interval | 60 sec | Determines how long to wait between fetching the listing for new files | |
| Search Recursively * | Search Recursively | false |
| If true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectories |
| Follow symlink * | follow-symlink | false |
| If true, will pull even symbolic files and also nested symbolic subdirectories; otherwise, will not read symbolic files and will not traverse symbolic link subdirectories |
| Ignore Dotted Files * | Ignore Dotted Files | true |
| If true, files whose names begin with a dot (".") will be ignored |
| Delete Original * | Delete Original | true |
| Determines whether or not the file is deleted from the remote system after it has been successfully transferred |
| Connection Timeout * | Connection Timeout | 30 sec | Amount of time to wait before timing out while creating a connection | |
| Data Timeout * | Data Timeout | 30 sec | When transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems | |
| Max Selects * | Max Selects | 100 | The maximum number of files to pull in a single connection | |
| Remote Poll Batch Size * | Remote Poll Batch Size | 5000 | The value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal. | |
| Use Natural Ordering * | Use Natural Ordering | false |
| If true, will pull files in the order in which they are naturally listed; otherwise, the order in which the files will be pulled is not defined |
| Proxy Configuration Service | proxy-configuration-service | Controller Service: ProxyConfigurationService Implementations: StandardProxyConfigurationService | Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthN, SOCKS + AuthN | |
| Internal Buffer Size | Internal Buffer Size | 16KB | Set the internal buffer size for buffered data streams | |
| Use UTF-8 Encoding * | ftp-use-utf8 | false |
| Tells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding. |
Dynamic Properties
This component does not support dynamic properties.
Relationships
| Name | Description |
|---|---|
| success | All FlowFiles that are received are routed to success |
Reads Attributes
This processor does not read attributes.
Writes Attributes
| Name | Description |
|---|---|
| absolute.path | The full/absolute path from where a file was picked up. The current 'path' attribute is still populated, but may be a relative path |
| file.group | The numeric group id of the source file |
| file.lastAccessTime | The date and time that the file was last accessed. May not work on all file systems |
| file.lastModifiedTime | The date and time that the source file was last modified |
| file.owner | The numeric owner id of the source file |
| file.permissions | The read/write/execute permissions of the source file |
| filename | The filename is set to the name of the file on the remote server |
| path | The path is set to the path of the file's directory on the remote server. For example, if the <Remote Path> property is set to /tmp, files picked up from /tmp will have the path attribute set to /tmp. If the <Search Recursively> property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to /tmp/abc/1/2/3 |
State Management
This component does not store state.
Restricted
This component is not restricted.
Input Requirement
This component does not allow an incoming relationship.
System Resource Considerations
This component does not specify system resource considerations.