PutDynamoDBRecord
Description
Inserts items into DynamoDB based on record-oriented data. The record fields are mapped into DynamoDB item fields, including partition and sort keys if set. Depending on the number of records the processor might execute the insert in multiple chunks in order to overcome DynamoDB's limitation on batch writing. This might result partially processed FlowFiles in which case the FlowFile will be transferred to the "unprocessed" relationship with the necessary attribute to retry later without duplicating the already executed inserts.
Tags
AWS, Amazon, DynamoDB, Insert, Put, Record
Properties
In the list below required Properties are shown with an asterisk (*). Other properties are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
| Display Name | API Name | Default Value | Allowable Values | Description |
|---|---|---|---|---|
| Table Name * | Table Name | The DynamoDB table name Supports Expression Language, using Environment variables. | ||
| Region * | Region | US West (Oregon) |
| |
| AWS Credentials Provider Service * | AWS Credentials Provider service | Controller Service: AWSCredentialsProviderService Implementations: AWSCredentialsProviderControllerService | The Controller Service that is used to obtain AWS credentials provider | |
| Record Reader * | record-reader | Controller Service: RecordReaderFactory Implementations: AvroReader CEFReader CSVReader ExcelReader GrokReader JsonPathReader JsonTreeReader ReaderLookup ScriptedReader Syslog5424Reader SyslogReader WindowsEventLogReader XMLReader YamlTreeReader | Specifies the Controller Service to use for parsing incoming data and determining the data's schema. | |
| Partition Key Strategy * | partition-key-strategy | Partition By Field |
| Defines the strategy the processor uses to assign partition key value to the inserted Items. |
| Partition Key Field * | partition-key-field | Defines the name of the partition key field in the DynamoDB table. Partition key is also known as hash key. Depending on the "Partition Key Strategy" the field value might come from the incoming Record or a generated one. Supports Expression Language, using Environment variables. | ||
| Partition Key Attribute * | partition-key-attribute | Specifies the FlowFile attribute that will be used as the value of the partition key when using "Partition by attribute" partition key strategy. Supports Expression Language, using Environment variables. This property is only considered if:
| ||
| Sort Key Strategy * | sort-key-strategy | None |
| Defines the strategy the processor uses to assign sort key to the inserted Items. |
| Sort Key Field * | sort-key-field | Defines the name of the sort key field in the DynamoDB table. Sort key is also known as range key. Supports Expression Language, using Environment variables. This property is only considered if:
| ||
| Communications Timeout * | Communications Timeout | 30 secs | ||
| Endpoint Override URL | Endpoint Override URL | Endpoint URL to use instead of the AWS default including scheme, host, port, and path. The AWS libraries select an endpoint URL based on the AWS region, but this property overrides the selected endpoint URL, allowing use with other S3-compatible endpoints. Supports Expression Language, using Environment variables. | ||
| Proxy Configuration Service | proxy-configuration-service | Controller Service: ProxyConfigurationService Implementations: StandardProxyConfigurationService | Specifies the Proxy Configuration Controller Service to proxy network requests. | |
| SSL Context Service | SSL Context Service | Controller Service: SSLContextService Implementations: StandardRestrictedSSLContextService StandardSSLContextService | Specifies an optional SSL Context Service that, if provided, will be used to create connections |
Dynamic Properties
This component does not support dynamic properties.
Relationships
| Name | Description |
|---|---|
| failure | FlowFiles are routed to failure relationship |
| success | FlowFiles are routed to success relationship |
| unprocessed | FlowFiles are routed to unprocessed relationship when DynamoDB is not able to process all the items in the request. Typical reasons are insufficient table throughput capacity and exceeding the maximum bytes per request. Unprocessed FlowFiles can be retried with a new request. |
Reads Attributes
| Name | Description |
|---|---|
| dynamodb.chunks.processed | Number of chunks successfully inserted into DynamoDB. If not set, it is considered as 0 |
Writes Attributes
| Name | Description |
|---|---|
| dynamodb.chunks.processed | Number of chunks successfully inserted into DynamoDB. If not set, it is considered as 0 |
| dynamodb.error.code | DynamoDB error code |
| dynamodb.error.exception.message | DynamoDB exception message |
| dynamodb.error.message | DynamoDB error message |
| dynamodb.error.request.id | DynamoDB error request id |
| dynamodb.error.retryable | DynamoDB error is retryable |
| dynamodb.error.service | DynamoDB error service |
| dynamodb.error.status.code | DynamoDB error status code |
| dynamodb.item.io.error | IO exception message on creating item |
| dynamodb.key.error.not.found | DynamoDB key not found |
| dynamodb.key.error.unprocessed | DynamoDB unprocessed keys |
| dynmodb.range.key.value.error | DynamoDB range key error |
State Management
This component does not store state.
Restricted
This component is not restricted.
Input Requirement
This component requires an incoming relationship.
System Resource Considerations
| Scope | Description |
|---|---|
| MEMORY | An instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance. |