IMPORTANT: This documentation has been discontinued. Read the updated Stream File Reader documentation on our new documentation portal.
Stream File Reader reads a local file in a JSON structure, that currently supports CSV only, and triggers subpipelines to process each message. It should be used for heavy files.
Take a look at the configuration parameters of the component:
File Name: name of the local file.
Charset: name of the characters code for the default file reading (default UTF-8).
Element Identifier: attribute to be sent in case of errors.
Paralell Execution Of Each Iteration: occurs in parallel with the loop execution.
Ignore Invalid Charset: if the option is activated, the invalid charset configured in the component will be ignored along with the received file.
Fail On Error: if the option is enabled, the execution of the pipeline with error will be interrupted; otherwise, the pipeline execution proceeds, but the result will show a false value for the “success” property.
Advanced: definition of advanced parameters.
Skip: number of lines to be skipped before the file reading.
Limit: maximum number of lines to be read.
Messages flow
Input
The component waits for a message in the following format:
{
"filename": "fileName"
}
Local File Name overrides the default local file.
Output
{
"total": 0,
"success": 0,
"failed": 0
}
total: total number of processed lines
success: total number of lines successfully processed
failed: total number of line whose processing failed
IMPORTANT: to know if a line has been correctly processed, there must be the return { "success": true } for every processed line.
The component throws an exception if the File Name doesn't exist or can't be read.
The files manipulation inside a pipeline occurs in a protected way. All the files can be accessed with a temporary directory only, where each pipeline key gives access to its own files set.
This component makes batch processing. To better understand the concept, click here.