IMPORTANT: This documentation has been discontinued. Read the updated Digibee Storage documentation on our new documentation portal.
Digibee Storage gets connected to the Digibee Platform storage, enabling the following operations with files: List, Download, Upload and Delete.
Take a look at the configuration parameters of the component:
Operation: operation to be executed (Upload, Download, List or Delete).
File Name: name of the file or full file path to the local file, available only for the Download and Upload operations. This parameter supports Double Braces.
Remote File Name: name of the remote file or relative path (eg.: tmp/file.txt) to the remote file. This parameter supports Double Braces and is presented in the Download, Upload and Delete operations.
Page Size: size of the page, which means, the amount of items to be returned when the List operation is used. If the value isn’t specified, all the items will be returned. If there’re more items than the amount specified in this parameter, it’s possible to request a second page (see Page Token), which returns all the remaining items.
Page Token: token used to request the following page when the List operation is executed. This following page will return the amount of items defined in the Page Size parameter.
Remote Directory: base remote directory in which the selected operation will be executed. This parameter supports Double Braces and is presented in the List, Download, Upload and Delete operations.
Generate Download Link: if the option is enabled, a public link for the file download is generated. This parameter only applies to the Upload operation.
Link Expiration: time for the link expiration on milliseconds. This parameter is valid only if the Generate Download Link option is enabled.
Fail On Error: if the option is enabled, the execution of the pipeline with error will be interrupted; otherwise, the pipeline execution continues, but the result will show a false value for the “success” property".
Messages flow
Input
No specific input message is expected, but the filling of the mandatory parameters that vary according to the selected operation. See below the mandatory parameters for each operation:
List: all the parameters are optional
Download: File Name and Remote File Name
Upload: File Name and Remote File Name
Delete: Remote File Name
When it comes to the Update operation, the file informed in the File Name parameter must be in the pipeline local directory.
Output
When executing the component by using the list operation, the following JSON structure is generated:
{
"content": [
{
"name": "temp//tmp/processed/file-3.txt",
"contentType": "application/octet-stream",
"contentEncoding": null,
"createTime": 1581689153448,
"bucket": "test-bucket",
"size": 10
},
{ ... }
],
"pageToken": "XXXXXXXXXXXXL3RtcC9wcm9jZXNzZWQvZmlsZS0zLnR4dA==",
"fileName": null,
"remoteFileName": null,
"remoteDirectory": "",
"success": true
}
content: an object array for each found file
name: name of the remote file
contentType: MIME type of the file
contentEncoding: file encoding, if it exists
createTime: timestamp for the file creation date
bucket: bucket where the file is found
size: size of the file in bytes
pageToken: token to recover the following page when the Page Size parameter is informed
fileName: name of the local file
remoteFileName: name of the remote file
remoteDirectory: name of the remote base folder
success: "true" if the last operation has been successful
error: appears when there’s an error and Fail on Error is "false"
When executing the component by using the Download, Upload and Delete operations, the following JSON structure is generated:
{
"fileName": "file.txt",
"remoteFileName": "tmp/iso-8859-1-test.txt",
"remoteDirectory": "",
"success": true
}
fileName: name of the local file
remoteFileName: name of the remote file
remoteDirectory: name of the remote base folder
success: "true" if the last operation has been successful
IMPORTANT: the files manipulation inside a pipeline occurs in a protected way. All the files can be accessed with a temporary directory only, where each pipeline key gives access to its own files set.
Digibee Storage in Action
LIST of files
Input
Parameters
- Operation: List
- Remote Directory: DGB-413
- Page Size: 2
Output
{
"content": [
{
"name": "DGB-413/",
"contentType": "text/plain",
"contentEncoding": null,
"createTime": 1552394033410,
"bucket": "digibee-test-digibee-test-bucket",
"size": 11
},
{
"name": "DGB-413/iso8859-2.txt",
"contentType": "text/plain",
"contentEncoding": null,
"createTime": 1552395963553,
"bucket": "digibee-test-digibee-test-bucket",
"size": 55
}
],
"pageToken": "ChVER0ItNDEzL2lzbzg4NTktMi50eHQ=",
"fileName": null,
"remoteFileName": null,
"remoteDirectory": "DGB-413",
"success": true
}
As it’s possible to see, the result above returns the pageToken property with a value of reference to the following page. This property is returned when the Page Size parameter is configured (in the example it’s configured with the value 2) and also when there’re more files to be listed.
LIST of multiple files using pagination
Input
Parameters
- Operation: List
- Remote Directory: DGB-413
- Page Size: 2
- Page Token: ChVER0ItNDEzL2lzbzg4NTktMi50eHQ=
Output
{
"content": [
{
"name": "DGB-413/utf-16-test.txt",
"contentType": "text/plain",
"contentEncoding": null,
"createTime": 1552394973030,
"bucket": "digibee-test-digibee-test-bucket",
"size": 70
},
{
"name": "DGB-413/utf-test.txt",
"contentType": "text/plain",
"contentEncoding": null,
"createTime": 1552394644597,
"bucket": "digibee-test-digibee-test-bucket",
"size": 55
}
],
"fileName": null,
"remoteFileName": null,
"remoteDirectory": "DGB-413",
"success": true
}
In the result above, the pageToken property wasn’t returned. It means there’re no more files to be listed.
DOWNLOAD of a file
Input
Parameters
- Operation: Download
- File Name: iso8859-2.txt
- Remote File Name: iso8859-2.txt
- Remote Directory: DGB-413
Output
{
"fileName": "iso8859-2.txt",
"remoteFileName": "iso8859-2.txt",
"remoteDirectory": "DGB-413",
"success": true
}
The file download will be made in the pipeline local directory.
UPLOAD of a file
Input
Local file: file.txt
Parameters
- Operation: Upload
- File Name: file.txt
- Remote File Name: file.txt
- Remote Directory: DGB-413
Output
{
"fileName": "file.txt",
"remoteFileName": "file.txt",
"remoteDirectory": "DGB-413",
"success": true
}
UPLOAD of a file generating a link for download
Input
Local file: file.txt
Parameters
- Operation: Upload
- File Name: file.txt
- Remote File Name: file.txt
- Remote Directory: DGB-413
- Generate A Download Link: true
- Link Expiration (in ms): 600000
Output
{
"fileName": "file.txt",
"remoteFileName": "file.txt",
"remoteDirectory": "DGB-413",
"success": true,
"urlGenerated": "<URL TO DOWNLOAD THE FILE>"
}
With the input parameters configuration above, the file will be available for download for 10 minutes (600000ms) through the link generated in the urlGenerated output property.
DELETE of a file
Input
Parameters
- Operation: Delete
- Remote File Name: file.txt
- Remote Directory: DGB-413
Output
{
"fileName": null,
"remoteFileName": "file.txt",
"remoteDirectory": "DGB-413",
"success": true
}