Digibee Storage gets connected to the Digibee Platform storage, enabling the following operations with files: List, Download, Upload and Delete.

Take a look at the configuration parameters of the component:

  • Operation: operation to be executed (Upload, Download, List or Delete).

  • File Name: name of the file or full file path to the local file, available only for the Download and Upload operations. This parameter supports Double Braces.

  • Remote File Name: name of the remote file or relative path (eg.: tmp/file.txt) to the remote file. This parameter supports Double Braces and is presented in the Download, Upload and Delete operations.

  • Page Size: size of the page, which means, the amount of items to be returned when the List operation is used. If the value isn’t specified, all the items will be returned. If there’re more items than the amount specified in this parameter, it’s possible to request a second page (see Page Token), which returns all the remaining items.

  • Page Token: token used to request the following page when the List operation is executed. This following page will return the amount of items defined in the Page Size parameter.

  • Remote Directory: base remote directory in which the selected operation will be executed. This parameter supports Double Braces and is presented in the List, Download, Upload and Delete operations.

  • Generate Download Link: if the option is enabled, a public link for the file download is generated. This parameter only applies to the Upload operation.

  • Link Expiration: time for the link expiration on milliseconds. This parameter is valid only if the Generate Download Link option is enabled.

  • Fail On Error: if the option is enabled, the execution of the pipeline with error will be interrupted; otherwise, the pipeline execution continues, but the result will show a false value for the “success” property".

Messages flow

Input

No specific input message is expected, but the filling of the mandatory parameters that vary according to the selected operation. See below the mandatory parameters for each operation:

  • List: all the parameters are optional

  • Download: File Name and Remote File Name

  • Upload: File Name and Remote File Name

  • Delete: Remote File Name

When it comes to the Update operation, the file informed in the File Name parameter must be in the pipeline local directory.

Output

When executing the component by using the list operation, the following JSON structure is generated:

{
"content": [
{
"name": "temp//tmp/processed/file-3.txt",
"contentType": "application/octet-stream",
"contentEncoding": null,
"createTime": 1581689153448,
"bucket": "test-bucket",
"size": 10
},
{ ... }
],
"pageToken": "XXXXXXXXXXXXL3RtcC9wcm9jZXNzZWQvZmlsZS0zLnR4dA==",
"fileName": null,
"remoteFileName": null,
"remoteDirectory": "",
"success": true
}

  • content: an object array for each found file

    • name: name of the remote file

    • contentType: MIME type of the file

    • contentEncoding: file encoding, if it exists

    • createTime: timestamp for the file creation date

    • bucket: bucket where the file is found

    • size: size of the file in bytes

  • pageToken: token to recover the following page when the Page Size parameter is informed

  • fileName: name of the local file

  • remoteFileName: name of the remote file

  • remoteDirectory: name of the remote base folder

  • success: "true" if the last operation has been successful

  • error: appears when there’s an error and Fail on Error is "false"

When executing the component by using the Download, Upload and Delete operations, the following JSON structure is generated:

{
"fileName": "file.txt",
"remoteFileName": "tmp/iso-8859-1-test.txt",
"remoteDirectory": "",
"success": true
}

  • fileName: name of the local file

  • remoteFileName: name of the remote file

  • remoteDirectory: name of the remote base folder

  • success: "true" if the last operation has been successful

IMPORTANT: the files manipulation inside a pipeline occurs in a protected way. All the files can be accessed with a temporary directory only, where each pipeline key gives access to its own files set.

Digibee Storage in Action

  • LIST of files

Input

Parameters

- Operation: List

- Remote Directory: DGB-413

- Page Size: 2

Output

{
"content": [
{
"name": "DGB-413/",
"contentType": "text/plain",
"contentEncoding": null,
"createTime": 1552394033410,
"bucket": "digibee-test-digibee-test-bucket",
"size": 11
},
{
"name": "DGB-413/iso8859-2.txt",
"contentType": "text/plain",
"contentEncoding": null,
"createTime": 1552395963553,
"bucket": "digibee-test-digibee-test-bucket",
"size": 55
}
],
"pageToken": "ChVER0ItNDEzL2lzbzg4NTktMi50eHQ=",
"fileName": null,
"remoteFileName": null,
"remoteDirectory": "DGB-413",
"success": true
}

As it’s possible to see, the result above returns the pageToken property with a value of reference to the following page. This property is returned when the Page Size parameter is configured (in the example it’s configured with the value 2) and also when there’re more files to be listed.

  • LIST of multiple files using pagination

Input

Parameters

- Operation: List

- Remote Directory: DGB-413

- Page Size: 2

- Page Token: ChVER0ItNDEzL2lzbzg4NTktMi50eHQ=

Output

{
"content": [
{
"name": "DGB-413/utf-16-test.txt",
"contentType": "text/plain",
"contentEncoding": null,
"createTime": 1552394973030,
"bucket": "digibee-test-digibee-test-bucket",
"size": 70
},
{
"name": "DGB-413/utf-test.txt",
"contentType": "text/plain",
"contentEncoding": null,
"createTime": 1552394644597,
"bucket": "digibee-test-digibee-test-bucket",
"size": 55
}
],
"fileName": null,
"remoteFileName": null,
"remoteDirectory": "DGB-413",
"success": true
}

In the result above, the pageToken property wasn’t returned. It means there’re no more files to be listed.

  • DOWNLOAD of a file

Input

Parameters

- Operation: Download

- File Name: iso8859-2.txt

- Remote File Name: iso8859-2.txt

- Remote Directory: DGB-413

Output

{
"fileName": "iso8859-2.txt",
"remoteFileName": "iso8859-2.txt",
"remoteDirectory": "DGB-413",
"success": true
}

The file download will be made in the pipeline local directory.

  • UPLOAD of a file

Input

Local file: file.txt

Parameters

- Operation: Upload

- File Name: file.txt

- Remote File Name: file.txt

- Remote Directory: DGB-413

Output

{
"fileName": "file.txt",
"remoteFileName": "file.txt",
"remoteDirectory": "DGB-413",
"success": true
}

  • UPLOAD of a file generating a link for download

Input

Local file: file.txt

Parameters

- Operation: Upload

- File Name: file.txt

- Remote File Name: file.txt

- Remote Directory: DGB-413

- Generate A Download Link: true

- Link Expiration (in ms): 600000

Output

{
"fileName": "file.txt",
"remoteFileName": "file.txt",
"remoteDirectory": "DGB-413",
"success": true,
"urlGenerated": "<URL TO DOWNLOAD THE FILE>"
}

With the input parameters configuration above, the file will be available for download for 10 minutes (600000ms) through the link generated in the urlGenerated output property.

  • DELETE of a file

Input

Parameters

- Operation: Delete

- Remote File Name: file.txt

- Remote Directory: DGB-413

Output

{
"fileName": null,
"remoteFileName": "file.txt",
"remoteDirectory": "DGB-413",
"success": true
}
Did this answer your question?