What is Bytespree Inbound?
Bytespree Inbound is designed to simplify the process of managing flat files. Many data teams receive flat files in various formats, and handling them can be cumbersome, especially when dealing with large volumes of data. Bytespree Inbound offers the tools to map, track, validate, and load flat files directly into your data lake.
Key Benefits:
Flat File Management: Simplifies the process of handling flat files, removing the need for manual and repetitive tasks.
Data Validation: Ensures that all files uploaded to your system are validated and meet the required specifications before loading.
Seamless Uploads: Provides a smooth experience for both internal users and external customers without requiring complex APIs.
Why Bytespree Inbound?
There are a lot of connectors in our catalog, but not every CRM and marketing tool has an API. Your organization might have internal business systems that Bytespree can't automatically sync with your data lake. In such instances, you may need to manually extract data from these systems as CSV or text files (referred to as "flat files") and then upload them to Bytespree. Over time, keeping track of which files need to be loaded, who is responsible, and when they need to be uploaded can become overwhelming. That's where Bytespree Inbound comes in—making it much simpler to manage your flat file processes!
Key Features:
Upload Directly to Your Data Lake: Flat files are uploaded straight to your data lake, giving you control over where, how, and when the data is loaded.
Reusable Mapping Templates: You only need to map your data once. These templates can be reused for future uploads, saving significant time and effort.
Scheduled Upload Reminders: You can set recurring reminders to upload new data, ensuring that updates occur on schedule. This feature can also be used to remind customers who need to submit files regularly.
Support for Compressed File Formats: Bytespree Inbound supports compressed formats, making the upload process more efficient by reducing file size.
Easy Sharing with Non-Users: Share upload links with external customers or team members who don't have Bytespree accounts. They can upload their files without needing an account, and you'll be notified when new data has been submitted.
Testing and Verification: Define tests for your templates using SQL. These tests run automatically whenever files are uploaded, ensuring data integrity.
How does Bytespree Inbound work?
Head to the Inbound Tab inside Bytepree:
File Templates
File templates are a fundamental concept in Bytespree Inbound. A file template represents a specific flat file that will be maintained in your data lake.
Example: Let's say you've decided you want to maintain a list of your email campaigns in Bytespree but an API connector isn't available for your campaign tool. Your plan is to log into that tool once per week, use the tool's export feature to create a CSV file containing your campaign data, and then load it to Bytespree.
Creating a File Template:
Step 1:
Head to the "File Templates" tab in the left side navigation and click on the "Add Template" button.
Step 2:
In the General tab give a name to the template and specify how many days prior to the due date the template's reservations can be fulfilled.
Template Name: The public name of the template that will be used with reservations and files
Prior days for fulfillment: How many days prior to any reservation's due date it could be fulfilled. If a reservation is due on 01.30.2027, and the prior days to fulfillment is set to 7, a reservation will be eligible for fulfillment on 01.23.2027.
Step 3:
In the Description tab, you need to choose which Database this file will be uploaded to, as well as the destination table.
If you want to truncate the table prior to uploading the data, then check the checkbox for this option. Truncating the table means it will reset all the table's existing data and replacing it with the data in the file, while maintaining the table's structure. If we were uploading incremental data, we could uncheck this checkbox to continue adding to the data in the database.
Step 4:
Next, in the Columns tab you can map the columns in your file to the columns in the target Table.
Column name in file - The name of the column within the files that will be using this template. If a file does not have a heading row, use
column_1,column_2,column_3, etc. Inbound will understand.Column name in table - The name of the column within the target database table. If the table is to be created, column names will be normalized into an SQL friendly format.
Data type - (defaulted to Text, only visible if creating a new table) The data type to be used for the field
SQL Transformation - An optional transformation to be applied to any value inserted,
Examples:
TRIM(column_name)
UPPER(column_name)
SUBSTRING(campaign_full_name, 0, 200) (grab the first 200 characters from the CSV file when loading to the database. This demonstrates the data cleanup power Bytespree Inbound offers.)
Step 5:
In the Checks tab, you can add a check to the import. Checks provide a way of checking imported data, and making sure it meets requirements before inserting the data and marking a reservation as fulfilled.
Check Type - The type of check to use. Currently, two are supported (see below)
Check Name - The name of the check to be shown in reporting
Step 6:
On the Reminders tab, you can choose the users that will receive a notification once the Reservation is due.
Note:
Reminders will be scheduled to be sent at 00:00UTC on the day the reminder is to go out.
Reminder delivery days:
The day the reservation is available for fulfillment
10 days before the due date
5 days before the due date and every day up until the due date
The due date the reservation
Every day after the due date with a past due notice for up to 14 days.
Once you are finished click on the "Create Template" button.
Reservations:
Reservations are like tasks in a project management system. In Bytespree Inbound, you can set up a reservation for a specific template for a specific date, meaning there will be a task on your Bytespree Inbound dashboard to have a specific file loaded to Bytespree on/by a specific date.
Creating a Reservation:
Step 1:
Go to the Reservations tab to add a reservation.
Click on the "Add Reservation" button to create a new Reservation.
Step 2:
Choose the Template you defined in Step 2, and set Due Dates for this Reservation to run.
You can click on several future dates to add multiple dates for this Reservation to run and then click on the "Select" button to choose them.
Once the dates are selected it should look like this:
Editing Reservations:
Step 1:
You can always go back and edit a Reservation by clicking on the "Edit this reservation" button:
You'll be able to change the date of the reservation and the template file.
Dashboard:
In the Dashboard you'll be able to visualize all upcoming or past reservations that have been fulfilled.
Fulfilling Reservations:
Step 1:
Once the Due Date of the reservation has arrived, you can go in the Dashboard and click on it to fulfill it.
Step 2:
You will be prompted to upload your file, choosing the data delimiter, escape character, file decoding, and other options.
Delimiter:
This is what separates values (or columns) within your data file. In a typical CSV, this is a comma (,). In pipe delimited files, you'd want to use a pipe (|). For tab delimited files, you'd want to use "\t". Delimiters can only be one character, with the exception of tab ("\t").
Escape Character:
The escape character is typically a backslash (\) in most files. Some files may not have an escape character. This field can be blank.
Enclosing Character:
The enclosing character is typically a double quote (") in most files. Some files may not have an enclosing character. This field can be blank.
File Encoding:
Inbound only supports the loading of UTF-8 and LATIN-1 encoded files. Most commonly, UTF-8 is used.
First Row Has Headers:
This box should remain checked if your data file has column headers in its first row. If your file does not have column headers, uncheck this box.
When a data file does not have column headers, data will be imported in ascending numeric order with an assumed name, in the format of column_1, column_2, column_3, etc.
Example:
1,john,doe
2,jane,doe
3,jack,doe
4.jan,doe
would result in:
column_1,column_2,column_3
1,john,doe
2,jane,doe
3,jack,doe
4.jan,doe
during import.
Ignore Empty Lines:
Data files aren't always perfect. You can let Inbound know to ignore any empty lines it encounters by checking this setting.
Step 3:
Once uploaded you will get a preview of the data to confirm it looks ok before submitting it. It's best to look over the data and make sure it is as you'd expect.
If the file and its import settings don't seem to match what Inbound expects, Inbound will show a warning, asking you to be certain the data looks to be correct.
If satisfied with the preview data, click Submit and Inbound will begin processing the data file in the background, taking you to the File screen.
If you see a problem with the data, click Cancel and you'll be taken back to the fulfillment form where you can change your file or adjust your import settings.
Click on Submit and your data will begin to process. You will be taken to a summary of your reservation that includes useful information such as status, record count, files size, error messages, etc.
Files Tab:
All the files that have been uploaded through the Reservations you will be able to see in the Files Tab, as well as their status, uploader, upload date, and number of records.
