Flattening Data
Most data sources apply "speed limits" relative to how fast Flatly may read data.
Flatly must follow these speed limits and has no ability to circumvent them.
Some data sources' APIs are faster than others. You may consider asking your data source app provider (their Client Success or Support teams) to make their API faster if you find your jobs are taking too long relative to your requirements.
Tricks to Speed up your Job
Ask your data source app provider if they can raise your API rate limit.
If you have several jobs using the same credentials (API key, etc.) start them hours apart so they do not run at the same time.
Use Filters in Advanced Settings in Flatly when available (varies by source file).
Processing Data
While we are constantly improving Flatly's engine and flattening speed, both are mostly fully optimized as-is. This means that the time it takes for a job to complete (to finish flattening data) is likely the shortest in the industry.
Flatly flattens data as fast as our servers can process data -limited only by modern processor architecture and the latency involved during transactions in the cloud.
General Averages
On average, every 100K records (roughly 1M cells) take 1 minute to flatten.
For jobs over 100M cells, it is best to set your frequency to Hourly or Daily.