How the Daily Data Pipeline Works
In Kapoq, your data goes through four sequential stages every day. Each stage must complete before the next one begins.
Stage 1: Overnight Data Load
The bulk of your account data begins loading shortly after midnight Pacific Time. This initial load covers the majority of your metrics — including high-level campaign performance, spend, and revenue figures. Once this stage completes, most of your data is available in the platform.
Important note on "yesterday's" data: The completeness of the previous day's data depends on the marketplace. Amazon's reporting for the most recent day can vary in completeness depending on their own processing timelines. Walmart, for example, never includes the current previous day in their overnight data delivery.
Stage 2: Tenant Completion
All accounts within your organization's tenant must finish the overnight data load before the next stage can begin. If your tenant includes multiple brands or large accounts, this means the overall timeline depends on the largest and most complex account finishing its load. One account cannot move ahead independently.
Stage 3: Bid Optimization (Bidder)
Once your entire tenant's data load is complete, your accounts enter a queue for bid optimization. The bidder analyzes your updated data and generates new bid recommendations. For larger tenants, bid optimization typically begins running around midday Central Time, though the exact start time depends on when the full tenant load finishes and where your accounts land in the processing queue.
Stage 4: Replication
After bid optimization completes for your accounts, replication runs. This is the final step that refreshes the remaining data points — specifically targets, search terms, products, and ads — and makes everything fully current in the platform.
What's Available When
Data Type | When It's Available |
Campaign performance, spend, revenue, and most metrics | Shortly after midnight PT (after overnight load) |
Targets, search terms, products, ads | After the bidder and replication complete (typically afternoon) |
Bid recommendations / suggestions | After the bidder runs (typically midday CT or later) |
Why Can't Data Be Manually Refreshed?
The processing pipeline has multiple interdependent layers. It is not possible to force a reload of a single account or a single data type on demand — the stages are sequential and tenant-wide by design to ensure data integrity and consistency across all recommendations and reporting.
What's Coming Next
We are actively working toward a more real-time data experience. As marketplace APIs expand their streaming data capabilities across all data points, we will be able to deliver significantly faster and more granular data updates — similar to the near-real-time experience you may already see in certain parts of the platform today. We will communicate updates as these enhancements become available.