Skip to main content

Mailchimp Connector (Technical Reference)

Written by Philippe Trussart

Overview

The Mailchimp connector syncs lists, members, segments, campaigns, unsubscribes, and per-recipient email activity from the Mailchimp v3 API into your Bytespree Data Lake. It supports incremental sync where possible, resilient pagination, rate-limit handling, retries, and emits per-stream metrics and state for reliable resumability.

Setup

Required settings

  • token – Mailchimp API key (used as Authorization: Bearer <token>).

  • dataCenter – Data center prefix (e.g., us14, us21).
    These two settings are read on test, discovery, and sync runs. The connector validates access by calling /lists.

What it syncs (streams)

Stream

Primary key(s)

Incremental filter / ordering

Page size

lists

id

since_date_created (created)

1000

list_members

id, list_id, unique_email_id, contact_id

since_last_changed (modified)

1000

list_segments

id, list_id

since_updated_at (modified)

200

list_segment_members

id, list_id, unique_email_id

No date filter (full scan by segment)

1000

campaigns

id, parent_campaign_id

since_create_time (created)

1000

campaign_unsubscribes

(campaign_id,list_id,email_id)

No date filter (full scan by campaign)

1000

campaign_email_activity

(campaign_id,list_id,email_id)

No date filter (full scan by campaign)

1000

Notes:

  • campaign_unsubscribes explicitly disables unique-key enforcement to allow duplicates from the source.

  • For the streams with “No date filter”, the connector enumerates all campaigns/lists first, then walks each child endpoint with paging.

Replication & state

  • The connector maintains a per-stream bookmark named last_started.

  • For each stream run, it sets start_date = last_started − 5 minutes to provide a safety overlap, then queries Mailchimp using the corresponding since_* parameter where available.

  • After a stream finishes, it writes a fresh last_started timestamp (ISO-8601) for that stream.

Deletions

  • Soft deletes for lists only: when syncing lists, the connector emits soft-delete events for the current batch of ids before upserting the fresh records. This lets downstream merges replace prior rows for those same primary keys.

  • Other streams do not emit delete events.

Error handling, retries, and rate limits

  • HTTP 429 (rate limit): the connector logs a slowdown message, sleeps 90 seconds, then retries the same request.

  • Automatic retries: failed requests are retried up to 10 times with a 15-second delay between attempts.

  • HTTP 400 with a structured body is logged (detail included) and treated as an empty page; unexpected HTTP codes raise a friendly exception.

Pagination

  • Uses Mailchimp’s count and offset parameters.

  • Typical page sizes: 1000 (lists, list members, campaigns, segment members, email activity, unsubscribes) and 200 (list segments).

  • For endpoints that don’t support since_*, the connector first gathers all parent IDs (lists or campaigns) and then iterates children with offsets until no more results are returned.

Metrics

After each stream completes, the connector emits a record_count metric with the total number of rows processed for that stream.

Known limitations & notes

  • Unsubscribes & email activity don’t support since_* filters; the connector walks all campaigns and paginates each report endpoint.

  • Segment members don’t support a since_* filter; the connector walks all lists → segments → members.

  • Soft deletes occur only in the lists stream (per batch of IDs).

  • Timestamps are normalized to ISO-8601; if parsing fails, the field is set to null.

Troubleshooting tips

  • HTTP 429: expect a 90-second pause before retry.

  • Persistent 400s: the connector logs Mailchimp’s detail message alongside the request URL for faster debugging.

  • Large accounts: the “full scan” streams can take longer because they iterate every campaign/list hierarchy with pages of 1000 items. Consider narrowing the active catalog to just the streams you need.

Did this answer your question?