Virtuos CRM connector (technical documentation)

1. Overview

The Virtuous connector pulls data from the Virtuous CRM API into your data platform.

Key points:

Supports a set of query endpoints (campaigns, contacts, gifts, projects, etc.).
Infers schema dynamically by sampling the API for each table.
Handles rate limits and retries automatically.
Can optionally consume webhook events from Virtuous to reflect near-real-time changes (including deletes) for certain entities.
Does not currently support Virtuous events or form submissions.

2. Supported Data & Tables

The connector exposes the following logical tables (streams):

campaigns
channel_types
communication_types
communications
contacts
contact_notes
gifts
gift_asks
gift_destinations
individuals
organization_groups
planned_gifts
pledges
projects
recurring_gifts
volunteers
volunteer_opportunities
segments

Some tables return full records (objects with many fields). Others return a simple array of values, which the connector maps into a single-column table.

3. Configuration & Authentication

Required configuration

api_key
Virtuous API key. Used as a Basic Authorization header on all API calls:
- Header: Authorization: Basic {api_key}

To use this connector, you need a Virtuous application key. In Virtuous CRM+, application keys are created from Settings → All Settings → Connectivity → Application Keys, where you click Create an Application Key and then copy the generated key for use in the integration. This is the same Application Keys area Virtuous documents for partner/integration setup.
The connector sends this value in the Authorization header on every API request.
Practical note: Because this is an application key, access depends on whatever permissions and API access are granted to the Virtuous environment and integration behind that key. If the key is invalid or lacks access, the connector will fail authentication or receive API errors. The connector treats 401 responses as invalid credentials and surfaces a clear error.

Tables selection
In the sync/catalog configuration you choose which tables (streams) to enable. Only those tables will be queried.
Webhook configuration (optional)
For webhook support (optional, not required for standard sync):
- api_key – same as above.
- tables – list of tables whose events you want to subscribe to (e.g. ["contacts","gifts"]).
- url – the webhook receiver URL (your system endpoint).
- meta – metadata returned by Virtuous when a webhook is created (used later to delete it).

Base API URL

All requests are made against:

Base URL: https://api.virtuoussoftware.com/api/

4. Sync Behaviour

4.1 Schema discovery (per table)

For each selected table during discover:

The connector looks up the table in its internal table_map.
- If the table is not defined, the run fails with a clear error (“table not supported”).
If the table is flagged as a value array (is_value_array = true), it creates a schema with one string column (see section 7.2).
Otherwise:
- It determines the HTTP method (method in table_map, default is POST).
- It requests one record using ?skip=0&take=1.
- It inspects the first record to build the column list and infer data types.

Type inference:

For each field the connector checks its runtime type and maps it to:
- array → array
- object → object
- boolean → boolean
- integer → integer
- null → string
- string → string

Implication: field lists and even field types may change over time as Virtuous adds/removes fields. The connector always uses live API data to define the schema for each run.

4.2 Data sync (per table)

For each enabled table during tap/sync:

The connector fetches the schema as above.
It writes that schema (including any inferred types) and the table’s unique keys/indexes.
It then pages through the full dataset using skip and take parameters.
Each page of results is written as records.

Pagination strategy:

Initial state per table:
- current_record = 0
- total_records = 1 (will be updated after the first call)
For each loop:
- Call GET or POST to the table endpoint with skip={current_record} and take={results_per_query}.
- After the first call in the loop, total_records is set from the API response’s total property.
- current_record += number_of_results_returned.
- Continue until current_record >= total_records.

Default page size:

results_per_query = 250 records per request.

4.3 Data typing & JSON encoding

Before writing each record, the connector:

Ensures every expected column is present (missing fields are set to NULL).
Coerces values to the inferred type:
- Integer columns → cast to integer.
- Boolean columns → cast to boolean.
- String columns:
  - If the value is an array or object, it is JSON-encoded to a string.
  - Otherwise, it is cast to string.
Drops any keys that are not part of the inferred column list.

This guarantees a consistent column set for the entire table.

4.4 Incremental vs full refresh

The Virtuous connector, as implemented here:

Does not implement explicit incremental bookmarks (no per-table updated_at high watermark is tracked).
Instead, built-in query endpoints (e.g. *Query, Search) are used to fetch all records, sorted by id. Filtering (by date or otherwise) is not configured at the connector level.

Result: each sync is effectively a full refresh per table (with pagination). Any incremental behaviour must be implemented downstream (e.g. merge-on-key in your warehouse).

4.5 Deletes

Normal sync via query endpoints does not produce delete records.
Delete awareness is only available when consuming webhook events for tables that support delete actions (e.g. gifts, projects, contact notes). See section 8.

5. API Endpoints & HTTP Methods

All calls are made using an HTTP client configured with:

Base URI: https://api.virtuoussoftware.com/api/
Header: Authorization: Basic {api_key}
JSON request bodies where applicable.

5.1 Table endpoints

For each table in table_map:

Table	Path	HTTP Method	Notes
`campaigns`	`Campaign/Query`	POST	Paged query with `skip`/`take`.
`channel_types`	`Communication/ChannelTypes`	GET	Returns a simple array; value-array table.
`communication_types`	`Communication/CommunicationTypes`	GET	Simple value array.
`communications`	`Communication/Query`	POST	Paged query.
`contacts`	`Contact/Query/FullContact`	POST	Full contacts; supports webhooks.
`contact_notes`	`ContactNote/Query`	POST	Notes per contact; supports webhooks.
`gifts`	`Gift/Query/FullGift`	POST	Full gift details; supports webhooks (including deletes).
`gift_asks`	`GiftAsk/Query`	POST	Gift ask objects.
`gift_destinations`	`GiftDesignation/Query`	POST	Gift designation/destination data.
`individuals`	`ContactIndividual/Query`	POST	Individuals related to contacts.
`organization_groups`	`OrganizationGroup`	GET	Organization groups.
`planned_gifts`	`PlannedGift/Query`	POST	Planned gifts.
`pledges`	`Pledge/Query`	POST	Pledges.
`projects`	`Project/Query`	POST	Projects; supports webhooks (including deletes).
`recurring_gifts`	`RecurringGift/Query`	POST	Recurring gifts.
`volunteers`	`Volunteer/Query`	POST	Volunteers.
`volunteer_opportunities`	`VolunteerOpportunity/Query`	POST	Volunteer opportunities.
`segments`	`Segment/Search`	POST	Segment definitions/search.

All POST-based query calls include a JSON body like:

{   "groups": [],   "sortBy": "id",   "descending": "false" }

This ensures queries are sorted by id ascending, with no additional filter groups.

5.2 Connectivity test endpoint

The test() routine validates your API key by calling:

Path: Organization
Method: GET

If the call succeeds, a meta flag test_result = true is written; otherwise test_result = false along with an error message.

5.3 Webhook endpoints

The connector can optionally manage Virtuous webhooks:

Create webhook

Path: Webhook
Method: POST
Body:
- Boolean flags for each possible event (contactCreate, giftUpdate, projectDelete, etc.), all initialized to false.
- For each selected table, the corresponding webhook events in the table map are set to true.
- Additional properties:
  - payloadUrl – your webhook endpoint URL.
  - secret – a shared secret (currently set to a static value in the code).
  - active – true.

Delete webhook

Path: Webhook/{id}
Method: DELETE
{id} comes from meta->id (stored when the webhook was created).

6. Tables & Columns

Because Virtuous schemas are dynamic and discovered at runtime, the exact column names and counts for most tables are not fixed in the connector and may change as Virtuous evolves or your account adds custom fields.

Where column names are known and stable, they are listed below.

6.1 Dynamic-schema tables

For these tables, columns are inferred by inspecting the first record returned from the API (?skip=0&take=1) and mapping every field in that object to a column:

campaigns
communications
contacts
contact_notes
gifts
gift_asks
gift_destinations
individuals
organization_groups
planned_gifts
pledges
projects
recurring_gifts
volunteers
volunteer_opportunities
segments

For each of these tables:

Columns: All top-level fields returned by the Virtuous API for that entity.
Types: Derived automatically from the first row (see section 4.3).
Unique keys & indexes: taken from table_map:
- campaigns → unique key: campaignId
- communications → unique key: communicationId
- contacts → unique key: id
- contact_notes → unique key: id; secondary index: contactId
- gifts → unique key: id; indices: transactionId, contactId, segmentId
- gift_asks → unique key: id; indices: contactId, projectId, segmentId
- gift_destinations → unique key: id; indices: giftId, contactId, projectId
- individuals → unique key: id; index: contactId
- organization_groups → unique key: id
- planned_gifts → unique key: id; indices: contactId, projectId, segmentId
- pledges → unique key: id; indices: contactId, projectId, segmentId
- projects → unique key: id; index: parentId
- recurring_gifts → unique key: id; indices: transactionId, contactId, segmentId
- volunteers → unique key: id; index: contactId
- volunteer_opportunities → unique key: id
- segments → unique key: id

Note: Because schema is discovered live, adding or removing fields in Virtuous will automatically change which columns appear in these tables on the next discover/sync.

6.2 Value-array tables (fixed columns)

Two tables return simple arrays of values. The connector flattens each into a single-column table.

table: channel_types (1 column)

Endpoint: Communication/ChannelTypes (GET)
Each API response is a simple array of values.
The connector maps each element to a row under the column channel.

1. channel

table: communication_types (1 column)

Endpoint: Communication/CommunicationTypes (GET)
Each API response is a simple array of values.
The connector maps each element to a row under the column communication_type.

1. communication_type

7. Webhook Support

The connector includes optional webhook handling for near-real-time updates.

7.1 Supported webhook event types

Per table, these webhook events can be enabled when creating a webhook:

contacts
- contactCreate, contactUpdate
contact_notes
- contactNoteCreate, contactNoteUpdate, contactNoteDelete
gifts
- giftCreate, giftUpdate, giftDelete
projects
- projectCreate, projectUpdate, projectDelete

7.2 Webhook payload handling

When a webhook payload is received:

The connector parses it and inspects the event field.
It searches table_map for a table whose webhook_events list contains that event name (case-insensitive).
Once matched, it uses the table’s webhook_root_object to extract the relevant object from the payload (e.g. contact, gift, project, contactNote).

Then:

If the event name ends with "delete":
- A delete record is emitted containing the table’s unique key(s) only.
Otherwise:
- A metadata message is emitted marking the table’s unique_keys.
- A full record message is emitted with the object as-is.

Important: Virtuous webhook payloads often contain partial data compared to the full query endpoints. Webhook-driven updates may therefore not include every field you see in the query-based tables. You may need to combine webhook data with periodic full refreshes for complete coverage.

7.3 Webhook lifecycle

Create webhook:
Use the createWebhook method with api_key, the tables to include, and the url of your webhook endpoint.
Delete webhook:
Use the deleteWebhook method with api_key and the stored meta (which includes the webhook id).

Errors during webhook operations are logged, and a webhook_created / webhook_deleted flag is written to metadata to indicate success/failure.

8. Rate Limiting & Retries

8.1 Retry behaviour

All API calls go through a common retry wrapper:

Max attempts: retry_limit = 20
Delay between attempts: retry_delay = 30 seconds
On any caught exception:
- The connector logs the error.
- Waits retry_delay seconds.
- Retries, up to the limit.

If the retry limit is reached, a friendly MaxRetryException is thrown.

8.2 Rate limit headers

The connector inspects these headers on each response:

x-ratelimit-remaining
- Parsed as an integer; warns when fewer than 100 requests remain.
x-ratelimit-reset
- Parsed as a Unix timestamp indicating when the limit will reset.
- If invalid/absent, the reset value is set to the current time.

8.3 HTTP 429 handling (rate limit exceeded)

When an HTTP 429 is returned:

In test mode: the method returns early (no sleep).
In normal mode:
1. The connector calculates target_time = rate_limit_reset + 60.
2. It computes seconds_to_sleep = target_time - current_time.
3. If that is negative, it defaults to 60 seconds.
4. It logs a message and sleeps for the computed duration.
5. It then re-issues the request through the retry wrapper.

For other status codes:

401 → treated as invalid credentials, with a clear error suggesting to re-check the API key.
Non-200 / non-429 / non-401 → a generic HTTP error is thrown.

9. Known Limitations & Caveats

No explicit incremental bookmarks
- All query-based tables are pulled as full datasets using paging. Any incremental logic (e.g. using lastUpdated fields) must be applied downstream.
Webhooks provide partial objects
- Webhook payloads may omit certain fields that are present in full query results. If you rely heavily on webhooks, plan for occasional full refreshes to keep data complete.
Events and form submissions not supported
- The connector does not currently support Virtuous “events” or “form submissions” as data sources.
Schema drift
- Because columns are inferred from live API responses, adding/removing fields in Virtuous can change your table schemas (new columns, removed columns, type changes). Be cautious with strict downstream schemas.
Rate limit sensitivity
- Heavy usage across many large tables can approach the Virtuous rate limit. The connector will back off and sleep when limits are hit, which can make large syncs take longer.
Delete handling limited to webhook flows
- Standard table sync does not detect deletions. To detect deletes, you must configure webhooks and rely on delete events for supported tables (gifts, projects, contact_notes, etc.).
- What this means:
  - Deletes are only captured when you use the connector’s webhook support.
  - Without webhooks enabled: the connector is effectively upsert-only. Deleted records in Virtuous may remain in your warehouse until you do a full refresh or handle removals downstream.
  - With webhooks enabled: deletes can be propagated, but only for the tables/events that have webhook delete mappings in the connector.
  - Not all tables support delete events: for example, dynamic query-based tables without matching webhook delete events will still behave as upsert-only even if webhooks are configured.