Blendr.io has a unique feature to improve the implementation of data synchronization templates, called the CDP (Central Data Platform).

The CDP should not be confused with a Customer Data Platform.

The Blendr.io CDP is a central database used as an intermediate storage (cache) to implement data synchronization templates.

A typical integration template using the CDP will consist of at least 2 Blends:

  • One blend to read data from a source platform and store it in the CDP
  • One blend to read data from the CDP and send it to a destination platform

The Blends are combined in an integration template using the "Bundle" concept. A Bundle is a grouping of Blends that form one template. See the article on Bundles for more information.

Following Blend template contains examples used in this article:
https://admin.blendr.io/templates/952d2140-d17b-11e9-91de-c5567f7759d6

Benefits of the CDP

  • Unified data model
  • Ability to re-use Blends in multiple Bundle templates
  • Central storage of the id's of an object from both source and destination (the CDP object has a direct link to its source object and its destination object)

CDP Datamodel

The CDP has a fixed data model. The goal is to unify integrations by mapping data from a given cloud platform to the CDP unified data model.

Custom fields can be added to objects, in order to store data that does not fit the CDP unified data model.

Pattern 1: from source to CDP

Get data from the source using webhooks (triggered Blend) or an incremental block (scheduled Blend). Use a CDP "Upsert" block to send each object to the CDP.

Example:

CDP Upserts

Configure the "Where" section of the Upsert block. This section defines how to identify existing records in the CDP:

The Where section should have 2 Key/Values:

  • external_id: equal to the id from the source (e.g. Salesforce id)
  • source: name of the source platform (text, e.g. "salesforce")

Configure the behaviour for creation of new objects and updates of existing objects:

Note: the CDP Upsert is smart, it will not increase the "timestamp last update" in the CDP when the object did not change. This avoids unnecessary webhooks to be triggered from the CDP to the destination.

Add field mapping from the source to the CDP object:

Make sure to map at least following fields:

  • external id: the id from the source (same mapping as in the Where section)
  • source: name of the source platform (text, e.g. "salesforce", same as in the Where section)

Use custom fields for data that cannot be mapped to the CDP Unified datamodel. For example, the full source object can be stored as a custom field:

Pattern 2: from CDP to destination

Get data from the CDP using webhooks (triggered Blend) or an incremental CDP block (scheduled Blend) and send it to the destination.

Using Incremental CDP blocks (scheduled Blend)

Example using an incremental block:

Check if the destination id is already present in the CDP:

If the destination id is not present in the CDP, create the object in the destination, otherwise, update the object in the destination.

Note: you can also do a lookup in the destination first, to check for existing records that were created outside the scope of the Blend. For example, when creating contacts in a destination CRM, you could lookup an existing contact in the CRM by email first, and when found, update this object instead of creating a new object.

If a new object is created in the destination, make sure to add the id to the CDP using an Upsert Block (see the block "Upsert Contact 2" in the above example):

In the Where section, add one key/value:

  • Key: id
  • Value: id from the object in the CDP

Set "Create new" to "no" and "Update on exist" to "yes".

In the field "External Id", map the id of the newly created record in the destination (in the above example, the output from the Teamleader block "Create Contact").

In the Source field, and the name (text) of the destination: "teamleader".

The result is that this external id will be stored in the CDP under the following path:

$.item.external_ids.teamleader

Leave the other fields of the CDP Upsert block empty.

Using CDP webhooks (triggered Blend)

If you use CDP Webhooks, make sure to create a Blend that listens to newly created objects in the CDP, and a second Blend that listens to updates coming from the CDP.

Example Blend using Webhooks, that listens to newly created Contacts in the CDP:

Directly below the top Block, use a Condition Block to check the source of the webhook event, and ignore updates coming from the destination platform. This is needed because the destination id is written to the CDP (see CDP block "Upsert Contact 3" in the above example) and this will trigger another webhook event coming from the CDP:

Raw path of the above argument:

{$.OnContactCreated.source}

When creating a new object in the destination platform, make sure to add the id in the CDP (see CDP block "Upsert Contact 3" in the above example):

Handling parent records (foreign keys, relations between child/parent objects)

When using a triggered Blend (CDP Webhooks), you can use the "Retry" block, in case the creation of an object in the destination fails due to the parent record not being available yet.

This can happen when using Webhooks and is called a "racing condition". The Blend that creates the parent record in the destination (e.g. a Company) may not have been executed yet when the Contact is created. In that case the Retry block will keep trying to create the Contact, and this will succeed once the Company is created.

Example:

The above example Blend listens for new Attendees in the CDP. As always, it will check if the event is not caused by the Blend itself. Next, it will get the parent record (Event) from the CDP:

The Blend will check if the parent record exists in the destination. This is done by checking the destination id in the CDP:

Raw path of destination id in CDP:

{$.getEvent.external_ids.some_destination_platform}

If this id is not set, this means the foreign key (the "event id" in the destination) is not known, and the Blend will be retried later on:

Limiting the scope to one bundle

Use the placeholder { bundleguid } to limit the scope of all Blends in a Bundle to one single bundle. This is needed to avoid processing data from other bundles, in case more than one integration is active in a single account.

Pattern 3: deleting records


This pattern describes how records can be deleted in a destination platform, when they are deleted in a source platform. This pattern uses the CDP and consists of 2 steps: 

  • Delete record in CDP when record is deleted in source platform
  • Delete record in destination platform on deletion in CDP

See also: patterns to delete records

Delete record in CDP when record is deleted in source

Records can be deleted in the CDP, based on an incoming webhook event "on Delete" from the source platform:

In case the source platform does not provide this type of webhook events, a comparison can be made between all records in the source platform and all records in the CDP. Note that this can quickly become very slow and does not scale for large amounts of records (e.g. 10K records or more):

Delete in destination platform on delete in CDP

Use the "On delete" webhook event from the CDP, to delete records in the destination platform:

Note that the CDP holds the external id from the record in the destination platform, this id can be used to delete the actual record.

Did this answer your question?