The multi-instance functionality allows the deployment of the same pipeline multiple times, taking into account different parameters. In this way, it is possible to configure multiple instances that process data according to the parameters configured in each one, using the same integration flow.

This feature is often used when we need to use the same pipeline, i.e. the same construction logic, to access different endpoints such as Accounts and Globals. This eliminates the need for multiple versions of the same pipeline.

Imagine that you need to extract large amounts of data from a single database and that the time to perform the extraction is limited. With this functionality, you can design an accelerated data extraction strategy that uses multi-instance pipelines to quickly and safely extract large amounts of data from a database. To do this, simply create a pipeline, define it as multi-instance, and also create a configuration with the instances to be used, specifying determined intervals for each of them. This way, each instance is responsible for extracting data between specific dates, as in the following example:

The Period 1 instance is responsible for extracting all data between October 1 and October 8, in as many runs as necessary. The instances Periods 2, 3, and 4 use the same extraction logic as the first one to search and extract all data in the other time intervals. Thus, you will be applying a strategy that uses specific dates and pipelines with multiple instances to extract a large amount of data faster. It would be perfectly possible to perform data extraction using a simple pipeline, however, if you use a multi-instance pipeline, the extraction can be performed up to 4 times faster.

In addition to the use case above, you can use a multi-instance pipeline to run an identical query against databases installed in different environments.

You can also publish pipelines in two different environments. To do this, simply deploy a multi-instance pipeline to the Digibee Platform test environment and configure it to report to two of your environments, for example, QA and test. This way, we would have two test deployments reporting to two environments.

On the Digibee Platform settings page, the Multi-Instance menu lists all the multi-instance models that have already been created within the Platform and their respective configuration parameters. They are:

It is possible to edit, configure and delete the multi-instance models listed through the following action buttons:

This action allows you to edit any configuration parameters that were defined at the time the new multi-instance model was created. These are Name, Description, and Fields.

This action allows creating, configuring and saving instances of each model, inserting and defining values for each previously created field.

Use this action to delete a multi-instance model when it is not in use on the Platform, whether in deployed, non-deployed, or archived pipelines.

Before you create the instances, you need to create a multi-instance model. To do this, access the Multi-instance page, under Settings, click “+ CREATE” and define a name, a description, and the fields of the instances, i.e. the variables that will receive the values in each environment (e.g. test, qa1, qa2, prod). Look at the following example:

Once this is done, click “Confirm” to create the model.

Now, you can create the instances that will match the model you created in the last step. To do this, find the model in the list and click on the corresponding "Configure" action button. This way you can create the instances of this model one by one, entering and setting values for each field.

In the following example, a and b are two stores that receive the values of each variable (which can be configured for both test and prod), and this is repeated for all other instances to be created.

Below is an example of creating another instance (c):

After entering the variables, just click “Save”.

In this step, we create a pipeline and define it as a multi-instance. To do this, access the settings by clicking the settings button:

Select the "Is it a multi-instance?" option, as in the following example:

After selecting the "Is it multi-instance?" option, specify which multi-instance model the pipeline will report to. In our example, we are using the "article-multi-instance" model.

IMPORTANT: Once the pipeline is set to multi-instance, it is not possible to undo this setting. On the other hand, it is possible to convert any pipeline, whether already deployed or not, into a multi-instance pipeline.

Since this is an event, we need to inform the trigger an instance variable that will be executed using the pattern -{{replica.instance_variable_name}}, as in the following example:

Running queries in Test mode

If you are running Test mode in a multi-instance pipeline, you can enter the instance you want to run in the first column of the TEST tab, as in the following example:

In this way, it is possible to consult the values previously configured in the selected instance via the pattern {{replica.instance_variable_name}}.

Now that you have configured the template and set the pipeline to multi-instance, you need to edit the pipeline request with the instance on which you want to deploy it. This way, the instance name will be added to the pipeline name after deployment. For example: once the "multi-instance-example" pipeline is deployed on instance a, its name on the "Run" tab will be "multi-instance-example-a".

In the example below, the "multi-instance-example" pipeline is being deployed on instance a.

In the example below, the "multi-instance-example" pipeline is being deployed to instance b.

After editing, simply click “DEPLOY”.

Below, the pipeline “multi-instance-example” already deployed on instances a and b, on the Run tab: