Pipes robots are the Swiss-army knives of Dexi: they can do pretty much anything. However, with great flexibility comes the potential for great confusion 😀
For an introduction to Pipes robots, head on over to Pipes.
This article covers a couple of advanced settings of Pipes robots.
Executing other robots ("Default" configurations)
To execute another robot in a Pipes robot use the "Execute robot" action:
When the Pipes robot is then executed (or more precisely, when a configuration of the Pipes robot is executed), a Default configuration of the other robot is automatically created by the platform.
The "Default" configuration is identical to any other standard robot configuration. However, to avoid confusing and unpredictable behaviour, the fields of "Default" configurations cannot be changed - except for the Proxies field.
This is to allow the user to control which proxies are being used for the executions of the "Default" configuration (the proxy specified on the Pipes configuration is not propagated to the "Default" configuration). See What should I know about proxy servers and anonymity? for more information about proxies.
Two other important settings, maximum number of retries and maximum concurrency, should be controlled from the "Execute robot" action as shown in the screenshot above. More on this below.
"Max retries"
"Maximum number of retries" has a couple of different meanings and can be controlled from a couple of different locations:
"Maximum attempts" on a Pipes configuration: how many times to retry execution of each input/result of the configuration. NB! Pipes by design rarely fail so it almost never makes sense to change this option.
"Max retries" on an "Execute robot" action: how many times to retry executions of the "Default" configuration.
"Maximum attempts" on a "Default" configuration: how many times to retry execution of each input/result of the configuration. Has no effect and cannot be changed. Use "Max retries" on "Execute robot" action instead.
To read more about retries, see section "About Retries" on How can I create a run?.
"Max concurrency"
Similarly, "maximum concurrency" has a couple of different meanings and can be controlled from a couple of different locations:
"Concurrent inputs/results" on a Pipes configuration: the number of inputs/results on the Pipes configuration that can run concurrently.
"Max concurrency" on an "Execute robot" action: the number of executions of the "Default" configuration that can run concurrently. This value is per input/result ("instance") of the Pipes execution.
"Concurrent inputs/results" on Extractor "Default" configuration: the number of inputs/results on the configuration that can run concurrently. Has no effect and cannot be changed. Use "Max concurrency" on "Execute robot" action instead.
Example 1
"Concurrent inputs/results" on Pipes configuration: 10
"Max concurrency" on "Execute robot" action: 3
Up to 3 * 10 = 30 workers can be used - three for each input/result of the Pipes execution.
Example 2
"Concurrent inputs/results" on Pipes configuration: 5
"Max concurrency" on "Execute robot" action #1: 5
"Max concurrency" on "Execute robot" action #2: 25
Up to 5 * (5 + 25) = 150 workers can be used - 30 for each input/result of the Pipes execution.
Worker Usage Limits
From the "Robot Settings" tab of your account settings, you can control how many workers can be used for each robot type, e.g. for Pipes:
The default is 50% for Pipes. The motivation for this limit is that your account needs free workers to be able to execute Extractor robots from a Pipes robot.
If you use up all workers for running Pipes robots, e.g. a Pipes robots with multiple inputs, Dexi won't be able to execute any other robots simultaneously, including robots executed by the Pipes robot. This could result in a deadlock where no executions are running, ie. seemingly are stuck.
So while it may be ok for some accounts to increase the limit for Pipes to, say, 70%-80%, we advise against it to ensure that there is always enough worker capacity.