A pipes robot connects other robots and data together - and can apply transformation to that data. Pipes can also interact with 3rd party services like API's, send HTTP requests and more.  We often refer to Pipes as a sort of super robot as it is able to control and govern other robots.

A few examples as to what you can use it for:

  • Execute multiple extractors and/or crawlers, connecting outputs to inputs
  • Transform text and images
  • Perform calculations
  • Join separate data sets
  • Parse JSON and XML
  • Send HTTP requests
  • Send data to an SQL database
  • Read data from datasets
  • Do lookups in a dictionary
  • Interact with 3rd party applications like Google Geocoding or Foursquare

In real world use, this means easier price comparisons, simple image resizing, unifying data sets from multiple web sites and also normal Extract-Transform-Load (ETL) operations.

Adding nodes

A pipes robot contains nodes, each of which performs a specific action. To add nodes to your pipe simply drag and drop them from the right-hand list of actions onto the pipes grid:


You can also debug pipes robots - both while they're running and once they've finished. You do this by opening up the execution and clicking the green "bug" icon found next to each result.

Read more about debugging

Did this answer your question?