You are browsing a tutorial guide for the latest Octoparse version. If you are running an older version of Octoparse, we strongly recommend you upgrade because it is faster, easier, and more robust! Download and upgrade here if you haven't already done so!
Let's say you have created a magnificent crawler with branches and loops, and you realize you have to start scraping from multiple links instead of a single link. Now you look back at your workflow, which looks like this.
We understand how much you don't want to start over at the moment!
Here are two simple solutions to save you from work.
Solution 1:
Click on the "Edit URL(s)" icon right next to the Browse button
Input a list of URLs and Click on Save
A Loop Item with the URLs will be created in your workflow automatically. So easy!
Solution 2:
Solution 1 is only working when you need to change the Starting URL which is in the first Go to Web Page step. If you have multiple Go to Web Page steps, you can manually create a Loop to include your URLs.
A. Add a Loop to open the list of links
Click Add Step to add a loop
Choose Loop in the dropdown menu
Set the Loop Mode as List of URLs
Click Manual Input [0 URLs]
Copy and paste the URL list into the input box on the following page
A new Loop has now been set up.
B. Combine the new Loop with the rest of your workflow
Drag the rest of the workflow (including Go to Web Page, other Loop Items, or Branches) into the new Loop Item with URLs as shown in the picture below:
Click on Go to Web Page
Tick Load URLs in the loop
Apply the setting
