You are browsing a tutorial guide for the latest Octoparse version. If you are running an older version of Octoparse, we strongly recommend you upgrade because it is faster, easier and more robust! Download and upgrade here if you haven't already done so!
Some websites (like e-Commerce) require scrolling down to the bottom of the page to load additional content. In this case, you can set up a Page scroll-down to have the page scrolled automatically to Octoparse. This is especially useful for extracting a web page that uses infinitive scrolling or lazy loading.
1. Set up the scroll for Go to Web Page action or Click Item action
Click the Go to Web Page action (or a Click Item) to access the settings menu. Go to the Options tab.
To set up the page scroll-down, check the box for Scroll down the page after it is loaded
Choose the Scroll Area
Default: Octoparse will scroll down the whole web page.
Partial: Some websites may include not only one scroll, like Google map reviews. To locate the exact right area you need, you can select partial and input the precise XPath for the area.
Tip: You can check out this tutorial on how to use this Partial Scroll feature: Scrolling within a designated area of a web page
Choose how you'd like to scroll down the page
There are two ways of scroll-down: scroll to the bottom of the page and scroll for one screen.
Scroll to the bottom of the page: Octoparse will scroll directly down to the bottom of the page without stopping in the middle of the page. You should consider using this option when the page only loads elements when you reach the page bottom, like Google Play reviews.
Scroll for one screen: Octoparse will scroll one screen only each time. Consider using this option if the page loads content as you scroll down the page continuously.
Scroll for one screen can be used for all websites while scroll to the bottom of the page may fail to work for specific websites. We recommend that you first test the options out by scrolling through the page manually, then pick the one that works best for your target web page.
Set up Wait time and Scroll times
Select the proper wait time between each of the two scrolls. You may want to pick a longer wait time for pages that take longer to load.
Next, enter a number for Scroll times which is the number of times you'd like to repeat the scroll-down (ie. the number of scrolls). Again, you may want to scroll the target web page manually first, just to find out how many scrolls you'd need to load all the required information.
2. Set up scroll with a Loop Item
Add a Loop Item to the workflow
Choose Scroll Page as Loop Mode
Set up scroll options as we have shown above
