You are browsing a tutorial guide for the latest Octoparse version. If you are running an older version of Octoparse, we strongly recommend you upgrade because it is faster, easier and more robust! Download and upgrade here if you haven't already done so!
If you've run into any of the issues below, you may want to try setting up IP proxies manually in Octoparse:
Unable to log into Octotoparse because your company or institution uses intranet and restricts external requests
Target website is very sensitive to web scraping and implements anti-scraping strategies such as IP blocking to stop scraping activities
In this tutorial, we are going to show how to set up proxies with the latest version to resolve the issues mentioned above.
1. Use a proxy to log in to Octoparse (for login only)
This is designed for users whose company restricts external requests. The proxy setup in the login interface will only be applied during the logging process and WON'T be applied for task editing or scraping.
Go to settings on the login interface
Choose Manual proxy configurations and input the IP address and Port
You can Test Connection to see if the connection is good and then Confirm the setup
2. Rotate external proxy servers to counteract IP blocking
Octoparse Cloud IPs are data center IPs, which may be blocked. If a website blocks your IP or our Cloud, you can set up an IP proxy in the task settings. You can use your own IP proxies or use Octoparse built-in proxies.
The proxy setup will only be applied when the task runs and won't be applied when you edit the task.
2.1 Set up Octoparse Built-in proxies (for both Local and Cloud)
Octoparse proxies are residential IPs, so they work better in avoiding being blocked.
Open the task you would like to set up proxies for
Go to Task Settings > Anti-blocking
Tick Access website via proxies
Click on Use Octoparse proxies
Select Default or Country/Region you'd like for the IP address (Default means to use IPs from random countries)
Choose Rotate time for the rotation interval
Click the Save button on the upper left corner of the interface to save your settings
NOTE:
Octoparse built-in proxies can also work when you are creating the task(starting from version 8.6.8).
Octoparse proxies cost $3/GB. Make sure you have enough Proxy Credits or Account Credits to use this feature. To enable IP proxies, you must have at least $3 in Account Credits. Account Credits displayed on the page can be used for proxies purchases. You can simply top up your credits by clicking the Add Credits button.
Proxy usage is based on page loading traffic.
1GB proxies can usually scrape 500 web pages. The proxies might not work for all web pages. You may top up some credits to do testing first.
Proxies credits cannot be refunded
2.2 Use your own IP proxies (for local only)
Another way to set up IP proxies is to access your own IP into Octoparse. Your own IP proxies can only be applied in local runs.
Tick Use my own proxies
Click Configure
Input IP Proxies in the box
Confirm the settings
NOTE: Octoparse can only support HTTP proxies - please separate the IP address of the proxy server and port number with a colon. e.g. 58.22.214.29:2318.