You are browsing a tutorial guide for the latest Octoparse version. If you are running an older version of Octoparse, we strongly recommend you upgrade because it is faster, easier and more robust! Download and upgrade here if you haven't already done so!
This tutorial will show you how to scrape product information, such as product name, price, brand, etc., from Target.com.
To follow through, you may want to use the URL below:
The main steps are shown in the menu on the right, and you can download the sample task file here.
1. Create a Go to Web Page - to open the target website
Enter the page URL on the home screen and click Start to create a new task
After starting the task, you will see a step of Go to Web Page.
Go to the settings of Go to Web Page
Click Options
Tick Scroll down the page after it is loaded
Set Scroll for one screen
Set the Wait time as 3s and Repeats 15 times
2. Save the cookies - to load product information
Since the product information will only be presented if there is a certain shop on Target.com, we need to choose one and tell Octoparse to save our choice.
Turn on the Browse Mode
Click Please select a Store
Input your zip code to find stores nearby > choose the store you need
Tick Use Cookie > Click Use cookie from current page > Apply
Turn off the Browse Mode
3. Auto-detect the webpage - to create a workflow
Click Auto-detect web page data and wait for the detection to complete
Uncheck Add a page scroll
Click Create workflow
After creating the workflow, you can go to the Data Preview to see if the fields look good to you.
Delete any fields you don't want
Rename the fields by double-clicking on the header
4. Modify the settings of Pagination - to fully load the content on the webpage
Choose Click to paginate in the workflow > Click Options
Tick Scroll down the page after it is loaded
Choose Scroll for one screen
Set the Wait time as 3s and Repeats 15 times
Choose the Pagination box in the workflow
Input the Matching XPath as: //button[@data-test='next' and not(@disabled)]
5. Run the task - to get your desired data
Click Save on the upper right to save your task
Click Run next to it and wait for a Run Task window to pop up
Select Run on your device to run the task on your local device
Wait for the task to complete
Here is the sample output from a local run:
