All Collections
Case Tutorial
E-Commerce
Scraping product details from Wayfair
Scraping product details from Wayfair
Updated over a week ago

You are browsing a tutorial guide for Octoparse's latest version. If you are running an older version of Octoparse, we strongly recommend you upgrade because it is faster, easier, and more robust! Download and upgrade here if you haven't already done so!

In this tutorial, we are going to show you how to scrape the product details from Wayfair, an American home improvement retailer commerce company that sells home goods.

To demonstrate, we will use the URL below in order to scrape data such as the product title, description, and price from each product details page.

The main steps are shown in the menu on the right, and you can download the sample task file here.


1. Go to Web Page - open the targeted web page

  • Enter the target URL into the search box at the center of the home page

  • Click Start to create a new task

URL.png

2. Create a pagination loop - scrape search results from all pages

  • Scroll down to the bottom of the page and click the Next button

  • Click Loop Click on the Tips panel

  • Set up AJAX as 7-10s

AJAX.png

3. Create a Loop Item - scrape all the items on each page

  • Click on any product title on the page

  • Click Select all similar elements on the Tips panel

  • Click Loop click each element

  • Click No when the tips panel ask if the website use pagination because we've already set up the pagination in Step 2


4. Extract data - select data for extraction

Now, Octoparse will open the details page of the first product.

  • Click on the data you need on the page

  • Select Text on Tips

  • Repeat the step for each desired data

  • Click on the Extract Data and untick Extract data in the loop

  • Rename the fields by double-clicking the field

  • Delete unwanted data by clicking the more icon

field.png

Your workflow will show below:

final.jpg

5. Run the task - to get the desired data

  • Click the Save button first to save all the settings you have made

  • Then click Run to run your task either on local device or in the cloud

Below is a sample data run from the local. Excel, CSV, HTML, and JSON formats are available for export.

DATA.png
Did this answer your question?