You are browsing a tutorial guide for the latest Octoparse version. If you are running an older version of Octoparse, we strongly recommend you upgrade because it is faster, easier and more robust! Download and upgrade here if you haven't already done so!
Coinmarketcap is a website that provides free access to current and historic data for Bitcoin and thousands of altcoins. From the website, people can have an overall view of the constant coin market.
In this tutorial, we are going to scrape crypto coin prices and other info from Coinmarketcap.
Sample URL: https://coinmarketcap.com/
The main steps are shown in the menu on the right, and you can download the sample task file here.
1. Create a Go to Web Page - to open the target website
Create your task by inputting the URL in the search box on the homepage
Click the "Start" button to move on
2. Auto-detect web page data - create a workflow
Click Auto-detect web page data and wait for it to complete
Click Create workflow to generate a workflow
The automatically generated workflow would look like the following:
Go to Data Preview to remove unwanted data or rename it if needed
Delete unwanted data fields directly by clicking More and Delete field
Modify the data field names by double-clicking the headers
3. Modify the action order - to correctly load more information
In this case, we need to drag the Loop Item out of the Scroll Page step which means extracting data after the whole page is loaded. Then set the scroll page for one screen.
Pull the Loop Item out of Scroll Page
Click Scroll Page
Set Scroll for one screen
Repeats 30 times
Wait 2s
Click Apply
Tip: For more tutorials for page scroll-down settings, you can check here.
4. Run your task - get the data you want
Click the Save button first to save all the settings you have made
Then click Run (You can either Run on your device or Run in the cloud)
Select Run on your device and click Standard mode to run the task on your local device
Wait for the task to complete
Here is the sample data from a local run.
TIP: Local runs are great for quick runs and small amounts of data. If you are dealing with more complicated tasks or a mass of data, Run in the Cloud is recommended for higher speed. You are very welcome to try the premium feature by signing up for the 14-day free trial here. Tasks could be scheduled hourly, daily, or weekly and data delivered regularly.