Learning Outcomes:
Identify the differences between real-time and historic data.
Learn how to access and utilise each data type in TRAC.
Pulsar's data comes in two forms: real-time data and historic data. Both types serve unique purposes and provide different insights depending on your needs.
By understanding the differences between real-time and historic data, you can make informed decisions on which type of data best serves your needs, ultimately enhancing your research and analysis capabilities in TRAC.
How can I collect data within TRAC?
You can find options for collecting both real-time and historic data in TRAC from the Search Management > Status menu, as shown below. From here you can collect data in real-time or historically. Historical availability will depend on the data sources you have enabled in your search.
Collecting Real-time Data
Real-time data collection means collecting data from the moment you initiate a real-time search onwards.
You can schedule real-time collection to commence at a later date in the future or start collecting data immediately.
Real-time collection can run infinitely, can be manually paused, or you can set a date in the future when you want the data collection to automatically stop.
Posts are collected at regular intervals, 24/7, and the frequency of data collection during a 24 hour period depends on the data source, for example, real-time X data is collected every 90 seconds.
📝 Note: Real-time data is available for all TRAC data sources.
Top Tip: When using TRAC to collect posts related to a time-limited event or campaign, you can schedule an end date for the real-time data collection to cease.
Collecting Historical Data
Historic data is content from the past, which is delivered to your TRAC searches based on a time period you specify. A search can contain both real-time data and historical data; or can have real-time data only; or can have historical data only. Unlike real-time data, historical data requests are processed separately, ensuring that you receive an estimated volume preview before launching the data collection.
To help you manage the amount of data you will collect historically, every historical data order comes with a data volume preview. If you are happy with your volume previews, you can proceed with launching your historical data collection. If the volume preview indicates a large amount of data, consider the following strategies to manage or reduce the volume effectively:
Refine Your Search Criteria: Use specific keywords or phrases to narrow down the scope of your search.
Adjust the Time Frame: Decrease the historical time frame to focus on a more targeted period.
Filter Out Retweets: Exclude retweets to reduce redundant mentions.
Strategies for Optimizing Historical Data Queries
Use Contextual AND Logic: Narrow your query by combining terms with AND logic to focus on specific contexts. For example:
Instead of generic terms like "shoes," use:
(running OR jogging OR marathon) AND (training OR shoes OR injury).
Avoid Generic Terms: Standalone terms like "outfits" or "accessories" can generate excessive results. Pair them with relevant keywords to refine your search.
Filter by Language or Location: If applicable, limit your query by language (e.g.,
AND LANG("en")) or geographic location to reduce the volume of mentions.
📝 Note: All historical data orders, up to 250K, are authorised automatically by the system, whilst orders above 250K will require authorisation by the account manager or the support team. Click here to learn more about historical data availability. The 250K historical mentions threshold is designed to balance user needs with platform capabilities, ensuring that large-scale data requests are manageable and do not disrupt overall system functionality. If a historical data job exceeds the allowed number of mentions per month, it will not be auto-approved, and the data will not be ingested. So it's encouraged to plan the queries with this threshold in mind to avoid delays or rejections.
We hope you enjoyed reading this article! 📚
If you have any questions or would like to learn more, please don't hesitate to reach out to our support team via live chat. 🚀