In the last month, X has announced they will be sunsetting the APIs that we’ve used for collecting data on TRAC. Quick to ensure we don’t lose access, we’ve worked tirelessly to implement the newer V2 endpoints they have provided access to; but with these new endpoints comes a few changes to X data.
Below is an outline of the major changes to X data on TRAC, going forwards.
Proximity Operators – Affects Historics Only
You may have already seen this change, which we communicated last week. From now onwards the proximity operator (~) will not be supported for X historics content.
For any existing or new searches with the proximity operator, you will need to either exclude X from your historics, or exclude the proximity operator in your setup, if you wish to consume X historical data.
Location Filtered Searches – Affects Real-time and Historics
The operators we relied on for filtering by location are no longer available in the new v2 APIs - so we’ve had to come up with some crafty solutions to ensure you continue to collect location specific content. The impact of this change is as follows:
For any real time searches with a country filter, you may see a sudden dip in the data volume; and this is expected.
Historic searches with a country filter remain unaffected therefore we recommend backfilling any missing real-time content historically.
Region filtering is no longer available for both historical data and real time data.
This means we will disable the region field when only X data is specified at search setup.
And existing X searches with region targeting will be updated and default to just a country filter.
Replies collected in Searches – Affects Real-time Only
Previously, a search would collect real-time replies, if they matched the search terms. Now, we no longer distinguish between matching and non matching replies; meaning we will collect any reply to a post.
Depending on your use case, you may find this change useful, as it means you get the complete view of a real-time conversation.
However, if you find this type of content too noisy, there is the option to exclude all replies from being collected at search setup.
We have spent the very limited time given to us to ensure these changes do not interrupt data collection, and we appreciate you bearing with us on how abrupt this change is.
While we have ensured a thorough check of the data now coming from the new endpoints, there can sometimes be unforeseen edge cases that are not always obvious until they appear, so if you notice any other discrepancies that are unexpected, please do not hesitate to reach out to our support team.
X data queries will remain a higher priority for some time after this change to ensure that any edge cases have been addressed.