About CoScheduleBot
Learn about CoScheduleBot, the web crawler for CoSchedule.
The data crawled by CoScheduleBot is used to power various features, such as the search technology that is integrated into Open Graph scraping and WordPress syncing operations.
CoScheduleBot accesses many kinds of resources from web servers, including but not limited to robots.txt, sitemaps, RSS feeds, HTML, sub resources needed to render pages such as javascript, Ajax requests, images, and more.
Identifying CoScheduleBot
Traffic coming from CoScheduleBot is generally identified by using reverse DNS in the *.coschedulebot.coschedule.com domain. Another way is to match the IP address with the below known IP addresses:- 34.202.82.63
- 52.71.109.154
Reverse DNS
The host command can be used to determine if an IP address is part of CoScheduleBot. These examples show the host command and its result:$ host 34-202-82-63.coschedulebot.coschedule.com
34-202-82-63.coschedulebot.coschedule.com has address 34.202.82.63
The host command can also be used to verify that the DNS points to the same IP address:
$ host 34.202.82.63
63.82.202.34.in-addr.arpa domain name pointer 34-202-82-63.coschedulebot.coschedule.com.
User agents
A user agent helps webmasters identify crawler traffic, so that they can get accurate access log reports of crawler activity. CoScheduleBot uses the following format: The user-agent string contains "CoScheduleBot" and other information. The following is the general format:Mozilla/5.0 (Device; OS_version) AppleWebKit/WebKit_version (KHTML, like Gecko) Chrome/Chrome_version Safari/Safari_version (CoScheduleBot/CoScheduleBot_version; +https://coschedule.com/coschedulebot)
Example:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 (CoScheduleBot/1.0; +https://coschedule.com/coschedulebot)
Occasionally, CoScheduleBot will update the browser version that it advertises while remaining in the general format above.