There was an unexpected error authorizing you. Please try again.
IAB Tech Lab Spiders & Bots List

IAB Tech Lab Spiders and Bots List

Last updated: January 16, 2025

What is this?

The IAB Tech Lab Spiders and Robots list helps companies identify automated traffic such as search engine crawlers, monitoring tools, and other nonhuman traffic that they don’t want showing up in their analytics and billable counts. Additionally, using an industry standard list like Spiders and Robots list reduces discrepancies between parties and achieves the requirement for detecting General Invalid Traffic as per the Invalid Traffic Detection and Filtration Guidelines.

You can read more about why IAB Tech Lab provides the Spiders and Bots list here.

Download a Sample File (Zip)
Download Spiders & Bots Best Practices

I'm interested in subscribing to the Spiders & Bots List

Subscription & Pricing

Subscriptions last 12 months from the date of sign up and access is provided via FTP. Access to the list is not available until payment and a signed data subscription agreement have been received by spiders@iabtechlab.com. Though members get discounts, you do not have to be a member of the IAB to subscribe to the list. 

A subscription gives you access to:

  • Industry leaders in spiders & bots detection and filtering
  • The Alliance for Audited Media and IAB support
  • A comprehensive guide to best practices in using the spiders & bots list

Pricing

  • IAB US General Member: $5,000
  • IAB Tech Lab Member: $5,000
  • IAB Associate Member: $7,500
  • Non-Member: $15,000

Submit Newly Discovered Browsers or Spiders & Bots

IAB requests that subscribers submit any new browsers or spiders and/or robots they discover as often as possible. List modifications may be recommended by any entity including non-IAB members. The list will be updated monthly to reflect changes that are brought to the attention of The Alliance for Audited Media, ABC UK, and the Policy Board.

Submit Here

About the International IAB/ABC Spiders & Bots List

The IAB has enlisted The Alliance for Audited Media to perform evaluation and management services associated with maintaining and publishing the industry Spiders and Robots list, the use of which is required for compliance to the IAB’s Client Side Counting (CSC) Measurement Guidelines for “double pass” filtering. We have also formed a Spiders & Robots Policy Board to oversee and approve list modifications (see list below).

Spiders & Bots Policy Board

  • Kathryn Hodgkinson, ABC UK
  • Martin Liljenback, Adobe
  • Katrina Eddy, Alliance for Audited Media
  • Todd Martens, Alliance for Audited Media
  • João Matos, Criteo
  • Thomas Kent, Extreme Reach
  • Breen Baker, Google
  • Zhong Wang, Google
  • Michael Smith, Hearst
  • Jill Wittkopp, IAB Tech Lab
  • Saif Jafri, Yahoo