Netlify

A Netlify Edge Function reports crawler requests after your site answers, on any Netlify plan. It runs only for requests whose user agent names an AI crawler, and if it fails, Netlify serves your site without it.

1. Add the file Setup names under netlify/edge-functions, with the code from Setup.
2. In Netlify, open Project configuration, then Environment variables, and add SEARCHLY_CRAWLER_TOKEN with your token.
3. Deploy the site.

The function records each response's status and the IP address Netlify reports, so requests from operators that publish IP ranges can be verified. Each crawler request is one Edge Function invocation on your Netlify plan.