All guides
AI crawler logs
Which AI crawler reads which page of your site, and how often. People are never sent anywhere.
What you need
- Plan
- Set up by AI WISE for you
- Who
- AI WISE, with your web team.
- You need
- Someone who can change your site: a Cloudflare account on its domain, or your Next.js code.
Set it up
- AI WISE sends your web team a short piece of code with your business's own key in it.
- Cloudflare: Workers › Create, paste it, and add a route for your site (example.com/*). The free plan carries 100,000 requests a day.
- Next.js on Vercel or anywhere: add it to middleware.ts and deploy.
- Each page loads as before; when an AI crawler reads one, one small note is sent to WISE Lens afterwards.
When it works
The card says Connected once an AI crawler visit arrives — usually within a day.
If it does not work
- The card still says Not read yet after two days
- Check the Worker's route covers your whole site, or that middleware.ts was deployed, then write to us.
Need help? Write to support@aiwise.sg and say which connection.
Turn it off
Remove the Worker's route, or the lines in middleware.ts.
Questions: support@aiwise.sg