Crawler
LookyBot, the Looky crawler
LookyBot is the program Looky uses to read a public website when its owner has asked Looky to. If you saw it in your server logs, this page explains what it does and how to stop it.
User agent
LookyBot/1.0 (+https://getlooky.ai/bot)Every request from LookyBot sends this header.
What it fetches
- Public pages of a website that a Looky customer entered as their own or as a competitor's, to build a brand profile, check the site for search and AI visibility and spot changes.
- The robots.txt file first, and the sitemap if one is listed there.
- Only pages that anyone can open without logging in. It never submits forms, logs in or tries to guess hidden addresses.
How it behaves
- It follows robots.txt. A path you disallow for LookyBot, or for all robots, is not fetched.
- It honors a Crawl-delay line, and otherwise waits at least half a second between requests to the same site.
- It reads a limited number of pages per run, one site at a time, and stays on the site it was given.
Crawler
How to stop it
Add this to your robots.txt and LookyBot stops reading your site:
User-agent: LookyBot Disallow: /