Skip to main content

About the Morv Crawler

Last updated: 28 April 2026

What is Morv?

Morv is the web crawler operated by Morv AI, an AI chatbot platform. It fetches web pages on behalf of Morv customers so their chatbots can answer questions using content from their own websites.

Why did Morv visit my site?

A Morv customer added a URL or sitemap on your domain as a knowledge source for their chatbot. Morv only crawls URLs that a customer has explicitly configured — it does not discover or crawl sites on its own.

If you did not expect a visit, it is likely that:

  • A colleague or member of your organisation set up a chatbot using your site as a source.
  • A third party added a URL on your domain as a source for their own chatbot. In this case, you can block the crawler (see below).

User-Agent string

Morv identifies itself with the following User-Agent header:

Mozilla/5.0 (compatible; Morv/1.0; +https://morv.ai/bot) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36

The token name used in robots.txt rules is Morv.

How often does it crawl?

  • Initial crawl: when a customer adds a URL or sitemap as a knowledge source.
  • Recrawl: each source has a recrawl frequency chosen by the customer — never, daily, weekly, or monthly.
  • Manual refresh: customers can trigger an on-demand recrawl at any time from their dashboard.

Morv fetches pages one at a time, with normal request timeouts, and does not parallelise aggressively. If you believe the crawl rate is affecting your site, please contact us.

How do I block Morv?

Morv respects robots.txt. To block all Morv crawling of your site, add the following to the robots.txt file at the root of your domain:

User-agent: Morv
Disallow: /

To block only specific paths:

User-agent: Morv
Disallow: /private/
Disallow: /admin/

Changes to robots.txt take effect on the next crawl attempt.

Contact

If you have questions about Morv, want to report abusive behaviour by a customer, or need a crawl stopped immediately, contact us at support@morv.ai.