Site SEO & AI Crawler Audit in VS Code

Questions it answers

Site SEO and AI Crawler Audit checks public web pages and reports what they publish right now: SEO tags, redirects, AI crawler rules, AI citation signals, landing page conversion and, for a site you own, what leaks publicly. People also call it Site Check.

The exposure check works only on a domain you prove you own: the first call gives a DNS TXT record or a file to publish, and the check runs once it is live. It never looks up a person, a user name or an email address.

Page and crawler checks: which of 14 AI crawlers robots.txt allows on a page and the rule that decided it; the title, description, canonical, Open Graph and Twitter tags and which are missing; and the 301 redirect chain with status codes and headers.

Store and AI checks: one product page's SEO and Product structured data (price, GTIN, availability); a domain's llms.txt, agents.md, UCP file, sitemap and AI crawler rules; five AI citation signals of one page, each with evidence and a fix; and a sample of up to 40 sitemap addresses for broken pages and redirects.

Landing page check: twelve conversion items such as the h1, the call to action and whether its links work, form fields, trust signals, a price section, viewport and a contact path, each with evidence and a fix. It is a checklist, not a forecast.

Exposure check: exposed or clean with the top 3 risks and fixes, from dangling subdomains, leaked files such as .env, admin paths and masked API keys in the page's scripts, plus six legal signals with official sources. No port scan, and not legal advice.

AI-built app check: for an app made with Lovable, Bolt, v0, Replit or any AI app builder, the missing security headers of any public app with the exact header lines for its host; and, for an app you prove you own, exposed files such as .env and .git, keys in the JavaScript it ships, and whether its Supabase tables or Firebase database, Firestore and storage rules let anyone read them. It reads only status codes and Supabase row counts, never a row's data, and never writes.

What it does not do: no speed test, no rankings or traffic, no JavaScript, no sign-in, no page text returned, and no private, local or internal addresses. It does not request a page that robots.txt disallows for Agent Tools. robots.txt is a request, not access control, and a firewall can treat real crawlers differently.

It stores nothing about you or the site. Answers are cached for five minutes, except owner checks (exposure_check, ai_app_check), which are never cached; the only lasting record is a daily count of calls per tool.

What it can do

• Shows whether robots.txt allows or blocks 14 published AI crawlers
• Shows the rule that decided each crawler, the sitemaps and any noindex or noai directives
• Reads a page's title, description, canonical link, language and icons
• Reads Open Graph and Twitter share tags and lists the common ones that are missing
• Follows the redirect chain with each status code and shows the final address and headers
• Checks one product page's title, description, h1, canonical link and Product structured data (price, GTIN, availability)
• Checks a domain for llms.txt, agents.md, the Shopify UCP file, a sitemap and AI crawler rules in robots.txt
• Checks a page for five AI citation signals: summary, author credentials, Q&A format, headings, structured data
• Checks up to 40 sitemap addresses for broken pages and redirect chains, skipping pages robots.txt disallows
• Checks a landing page's h1, call to action, form fields, trust signals, listed prices, viewport and contact path
• Tests up to five call-to-action links of a page and gives each finding with evidence and a fix
• Checks the exposed surface of a domain its owner proves: leaked files, dangling subdomains, default pages
• Finds API keys in the JavaScript an owned app ships, public source maps and open admin paths, keys masked
• Flags 6 legal signals on an owned app (COPPA, fonts, session replay, CAN-SPAM, renewals, DMCA) with sources
• Checks 8 security headers of any public app and gives the exact lines for Vercel, Netlify, Cloudflare and others
• Tells an owner whether anyone can read their Supabase tables or Firebase data, by status and row count only
• Refuses private, local and internal addresses and redirects into them

Tools

• check_ai_crawler_access, Check AI crawler access in robots.txt. Checks whether 14 AI crawlers may fetch a page. Use it when the user asks whether AI crawlers can read a website or page, for example "can AI crawlers read example.com?", "is GPTBot blocked in my robots.txt?" or "which AI bots are allowed on https://example.com/blog?". Pass the page address. Returns, for 14 AI crawlers, whether robots.txt allows the page and the rule that decided it, plus the sitemaps, the redirect chain, and the page's X-Robots-Tag header and robots meta tag. It reads robots.txt only and does not test whether a firewall blocks crawlers. Do not use it for private or local addresses, for security scans, or to collect a site's content.
• check_page_tags, Check Open Graph and share tags. Reads title, description and share tags of a page. Use it when the user asks about the meta tags, canonical tag or Open Graph social preview of a web page, for example "what will the preview look like when I share this page?", "does my homepage have a meta description and canonical tag?" or "which Open Graph tags is my page missing?". Pass the page address. Returns the title, meta description, canonical link, language, viewport, icons, Open Graph and Twitter tags, and a list of the common tags that are missing. Tags added by JavaScript are not seen. Do not use it to read a page's text or contact details, for private or local addresses, or for rankings and traffic.
• trace_redirects, Trace a 301 redirect chain. Traces the redirects and headers of a web address. Use it when the user asks where a web address redirects to, or for a 301 redirect chain, for example "why does http://example.com end up on another address?", "show the redirect chain for this link" or "does this site redirect http to https?". Pass the address. Returns each hop with its status code and target, the final address and status, whether http is upgraded to https, and the main response headers such as cache-control and strict-transport-security. At most 6 redirects are followed. Do not use it for private or local addresses, to read page content, or to scan a site for vulnerabilities.
• product_page_seo, Check product JSON-LD and page SEO. Checks the SEO and Product data of a product page. Use it when the user asks whether one product page of an online store is set up for search, for example "is https://shop.example.com/products/blue-mug set up for search?", "does my product page have Product structured data?" or "why does Google show no price for this product?". Pass the product page address. Returns title and meta description length, h1 headings, canonical link, Open Graph tags, and the Product structured data (JSON-LD): name, brand, SKU, GTIN, MPN, price, currency, availability, variant count and rating summary, plus a list of issues found. On Shopify it adds the product handle and whether a variant parameter is in the address. Works on any store. It reads one page, skips it if robots.txt disallows it for Agent Tools, and does not run JavaScript. Do not use it for page speed, rankings or traffic, to collect prices or catalogs, or for private or local addresses.
• ai_readiness_check, Check AI readiness of a store or page. Checks if AI agents can read and cite a store or page. Use it when the user asks whether AI agents can read an online store or cite a page, for example "does shop.example.com have an llms.txt?" or "will AI assistants cite this page?". Pass url for a page or domain for the domain files only; one is enough. Returns whether the domain has /llms.txt, /agents.md, the UCP file at /.well-known/ucp (version and services shown) and /sitemap.xml, and which of 14 AI crawlers robots.txt blocks site-wide. With a page address it also rates five citation signals (summary near the top, author credentials, Q&A format, heading structure, structured data) as pass, warn or fail with evidence and a fix. It requests at most five domain files and one page, skips what robots.txt disallows for Agent Tools, and shows what is published, not whether an agent can buy. Do not use it for private or local addresses, to read page content, or to scan for vulnerabilities.
• link_sample_check, Check a sitemap sample for broken links. Finds broken or redirected pages in a sitemap sample. Use it when the user asks whether pages of an online store or website are broken or redirected, or for a broken link in the sitemap, for example "are any pages of shop.example.com broken?", "check my sitemap for 404s" or "do my product pages redirect?". Pass the domain. Reads the sitemap (for a sitemap index, the product, page, collection and blog sitemaps first) and checks up to 40 addresses spread across it, on that host only, returning counts by status, and for each broken, unreachable or redirected address its status and final address. It is a sample, not every page; it uses HEAD requests at low concurrency, skips addresses that robots.txt disallows for Agent Tools, and makes at most 46 requests. Do not use it for private or local addresses, to read page content, or to monitor a site over time.
• landing_page_check, Check a landing page for conversion. Checks a landing page for conversion problems. Use it when the user asks how to get more sign-ups, sales or enquiries from a landing page, for example "review https://example.com/ for conversion" or "does my landing page have a clear call to action?". Pass the page address. Returns twelve checks, each with a status, evidence and a fix: the h1 and the sentence under it, the primary call to action and its repeats, whether up to five call-to-action links work, form field count, trust signals, a visible price or next step, the viewport tag, HTML size, render-blocking hints and a contact path. It reads one page, tests a few links, skips what robots.txt disallows for Agent Tools, does not run JavaScript, and returns no page copy, reviews or contact details. It is a checklist, not a conversion forecast, a speed test or a design review. Do not use it for private or local addresses, rankings or traffic.
• exposure_check, Check the exposed surface of a site you own. Checks what a site or app the user owns leaks publicly. Use it when the user asks what an attacker could find on their own domain, host or app, for example "does my app leak a .env file?". Pass the domain, host or app page address, not a bare IP. The first call returns ownership_not_proven with a DNS TXT record or file: publish it, then call again with the same target. Then it gives exposed or clean, the top three risks with fixes, and evidence: dangling subdomains, leaked files (never their contents), masked API keys, source maps, admin paths, dev servers and default pages. It adds six legal signals with law and source (COPPA, Google Fonts, session replay, renewals, DMCA, CAN-SPAM for a passed email); not legal advice. Findings are labeled confirmed, needs verification or rejected. GET only, at most 32 fixed paths; no crawl, sign-in or port scan. Do not use it on a target the user does not own, to look up a person, user name or email, or for private or local addresses.
• ai_app_check, Check an AI-built app for leaks and open data. Checks an app made with an AI app builder (Lovable, Bolt, v0, Replit, Cursor) for leaks and launch readiness. Use it when the user asks "is my Lovable app secure?", "is my Supabase data public?", "which security headers is my app missing?" or "is my app ready to launch?". Pass the app address. For any public app it returns 8 security headers with the exact lines for the detected host (Vercel, Netlify, nginx and more), and launchList: 20 items of a 160-item launch list read from outside (renewals, mail records, security.txt, caching, legal links), each pass, fail or warn with a fix; checklist true adds the owner-only items with how to check each. The first call also returns ownership_not_proven with a DNS TXT record or file: once the owner publishes it, the next call adds exposed files (.env, .git), masked keys in the shipped JavaScript, source maps, open Supabase or Firebase data, and backend signals. GET only; database checks read status codes and row counts, never a row's data, and never write. Do not use it on an app the user does not own beyond the public checks, for private or local addresses, or to scan many sites.

Add Site SEO & AI Crawler Audit in VS Code

VS Code reads MCP servers from mcp.json and offers their tools to Copilot Chat in Agent mode. A workspace file at .vscode/mcp.json applies to that project. For every workspace, run MCP: Open User Configuration from the Command Palette and add the same entry there. These steps were checked against the VS Code MCP servers guide on 2026-10-09.

Create or edit the file so it contains this server. VS Code uses a top-level servers object, not mcpServers. Merge it with servers you already have.

{
  "servers": {
    "site-check": {
      "type": "http",
      "url": "https://sitecheck.openkrill.app/mcp"
    }
  }
}

No command, no headers and no input variables for a key: Site SEO & AI Crawler Audit answers without a login. Save the file and select Start above the entry. VS Code asks once whether you trust the server: confirm it. Run MCP: List Servers to check that Site SEO & AI Crawler Audit is running. You should see check_ai_crawler_access, check_page_tags, trace_redirects, product_page_seo, ai_readiness_check, link_sample_check, landing_page_check, exposure_check, ai_app_check.

Open the Chat view, choose Agent mode and ask: Will AI assistants cite https://example.com/blog/guide? The Configure Tools button in the chat input lists the tools of each server, and VS Code may ask you to confirm the first call. The server is public and read-only. It sees only the arguments of each call, so do not paste secrets into the question.

Command line users can add it to the user profile in one step with code --add-mcp and the same name, type and url as JSON. The portable .mcp.json at the project root also works in VS Code, with mcpServers as its top-level key. An organization can turn MCP off through Copilot policy: if the MCP commands are missing, ask the admin.

If it does not work

A call checked on 2026-10-17

example.com is a public page. The tool reads the head only. The request below was posted to https://sitecheck.openkrill.app/mcp as tools/call. HTTP 200. Source: the Site SEO & AI Crawler Audit server, read 2026-10-17.

{
  "method": "tools/call",
  "params": {
    "name": "check_page_tags",
    "arguments": {
      "url": "https://example.com"
    }
  }
}

• status. 200
• source. The website itself, requested from Agent Tools servers
• notice. This shows what the site publishes to a request from Agent Tools servers at this moment. Firewalls and CDNs can treat real crawlers or visitors differently, and robots.txt is a request that crawlers follow by choice, not
• url. https://example.com/
• finalUrl. https://example.com/

Repeat the call yourself if you need a newer reading. Cached answers expire. A rate limit is not a result: wait and try again. Nothing in the call is a ranking, a filing, or advice.

Using it in production

Nothing to install and nothing to run: no package, no Python or Node.js version and no environment variable. Site SEO & AI Crawler Audit runs on our servers, and your client only needs the URL https://sitecheck.openkrill.app/mcp.

Use it in other apps

• Site SEO & AI Crawler Audit in ChatGPT
• Site SEO & AI Crawler Audit in Claude
• Site SEO & AI Crawler Audit in Cursor
• Site SEO & AI Crawler Audit in Pi

Roles that use Site SEO & AI Crawler Audit

• SEO specialist guide

Frequently asked questions

Words on this page

• MCP (Model Context Protocol) definition
• Streamable HTTP transport definition
• Tool primitive definition
• Rate limit policy definition

Source: plugin listing for site-check in this repository, and a live tools/call on 2026-10-17. As of 2026-10-17.