Can ChatGPT and Claude see my JavaScript website?
Only the text that is already in the HTML your server sends. A December 2024 study of Vercel's network found that the crawlers of OpenAI, Anthropic and Perplexity download JavaScript files but do not run them. If your pages are built in the browser, as many React and AI-built sites are, these crawlers see an almost empty page.
Facts
- Vercel reported on December 17, 2024 that none of the major AI crawlers, including GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot and PerplexityBot, render JavaScript. (Vercel: The rise of the AI crawler (Dec 2024))
- The same study found that Gemini, which uses Googlebot's infrastructure, and AppleBot do render JavaScript. (Vercel: The rise of the AI crawler (Dec 2024))
- In one month on Vercel's network, ChatGPT's crawler fetched JavaScript files in 11.50% of its requests and Claude's in 23.84%, without executing them. (Vercel: The rise of the AI crawler (Dec 2024))
- OpenAI says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, so the page that crawler reads is the one ChatGPT can cite. (OpenAI: Overview of OpenAI crawlers)
- In the Next.js App Router, layouts and pages are Server Components by default and are prerendered to HTML on the server. (Next.js docs: Server and Client Components)
How do I test whether AI crawlers can read my site?
Fetch a page the way a crawler does, without a browser, and look for its main text in the result. If the price, the headline or the answer is missing, AI crawlers are not seeing it.
# What an AI crawler receives: the raw HTML, no JavaScript run curl -s -A "GPTBot" https://www.example.com/pricing | grep -i "your price or headline" # Nothing found? The text only exists after JavaScript runs.
Which kinds of sites have this problem?
Sites whose content is created by JavaScript in the browser after the page loads:
- Single-page apps built with React, Vue or Angular and Vite, with no server rendering
- Sites generated by AI app builders that ship a client-rendered app
- Pages that load their text from an API after the first paint
- Menus and links made of click handlers instead of real <a href> links
What is the fix for each setup?
The fix is to send the finished text in the first HTML response.
| Setup | What to ask the coding agent |
|---|---|
| Next.js App Router | Keep pages as Server Components; mark only interactive parts with "use client" |
| React or Vue with Vite | Prerender the public pages to static HTML at build time, or move them to a framework with server rendering |
| Content loaded from an API | Fetch it on the server or at build time, not in useEffect |
| Any framework | Real <a href> links, a sitemap.xml and the text visible without clicks |
Does Google have the same problem?
Less so: Google's crawler runs JavaScript, so Google Search and Gemini can see client-rendered text, though later than plain HTML. ChatGPT, Claude and Perplexity are the ones that read only the raw HTML.
How does easyIndex check this?
easyIndex reads each published page itself, the way an AI crawler does, without running JavaScript. A page your coding agent builds counts as done only when the audited text is in that raw HTML on the live site.
Step by step
- Test your key pages. Run the curl test on the home, pricing and main service pages and search for their main text.
- Find where the text is built. Ask your coding agent which pages render their content in the browser and why.
- Move the text to the server. Have the agent switch those pages to server rendering or static prerendering, keeping interactivity only where it is needed.
- Fix links and the sitemap. Make every internal link a real <a href> and list every public page in sitemap.xml.
- Test again on the live site. Repeat the curl test after the deploy; the text must appear in the raw HTML.
Common mistakes
- Checking the page in a browser, which runs JavaScript, and assuming crawlers see the same.
- Server-rendering the home page but leaving pricing and service pages client-rendered.
- Blocking AI crawlers by accident while fixing rendering, in robots.txt or the CDN.
- Paying for a prerender layer when the framework already renders on the server.
How easyIndex solves it
Your coding agent fixes the rendering in your repository, guided over MCP; easyIndex reads each page live without running JavaScript, as AI crawlers do, and then records which pages AI crawlers actually read. US$ 30 per website a month, with a 7-day free trial.
Frequently asked questions
Is a Next.js site safe by default?
Mostly: App Router pages are server-rendered by default, but text fetched inside a Client Component after loading is still invisible to AI crawlers.
Does an llms.txt file solve the JavaScript problem?
No. It is a short text file; the pages themselves still need their text in the HTML for crawlers to read and quote them.
Will AI crawlers start running JavaScript?
Nobody can say. As of the December 2024 study they did not, and the safe choice is to send the text in the HTML.
Does this affect visitors who come from ChatGPT?
No. People open your site in a normal browser that runs JavaScript; the problem is only what the crawler reads before citing you.
Continue reading
- How does easyIndex work, what do I need, and how long until results?
- How much does easyIndex cost?
- Which AI crawlers exist and how do I allow them in robots.txt?
- Does llms.txt work?
- How do I do GEO with Claude Code or Codex?
- Is Cloudflare blocking ChatGPT and other AI crawlers from my site?
Last verified on .