A site crawler that finds the pages quietly telling Google to stay away.
Prism fetches, checks and Lighthouse-scores every page, and reads everything each page tells Google, not just what it shows people.
The same checks, every time.
Every page the crawler touches gets the same treatment, not a sample.
- Can Google have it
- The status it returned, where its redirects end up, what its canonical points at, and whether the page is telling Google to index it in one place and to stay away in another
- Is it worth having
- Length and substance, near-duplicates of other pages on the site, and titles and meta descriptions that are missing, cut off or reused
- Can anything reach it
- What links to it from inside the site, what it links out to and whether those still resolve, and whether anything links to it at all
- How fast is it
- Lighthouse runs against the page and returns performance, accessibility, best-practices and SEO scores, so a slow template comes back as a list of URLs
- How does it present
- Open Graph and Twitter card tags for when somebody shares it, and structured data checked for the errors that stop it being used
A page can look indexable and quietly not be.
One stray rule can pull a healthy-looking page out of Google, and nothing in the browser shows it. Prism reads the response headers as well as the markup, so the contradiction arrives as a finding with a URL list attached.
- Caught, with the URLs
Every page saying index in one place and noindex in another, listed. Usually a CDN rule or stray middleware.
- See what a release changed
Crawl Comparison sets two runs side by side: pages, health score and every issue type, diffed.
- Lighthouse in a real browser
Each page is scored on what it renders; the crawl checks read the HTML your server returned.
Frequently asked questions
What does the crawler check?
Every page gets the same treatment: the status it returned and where its redirects end up, what its canonical points at, whether the robots meta tag and the X-Robots-Tag response header agree, how substantial the content is and whether another page duplicates it, what links to it and what it links out to, whether its images carry alt text, whether it has the tags that make it shareable, and how it scores when Lighthouse runs against it.
Why do pages silently drop out of Google?
Often because of an instruction nobody can see. A page can return 200, say "index, follow" in its markup, and still carry "noindex" in its X-Robots-Tag response header, which never shows up in view-source. Prism records both, so the contradiction is a finding with a URL list rather than a mystery.
Can I compare two crawls?
Yes. Because every run does the same work on every page, Crawl Comparison can set two runs side by side and show the movement in page count, health score and every issue type between them.
Does it render JavaScript?
Lighthouse runs against each page in a real browser and scores what it renders. The crawl checks themselves read the HTML your server returns, headers included, so those findings describe exactly what the server served. Content drawn only by a client-side fetch shows up in the Lighthouse scores rather than the parsed crawl record.
Elsewhere in Prism
After the crawl finishes
- Site Health 14 reports over one crawl, including Fix First, which sorts findings by their effect on traffic.
- Data warehousing Classification needs Search Console. Without it, findings list by severity and are labelled unpriced.
- AI answer monitoring Page Intel crawls the pages the answer engines keep citing and scores them the same way.
More features
- Data warehousing Google serves a rolling 16 months and drops the rest. Prism syncs Search Console into its own store daily, re-pulls the days Google revises, and keeps all of them.
- AI answer monitoring Prompts put to ChatGPT, Claude, Gemini and Perplexity on a schedule, the complete answer kept, so every report number traces back to the sentence it came from.
- Smart Campaigns Define a campaign by URL and query pattern plus dates, and Prism tracks the group as one thing. The indexed view rebases each campaign to 100 so big and small compare.
- Segments A saved query or URL group built from plain rules. Pick one in the toolbar and every Search Performance report stays scoped to it. Brand vs non-brand in one click.
- Keyword cannibalization Find the queries where two of your own pages split the impressions, with each contender’s share and the best-positioned page named as the one to consolidate into. Read from your own Search Console data.
- Striking distance keywords Queries averaging positions 11 to 20 with real impression volume, ranked by what page one would be worth against your own click-through curve. No industry CTR table, no purchased benchmark.
- Internal link analysis Your internal links as a graph: link equity per page, orphans, click depth, the anchor text you actually use, and pages that rank for the same query with no link between them.
Run the first crawl and read the findings.
Point Prism at a site: every page crawled and Lighthouse-scored, and every finding classified by what it does to your traffic, on the free plan, for ten properties.