AIO Copilot

Technical SEO Audit: A 15-Point Checklist in the Order Problems Block Rankings

Crawling first, indexing second, everything else after. Each check below says what to look for, how to test it for free, and what Google's own documentation says, because a good share of the audit advice in circulation contradicts it or describes tools Google has retired.

Updated 2 November 202619 min read

What is a technical SEO audit?

A technical SEO audit checks whether search engines can crawl a site, index the right version of each page and see what users see, then lists the faults in the order they cost traffic. The order matters more than the length of the list. A page Google cannot crawl, or has decided not to index, gains nothing from a faster load time or a tidier schema block.

The 15 checks below run in that order, from crawling to AI crawler access, and our glossary entry on technical SEO covers the vocabulary. Most of the evidence comes from one free source, Google Search Console, whose reports record what Google actually did with the site. A crawler (Screaming Frog, Sitebulb or a cloud suite) shows what is there to be found. A sound audit uses both and says which one each finding came from.

Can Google crawl the site? Robots.txt and server errors

Start with robots.txt, because one wrong line can stop Googlebot fetching an entire site. The file must sit in the top-level directory, and its rules apply only to the host, protocol and port it is served from, according to Google's robots.txt documentation. A file on www.example.com says nothing about shop.example.com.

The format is an internet standard, RFC 9309, published by the IETF in September 2022, and Google adds details that matter in an audit. It reads only the first 500 kibibytes of the file, generally caches it for up to 24 hours (so a fix is not seen at once) and ignores crawl-delay. It also reacts to the server's answer: a 4xx other than 429 is treated as if no robots.txt existed, so everything may be crawled, while a server error makes Google stop crawling the site for the first 12 hours as it retries the file.

Two misreadings do most of the damage. Robots.txt controls crawling, not indexing: Google says a blocked page can still appear in search results, without its content, if other pages link to it. And it is not security. The RFC says its rules "are not a form of access authorization", and listing a private path in the file advertises it.

To check it, paste the file into the free robots.txt validator to catch syntax errors and rules that block more than intended. Then open Search Console's robots.txt report, under Settings, which shows the files Google found for the top 20 hosts on a Domain property, when it last fetched each and any parsing errors. After removing a bad rule, use the same report to request a recrawl of the file.

One newer limit belongs here. Googlebot reads only the first 2MB of an HTML file, measured uncompressed, and fetches each referenced script and stylesheet separately under the same cap, according to Google's Googlebot documentation. A page that inlines megabytes of JSON or CSS can lose whatever sits past that point. Crawl budget, by contrast, rarely matters: Google's guide to it is written for sites with more than a million pages that change weekly, or more than 10,000 that change daily.

Is Google indexing the right pages? Noindex, canonicals and the Page indexing report

Open the Page indexing report next. It groups every URL Google knows about but has not indexed by the reason, which makes it the most direct diagnosis an audit gets. Google's help page for the report also sets the expectation: "Don't expect every URL on your site to be indexed." The job is to make sure the pages meant to rank are in, and the pages left out are left out on purpose. The reasons that matter most:

  • "Crawled - currently not indexed": Google fetched the page and chose not to index it. It "may or may not be indexed in the future; no need to resubmit this URL for crawling." Since resubmitting is pointless, the useful responses are to improve the page, merge it into a stronger one, or link to it from relevant pages.
  • "Discovered - currently not indexed": Google knows the URL but has not crawled it. On a very large site a big share here is the crawl budget signal Google's guide describes; on a small one, check the page is linked internally and listed in the sitemap.
  • "Duplicate without user-selected canonical" and "Duplicate, Google chose different canonical than user": Google sees the page as a copy. See the duplicates check below.
  • "URL marked 'noindex'": confirm each one was deliberate.
  • "Indexed, though blocked by robots.txt": the robots.txt misreading above, in the wild.

For noindex to work, Google must be allowed to crawl the page, since a URL blocked in robots.txt never shows Googlebot the rule. Put it in a robots meta tag in the original HTML (or an X-Robots-Tag header for PDFs and other files), not in JavaScript: Google's JavaScript guidance says that on meeting a noindex tag it may skip rendering, so a script that changes the tag may never run. Google also honours robots meta tags placed in the body, so a stray tag injected by a widget can deindex a page.

Canonical tags come next. Google treats a redirect and a rel="canonical" link as strong signals and sitemap inclusion as a weak one, and its canonicalization page says "indicating a canonical preference is a hint, not a rule." URL Inspection shows the canonical you declared beside the one Google selected. Where they differ on an important page, the declared canonical usually points at a URL that redirects, is noindexed or carries different content. The free canonical tag checker flags missing, duplicated and relative canonical tags on a page.

Can Google find every page that matters? Site architecture and internal links

Google finds most pages by following links, and it follows one kind reliably. Its link guidance says "Google can only crawl your link if it's an <a> HTML element" with an href attribute. A span carrying an href, or an anchor that navigates only through an onclick handler, may be missed. The same page sets the coverage rule, "Every page you care about should have a link from at least one other page on your site", and says there is "no magical ideal number of links" a page should carry.

So the audit checks three things. Orphans first: export the URLs a crawl found and compare them with the sitemap and with the pages Search Console reports; any URL that appears only in the sitemap has no internal link pointing at it. Then links that are not links, which Screaming Frog added a "Pages With Uncrawlable Internal Outlinks" filter for in version 24.0 in May 2026. Then whether the pages that earn money are linked from the pages that attract visits, with anchor text that says what the target is, as Google's guidance asks.

Google publishes no click-depth rule, so treat "three clicks from the homepage" as a usability heuristic rather than a ranking requirement. For one page, the free internal link analyzer lists every link with its anchor text and nofollow status. For the shape of the whole site, our guide to topic clusters explains how to give each topic one owner page that the related pages link to.

Which duplicate URLs are competing with each other?

Duplicate content is not a penalty. Google's canonicalization documentation says "some duplicate content on a site is normal and it's not a violation" of its spam policies. The cost is dilution: signals split across several URLs, and Google may pick a version you did not want, which the Page indexing report shows as "Duplicate, Google chose different canonical than user".

The usual causes are mechanical. HTTP and HTTPS both answer, www and bare domain both resolve, URLs exist with and without a trailing slash or in mixed case, tracking and sort parameters create copies, and faceted navigation mints a URL for every filter combination. A crawl sorted by title or by content finds most of them. Screaming Frog's near-duplicate check, set under Config > Content > Duplicates, compares page text with a minhash algorithm at a 90% match by default, according to its user guide.

Fix each with the strongest signal that fits: a 301 redirect where the duplicate has no reason to exist, a canonical tag where it does (a filter users need, for instance). Google's guide to consolidating duplicates rules out three shortcuts. Robots.txt hides the content but Google may still index the URL. Noindex "will completely block the page from Search" rather than consolidating it. The URL removal tool hides every version of the URL. Never point different methods at different canonicals for the same page.

Two distinct pages chasing the same queries is a different problem, keyword cannibalization, and a content decision rather than a technical one.

Do redirects resolve in one hop?

Every redirect should go straight to its final URL with a permanent status code. Google's documentation on status codes says it treats 301 and 308 as a strong signal that the target should be the URL indexed, and 302, 303 and 307 as a weak one, so a permanent move made with a 302 leaves Google guessing. Its crawlers follow up to 10 redirect hops, and well before that limit chains cost crawling: the crawl budget guide says to "avoid long redirect chains, which have a negative effect on crawling."

Look for chains (A to B to C), loops, redirects that end at a 404, internal links still pointing at redirected URLs, and HTTPS pages that redirect back to HTTP. After a migration, check the old URLs that had traffic first: export the top pages from Search Console's Performance report for the months before the move and run that list through a crawler in list mode. The free redirect chain checker takes a chain you type in, hop by hop, and flags loops, extra hops, HTTPS downgrades and 302s that should be 301s; to find the chains in the first place, use a crawler or URL Inspection.

Do pages return the right status codes? 4xx, 5xx and soft 404s

A live page should return 200, a removed one 404 or 410, and a struggling server should say so with a 5xx or 429. Google treats every 4xx except 429 alike: a previously indexed URL is removed from the index and crawled less often. Server errors and 429 make Google slow its crawling; indexed URLs are kept for a while, but URLs that "persistently return a server error" are dropped, according to the same status code documentation.

Soft 404s are the quiet version: a page that returns 200 but shows an error message, an empty template or nothing at all. Search Console lists them in the Page indexing report, and the crawl budget guide notes that soft 404 pages "will continue to be crawled, and waste your budget." Typical sources are product pages emptied when stock runs out, empty category or internal search pages, and single-page apps that show a "not found" view on a URL returning 200. For those apps Google's JavaScript guidance offers two fixes: redirect to a URL that returns a real 404, or add a noindex robots tag to the error view.

Check the Crawl Stats report's breakdown by response for spikes in 404s and 5xx, then run URL Inspection on any URL that looks wrong.

Is the XML sitemap clean, and within Google's limits?

A sitemap should list only canonical, indexable URLs you want found, written as absolute URLs. Google's sitemap documentation limits each file to 50,000 URLs or 50MB uncompressed; larger sites split into several files listed in a sitemap index. Google says it "ignores <priority> and <changefreq> values" and uses <lastmod> only when it is "consistently and verifiably" accurate, so a CMS that stamps every URL with today's date teaches Google to ignore the field.

Submitting a sitemap "is merely a hint" and does not guarantee crawling. Reference it in robots.txt and submit it in Search Console. Do not ping: Google deprecated its sitemap ping endpoint in June 2023 and said it would stop working within six months. The free sitemap validator checks the format, duplicate URLs and the 50,000-URL limit. Comparing the sitemap with a crawl catches the rest: URLs that redirect, return errors or carry noindex, none of which belong in it.

Does Google see the content after JavaScript runs?

Google processes JavaScript sites in three phases, crawling, rendering and indexing, and queues every page that returns 200 for rendering, where it can wait a few seconds or longer. In December 2025 Google added that pages returning other status codes might not be rendered at all. Links only count if they are real <a href> elements, and single-page apps should route with the History API rather than URL fragments.

The test is URL Inspection. Run a live test, open the tested page and read the rendered HTML for the text, links, canonical and structured data that matter; if they are missing there, Google does not have them. Google's JavaScript guidance names the Rich Results Test as the other way to see rendered HTML. It still recommends server-side rendering or pre-rendering, because it "makes your website faster for users and crawlers" and not every bot can run JavaScript.

The faults that turn up most often: navigation built from buttons, content that loads only after a click or scroll, canonical or robots tags rewritten by script, and client-side routes that return 200 for pages that do not exist.

Does the mobile version carry everything? Mobile-first indexing

Google indexes and ranks the mobile version of every site. It announced on 31 October 2023 that "the trek to Mobile First Indexing is now complete", and since 5 July 2024 it has crawled sites for Search with Googlebot Smartphone, warning that a site whose content "is not accessible at all with a mobile device" will "no longer be indexable".

The audit check is parity. Google's mobile-first documentation asks for the same content, structured data, title and meta description, headings, image alt text and robots meta tags on mobile as on desktop. It also says Google "won't load content that requires user interactions (for example, swiping, clicking, or typing) to load", so primary content lazy-loaded on a tap is invisible to indexing.

Google retired its Mobile-Friendly Test and the Mobile Usability report on 1 December 2023, so an audit that cites either is out of date. Use URL Inspection's smartphone render and Lighthouse in Chrome DevTools instead, and the free mobile-friendly checker to review one page's viewport tag, tap targets and font sizes from its HTML.

Does the site pass Core Web Vitals? Current metrics and thresholds

Core Web Vitals are three metrics, each judged at the 75th percentile of real page loads and segmented by mobile and desktop, per web.dev. Interaction to Next Paint replaced First Input Delay as the responsiveness metric on 12 March 2024, so any audit still reporting FID is out of date. The thresholds, from Google's PageSpeed Insights documentation:

MetricMeasuresGoodNeeds improvementPoor
Largest Contentful Paint (LCP)When the main content has loaded2.5 s or lessOver 2.5 s, up to 4 sOver 4 s
Interaction to Next Paint (INP)How fast the page responds to taps, clicks and keys200 ms or lessOver 200 ms, up to 500 msOver 500 ms
Cumulative Layout Shift (CLS)How much the layout jumps while loading0.1 or lessOver 0.1, up to 0.25Over 0.25

How much they matter is easy to overstate. Google's page experience documentation says "Core Web Vitals are used by our ranking systems", and also that "Google Search always seeks to show the most relevant content, even if the page experience is sub-par." Schedule a failing template by the traffic it carries, not as an emergency, unless visitors are visibly suffering.

The numbers that count are field data from the Chrome UX Report, which PageSpeed Insights reports over the previous 28 days. Lighthouse lab scores help find causes, but they come from one simulated load on one device. To see where a measured value falls, enter it in the free Core Web Vitals calculator; to find likely causes in a page's markup (render-blocking scripts, unsized images, missing lazy loading), use the page speed analyzer.

Is every page served over HTTPS?

Every URL should load over HTTPS, and every HTTP URL should 301 to its HTTPS twin. When Google started using HTTPS for ranking in August 2014, it called it "a very lightweight signal" affecting "fewer than 1% of global queries". Its larger effect today is on canonicalization: Google lists whether a page is served over HTTP or HTTPS among the factors it uses to choose a canonical, so a site that answers on both splits its signals.

Check for HTTP URLs in internal links, canonical tags, hreflang and sitemaps; pages that load scripts, styles or images over HTTP; expired or mismatched certificates; and any redirect that goes from HTTPS to HTTP. Google's page experience checklist asks about HTTPS, not about security headers, so treat headers such as HSTS and Content-Security-Policy as security work rather than SEO; the free security headers analyzer grades them from a page's response headers.

Is the structured data valid, and which rich results still exist?

Structured data has to describe what is on the page. Google's structured data policies say markup "must be a true representation of the page content" and forbid marking up content readers cannot see. A breach can bring a manual action that removes rich result eligibility, though not ranking. Valid markup carries no promise either: "Google does not guarantee that your structured data will show up in search results."

The audit has two parts. Validate each template with the Rich Results Test and watch Search Console's rich result reports after every deployment. Then check the markup is for something Google still shows, because the list keeps shrinking:

  • FAQ rich results were limited to "well-known, authoritative government and health websites" in August 2023, then stopped appearing in Google Search for everyone from 7 May 2026.
  • HowTo rich results disappeared on 13 September 2023, according to an update to the same August 2023 post.
  • Google began removing the sitelinks search box on 21 November 2024.
  • Course Info, Claim Review, Estimated Salary, Learning Video, Special Announcement and Vehicle Listing were phased out after Google's June 2025 announcement. Book Actions, on the same list, was later kept because a Search feature still uses it, Google's documentation changelog noted in November 2025.
  • Practice problem documentation was removed in January 2026, and Dataset markup is now used only by Dataset Search, not Google Search, per the same changelog.

None of this means tearing markup out. Google said there was "no need to proactively remove" FAQ markup, and that unsupported structured data "won't cause issues in Search". It does mean that adding FAQ or HowTo markup to win a rich result is wasted effort. Spend it on the types in Google's search gallery; our schema markup guide covers the ones most sites need, and the free schema markup generator writes JSON-LD for Article, Product, LocalBusiness, BreadcrumbList and Organization.

Are hreflang annotations reciprocal and valid?

Hreflang tells Google which language or regional version of a page to show, and it breaks in predictable ways. Google's localized versions documentation accepts it in HTML link elements, HTTP headers or the XML sitemap, and sets the rules: "Each language version must list itself as well as all other language versions", and "If two pages don't both point to each other, the tags will be ignored." URLs must be fully qualified. Codes must be ISO 639-1 languages, optionally followed by an ISO 3166-1 Alpha 2 region, so en-GB works and en-UK does not. The x-default value marks the fallback page.

Audit it with a crawler that reports missing return links and alternates that do not return 200, and confirm every hreflang URL is the canonical version, since an alternate pointing at a redirected or noindexed URL is wasted. Google also says it "doesn't use hreflang or the HTML lang attribute to detect the language of a page", so the text itself has to be in the language the tag claims.

What do server logs show about how Googlebot crawls the site?

Server logs record every request a crawler made, which answers questions no other source can: where Googlebot spends its time, whether it fetches parameter URLs you thought were blocked, how fast it reaches new pages and what status codes it actually received. The free summary is Search Console's Crawl Stats report (Settings, then Crawl stats), which shows total requests, download size, average response time and host status, broken down by response code, file type, purpose (discovery or refresh) and Googlebot type. It is available only for root-level properties.

Before trusting raw logs, verify the bot, because user-agent strings are easy to fake. Google's verification method: run a reverse DNS lookup on the IP, confirm the hostname ends in googlebot.com, google.com or googleusercontent.com, then run a forward lookup on that hostname and confirm it returns the same IP; or match the IP against Google's published ranges. Screaming Frog's Log File Analyser verifies bots automatically; its free version reads 1,000 log lines in one project, and a licence is £99 a year (Screaming Frog's site, November 2026).

Look for a large share of requests to URLs that should not be crawled (filters, internal search, session parameters), important templates crawled rarely, and spikes in errors or response time, which make Google slow down.

Can AI crawlers reach the site, and which ones should?

Treat AI crawler access as separate switches: usually one for appearing in a company's search answers and one for model training. Blocking the wrong one removes a site from an AI search product while doing nothing about training, or the reverse.

For Google there is no separate AI crawler to allow. AI Overviews and AI Mode draw on pages Googlebot crawls, and a page must be "indexed and eligible to be shown in Google Search with a snippet" to be a supporting link, according to Google's page on AI features and your website. The snippet controls apply: nosnippet stops content being used "as a direct input for AI Overviews and AI Mode". Since 31 August 2026 every site also has a Search generative AI setting in Search Console that takes it out of AI Overviews and AI Mode, generally within one to two days, and which Google says is not a ranking or inclusion signal for the rest of Search.

Google-Extended is a robots.txt token, not a crawler: it "doesn't have a separate HTTP request user agent string", according to Google's crawler documentation. It governs whether content Google crawls may train future Gemini models and ground answers in Gemini Apps and Vertex AI, and it "does not impact a site's inclusion in Google Search nor is it used as a ranking signal". Blocking it is not the way to leave AI Overviews; the Search Console setting is.

CompanySearch and answersTrainingFetches when a user asks
GoogleGooglebot, plus the Search generative AI settingGoogle-Extended (robots.txt token)User-triggered fetchers such as Google-NotebookLM
OpenAIOAI-SearchBot: blocked sites are not shown in ChatGPT search answersGPTBotChatGPT-User: robots.txt rules "may not apply"
AnthropicClaude-SearchBotClaudeBotClaude-User
PerplexityPerplexityBot, which is not used to train foundation modelsNone listedPerplexity-User: "generally ignores robots.txt rules"

Sources for the table: OpenAI's crawler documentation, which says each setting "is independent of the others" and that ChatGPT search takes about 24 hours to reflect a robots.txt change; Anthropic's help page, which says blocking Claude-SearchBot or Claude-User may reduce a site's visibility in Claude's search results; Perplexity's crawler page; and, for Google's fetchers, its documentation changelog.

Check the CDN and firewall as well as robots.txt, because a bot-management rule can block a crawler the file allows. For a business that wants to be recommended, the sensible default is to allow the search crawlers and treat training as a separate decision. Our guide to AIO and AEO covers what earns a citation once the crawlers can get in.

What has changed in technical SEO since 2023?

Many audit templates still test things Google has retired. These are the changes since 2023 that alter what an audit should check, each dated from Google's own announcement or documentation.

Since 2023 Google has retired its mobile tools, swapped a Core Web Vitals metric, ended FAQ rich results and given sites an AI Overviews switch.
26 June 2023Sitemap ping endpoint deprecatedGoogle said pings would stop working within six months, and that lastmod is used only when it matches reality.
8 August 2023FAQ rich results limited, HowTo cut backFAQ shown only for authoritative government and health sites; HowTo gone from results by 13 September.
28 September 2023Google-Extended introducedA robots.txt token to opt out of helping improve Google's generative AI models.
31 October 2023Mobile-first indexing declared complete
1 December 2023Mobile-Friendly Test and Mobile Usability report retired
8 January 2024Crawl rate limiter removed from Search ConsoleGooglebot slows itself when a server returns errors or responds slowly.
12 March 2024INP replaces FID in Core Web Vitals
5 July 2024All sites crawled with Googlebot SmartphoneContent not accessible on mobile is no longer indexable.
21 November 2024Sitelinks search box removed
12 June 2025Seven structured data features retiredCourse Info, Claim Review, Estimated Salary, Learning Video, Special Announcement and Vehicle Listing phased out; Book Actions later kept.
3 February 2026Googlebot's 2MB fetch limit documentedGooglebot reads the first 2MB of an HTML file, uncompressed, and the first 64MB of a PDF.
7 May 2026FAQ rich results end for all sites
31 August 2026AI Overviews setting and report reach every siteA Search Console control to leave AI Overviews and AI Mode, and a report of impressions from them.
Sources, all Google: sitemaps ping (26 June 2023); FAQ and HowTo changes (8 August 2023, updated 14 September 2023); Google-Extended (28 September 2023); mobile-first indexing (31 October 2023); Mobile-Friendly Test retirement note (1 December 2023); crawl rate limiter (24 November 2023); INP (updated 12 March 2024); Googlebot Smartphone (3 June 2024); sitelinks search box (21 October 2024); structured data features (12 June 2025); Search Central documentation changelog and Googlebot documentation (February and May 2026); new controls for website owners (3 June 2026, updated 31 August 2026). Checked November 2026.

Which technical SEO audit tools are worth using in 2026?

Free tools cover most of a technical audit for a site of a few thousand pages. Search Console supplies indexing, crawl stats, robots.txt status and URL Inspection; PageSpeed Insights and Lighthouse supply speed data; the Rich Results Test validates markup. What they lack is a full crawl of the site as a visitor would move through it, which is where a crawler earns its place.

ToolWhat it does in an auditCost or free allowance (November 2026)
Google Search ConsoleIndexing, crawl stats, robots.txt, URL InspectionFree
PageSpeed Insights28 days of Chrome UX Report field data plus a Lighthouse lab testFree
Screaming Frog SEO SpiderDesktop crawlerFree up to 500 URLs per crawl; £199 per user per year
Ahrefs Webmaster ToolsCloud site audit checking 170+ issuesFree for verified sites, 5,000 crawl credits per project per month
SitebulbDesktop or cloud crawler14-day free trial; Lite audits up to 10,000 URLs, Pro 500,000; Cloud from £95 a month
Screaming Frog Log File AnalyserServer log analysis with bot verificationFree up to 1,000 lines; £99 per year

Sitebulb's pricing page did not show its desktop prices in its page text in November 2026, so they are left out. Which crawler you pick matters less than how it is set up: crawl with JavaScript rendering on if the site depends on it, connect Search Console so every URL carries its clicks, and save each crawl so the next can be compared with it. For the paid suites side by side, see our comparison of SEO audit tools; for the AI features now built into Screaming Frog, see our guide to Screaming Frog with AI.

What should a technical SEO audit report contain?

A technical SEO audit report is a ranked list of fixes with the evidence for each. A long PDF of every warning a crawler raised is raw material, not a report. Each finding should carry:

  • The issue in one sentence, with the Google documentation that explains why it matters.
  • The affected URLs: a count, a few examples and the full list as an export.
  • The traffic at stake: Search Console clicks and impressions for those URLs over the last three months, or a note that they are not indexed at all.
  • The fix, written for whoever will make it: developer, editor or CMS administrator.
  • How to confirm it worked: which report or test should change.

Order the findings by what each one blocks, following the sequence of this checklist, and then by the traffic at stake. A noindex on a template outranks every speed finding; a slow LCP on a page with 20 visits a month can wait. Leave out measures Google does not document, such as a tool's proprietary health score or word-count thresholds, as reasons to do work.

Here is one finding written that way, for Harbour & Finch LLP, a sample Toronto law firm (sample data, not a real audit). Fourteen practice-area pages declare canonicals pointing at their HTTP versions, which redirect back to HTTPS. Google has chosen a different canonical for nine of them, listed in the Page indexing report as "Duplicate, Google chose different canonical than user". Those nine had 3,100 impressions and 41 clicks in the last three months. The fix is one change to the canonical template so it outputs the HTTPS URL. Confirmation: URL Inspection shows the declared and Google-selected canonicals matching after the pages are recrawled.

A technical SEO audit service should hand over that list plus its raw material (the crawl file, the Search Console exports and the settings used) so the next audit can be compared with this one. Ask how the effect of the fixes will be measured. The honest method compares the fixed pages with similar pages left alone over the same weeks; our guide to proving an SEO change worked explains it, and the free did-it-work check runs it on a Search Console export. The SEO view in AIO Copilot's monthly plan reads a crawl alongside Search Console, ranks fixes by what they would earn and measures each change against a control group of untouched pages.

What can a one-off or single-page technical audit tell you?

A single-page audit examines one URL closely: its status code and redirects, canonical, robots directives, title and headings, structured data, rendered HTML, mobile render and, where Chrome UX Report data exists for it, its field Core Web Vitals. That is the right tool before a launch, after a template change, or when one important page drops. Nearly all of it can be done free with URL Inspection, PageSpeed Insights and the Rich Results Test, or with our free SEO tools that read a page's HTML.

What it cannot see is everything that depends on the rest of the site: which duplicate Google chose, orphan pages, how internal links are distributed, sitemap coverage, redirect chains across a migration, crawl traps and how Googlebot spends its time. Those need a full crawl and Search Console's reports. A one-off audit of the whole site has the opposite limit, because it is a snapshot and faults return with the next deployment. Its most useful output is the list of checks to repeat, ideally on a schedule: crawls compared with the previous one, and a monthly look at the Page indexing and Crawl Stats reports.

With only an hour, open the Page indexing report and the robots.txt report before anything else. Between them they show whether Google can reach the site and what it decided to keep, and every other check on this list pays off only once those two are clean.

Questions

How often should you run a technical SEO audit?

Run a full crawl-based audit after any migration, redesign, CMS change or large template release, because those are when faults arrive. Between audits, Google's own advice for sites that are not very large is to keep the sitemap up to date and check the Page indexing report regularly. Large sites that change daily get more from a scheduled weekly crawl compared with the previous one than from an annual audit.

Is crawl budget a problem for my site?

Probably not. Google's crawl budget guide is written for sites with more than a million unique pages that change about weekly, sites with more than 10,000 pages that change daily, and sites with a large share of URLs stuck in "Discovered - currently not indexed". Google calls those numbers rough estimates. Below that size, wasted crawling usually shows up as duplicate and parameter URLs, which are worth fixing for their own sake.

Do Core Web Vitals affect Google rankings?

Yes, modestly. Google says Core Web Vitals are used by its ranking systems, and in the same document that Search always seeks to show the most relevant content even when page experience is poor. The thresholds for a good score are LCP within 2.5 seconds, INP of 200 milliseconds or less and CLS of 0.1 or less, measured at the 75th percentile of real visits.

Should I block GPTBot in robots.txt?

Only if you object to OpenAI training models on your content. GPTBot is OpenAI's training crawler. OAI-SearchBot is the one that decides whether a site can appear in ChatGPT search answers, and OpenAI says each setting is independent, so a site can allow OAI-SearchBot and disallow GPTBot.

Do I need to remove FAQ and HowTo structured data?

No. When Google limited FAQ rich results in 2023 it said there was no need to proactively remove the markup, and it has said unsupported structured data does not cause problems in Search. FAQ rich results stopped appearing in Google Search on 7 May 2026 and HowTo results on 13 September 2023, so neither is worth adding to win a rich result.

What is the difference between a technical SEO audit and a full SEO audit?

A technical audit asks whether search engines can crawl, render and index the site, and how its pages load and behave. A full SEO audit adds content quality, search intent, keyword targeting, links and competitors. Run the technical part first: content work on pages Google cannot index has no effect.

New studies, by email

Our own studies, and what each new release changes for SEO, AEO and Google Ads.

Or get your price for the app