- robots.txt + meta directives
- XML sitemap integrity
- status codes by template
- canonical consistency
- duplicate clusters
- index coverage causes
Technical SEO agency · US market
If a crawler can't reach it, nothing else you do matters
Crawling, rendering, indexation, architecture, Core Web Vitals and migrations — diagnosed with source, rendered and index evidence, then written as tickets your developers can estimate and ship.
Get a technical SEO auditScoped in 2 days
crawl-trace — user-agent: Googlebot/2.1live trace
Select a URL pattern
01GET /app/pricing HTTP/2
02200 OK · text/html · 4.2 kB
03robots.txt -> allow
04source html: div#root is empty
05source: 0 words · 0 internal links
06render queue -> deferred (bundle 812 kB)
07rendered: 640 words · 24 links
08INDEX: partial — crawled, not indexed
FailContent never reaches the index
The first response contains an empty root div. Rendering is deferred to a client bundle, so first-pass indexing sees no copy, no links and no metadata. Anything discovered later is discovered slowly and inconsistently.
Typical fix1 sprint01GET /shoes?color=black&size=9 HTTP/2
02200 OK · canonical -> self
03variants discovered: 41,812
04log sample: 68% of hits = parameter URLs
05sitemap: 2,140 URLs (products only)
06new product discovery: 6.4 days median
07INDEX: 3,900 indexed / 41,812 crawled
FailCrawl budget burned on 41,800 near-duplicates
Every filter combination is a linked, self-canonical, indexable URL. Crawlers spend their allocation enumerating variants while genuinely new products wait days for discovery.
Typical fix2–3 sprints01GET /old-category-url/ HTTP/2
02301 -> /category/old-category-url
03301 -> /c/old-category-url?ref=legacy
04302 -> /c/new-category
05200 OK · canonical -> /c/new-category/all
06hops: 3 · one temporary · canonical mismatch
07INDEX: legacy URL dropped, equity split
FailRedirect chain drops the signal
The legacy URL resolves through three hops and finishes on a page that canonicalises somewhere else again. Each hop loses clarity, and the destination is not the page that used to rank.
Typical fixDays01GET /blog/ (ua: Googlebot) HTTP/2
02403 Forbidden · mitigated: challenge
03GET /blog/ (ua: Chrome) -> 200 OK
04GET /blog/ (ua: OAI-SearchBot) -> 403
05reverse DNS: crawler verified
06waf rule: rate-limit bot-score < 30
07INDEX: 0 new URLs in 118 days
FailThe crawler is being challenged, not served
Bot management returns a 403 to some verified crawler ranges and a JS challenge to others. Browsers are fine, which is why nobody noticed for four months.
Typical fixHours01GET /products/ridge-jacket HTTP/2
02200 OK · text/html · 38 kB
03source: 1,240 words · 42 internal links
04canonical -> self · 1 declaration
05structured data: Product + Offer valid
06field CWV: LCP 1.9s · INP 140ms · CLS 0.04
07INDEX: indexed · submitted + discovered
PassThis is what a clean fetch looks like
Server-rendered HTML with content, links and metadata in the first response, one canonical, correct status codes and field vitals inside threshold. Nothing here needs a ticket.
Typical fixMonitorDirect answer
Technical SEO improves how search engines and search-connected AI systems crawl, render, interpret, index and retrieve a website. We audit architecture, JavaScript rendering, performance, structured data, migrations, robots controls and crawler access — then turn findings into implementation work that actually ships.
The pipeline
Five stages between your HTML and a ranking result
A page can fail at any one of them and look perfectly fine in a browser. Select a stage to see the failure modes, the evidence we collect, and what we change.
Stage 01 — Discover
A URL has to be found before anything else happens
Discovery comes from internal links, sitemaps and external references. If a page has none of those, it does not exist as far as search is concerned.
Failure modes- Orphan pages with zero internal links
- Sitemaps listing redirects or 404s
- Pagination that only exists in JavaScript
- Infinite calendar or filter spaces
sitemap.xmlxml
1<url>
2 <loc>https://site.com/c/boots</loc>
3 <lastmod>2026-07-31</lastmod>
4</url>
5# 412 of 2,140 entries resolve to 301
6# 88 entries return 404
7# 1,204 indexable URLs missing entirely
What we change
Rebuild sitemap generation from the canonical set, add hub links for orphans, and cap crawlable parameter space.
Stage 02 — Crawl
Crawl allocation is finite, and mostly self-inflicted
Log files show where crawler attention actually goes. On large sites the answer is usually parameters, duplicates and dead ends rather than the templates that earn revenue.
Failure modes- Parameter permutations linked from facets
- Session or tracking IDs in URLs
- Soft 404s returning 200
- Slow responses throttling crawl rate
access.log — aggregatedlog
1# googlebot hits by path group (30d)
2/shoes?* 2,841,502 68.1%
3/products/* 612,004 14.7%
4/c/* 388,110 9.3%
5/blog/* 142,880 3.4%
65xx responses 18,402 0.4%
7avg response time 840ms
What we change
Disallow non-indexable parameter patterns, remove facet links from crawl paths, fix 5xx spikes and cut TTFB on the heaviest templates.
Stage 03 — Render
Rendering decides whether your content is content
We compare the raw response with the rendered DOM, template by template. Everything that only appears after hydration is a risk, and metadata injected client-side is worse.
Failure modes- Copy and links only in the client bundle
- Titles or canonicals set after hydration
- Content behind interaction or scroll
- Blocked JS or CSS resources
render-diff.jsonjson
1{
2}
What we change
Move critical content, metadata and links into the server response — SSR, SSG or ISR depending on the framework — and keep hydration for interaction only.
Stage 04 — Index
Indexation is a decision, made from your signals
Canonicals, status codes, duplicate clusters and quality signals decide which URL represents a group. Conflicting signals get resolved for you, usually not the way you wanted.
Failure modes- Self-canonical duplicates splitting equity
- noindex left in a production release
- Canonical pointing at a redirect
- Thin templates cannibalising each other
head — /shoes?color=blackhtml
1# expected: canonical -> /shoes
2# 41,812 URLs share 96% of their DOM
What we change
Canonicalise filter combinations to the parent category, keep a curated set of indexable facets, and add regression tests on head output.
Stage 05 — Serve
Being indexed is not the same as being retrievable
Search features and search-connected AI systems fetch pages on their own terms. Bot management, paywalls and client-side gating decide whether your content is available to them at all.
Failure modes- WAF challenging verified crawler ranges
- Geo or rate limits blocking fetches
- Content gated behind consent walls
- Field vitals failing on key templates
robots.txt + waf rulesconf
1User-agent: Googlebot
2Allow: /
3User-agent: OAI-SearchBot
4Disallow: / # added by mistake
5# waf: bot_score < 30 -> challenge
6# 403 rate on verified crawlers: 11.4%
What we change
Allow-list verified crawler ranges at the edge, correct robots directives per agent, and monitor crawler status codes as a standing alert.
Audit manifest
218 checks, grouped the way engineers work
Not a plugin scan. Crawl data, Search Console, source HTML, rendered DOM, server logs, CDN behaviour and release history, cross-referenced by template.
- click depth distribution
- orphan page detection
- pagination + faceting
- breadcrumb + hub logic
- anchor text patterns
- authority flow to money pages
- source vs rendered parity
- routing + history API
- metadata injection timing
- hydration + streaming
- blocked resources
- status handling in SPA routes
- field CWV by template
- LCP element per template
- long tasks + INP sources
- layout shift sources
- payload + third-party weight
- cache + CDN behaviour
- schema validity + eligibility
- entity + sameAs consistency
- content-to-markup parity
- hreflang + locale signals
- heading + landmark structure
- image + media semantics
- redirect map coverage
- chain + loop detection
- staging noindex leaks
- template regression diffs
- analytics + GSC continuity
- rollback readiness
JavaScript SEO
Source versus rendered — where SPAs quietly lose pages
The browser shows you the right-hand column. First-pass crawling sees the left. Everything that only exists on the right is a coin flip.
response.html — first fetchwhat crawlers get
<!doctype html>
<html lang=\"en\">
<head>
<title>App</title>
# no canonical declared
</head>
<body>
<div id=\"root\"></div>
<script src=\"/_app/main.812kb.js\">
</body>
# 0 words · 0 internal links
rendered DOM — after hydrationwhat browsers get
<h1>Ridge Jacket — Waterproof Shell</h1>
<p>1,240 words of product copy…</p>
<nav>42 internal links</nav>
<link rel=\"canonical\" href=\"/products/…\">
<script type=\"application/ld+json\">
{ \"@type\": \"Product\", \"offers\": {…} }
</script>
# injected 2.4s after first paint
# render queue: not guaranteed
# parity with source: 0.02
TestEvery template, not one URL
Parity is measured per template group, because one broken layout is thousands of URLs.
MetadataTitles set after hydration
A title injected client-side may never be the one used. It belongs in the first response.
RoutingSoft 404s in SPA routes
Client routers return 200 for missing pages unless status handling is explicit.
FixSSR, SSG or ISR per route
We recommend a rendering strategy route by route against traffic and freshness needs.
Core Web Vitals
Field thresholds, and the code that breaks them
We work against 75th-percentile field data, per template — not a lab score from one run on one URL.
When the main content of the viewport is actually painted. On most templates the culprit is a hero image or a font, not the server.
Goodunder 2.5sNeeds work4.0sPoorover 4.0sUsual culprits- render-blocking CSS + fonts
- unoptimised hero images
- slow TTFB / no edge cache
- client-side fetch before paint
How quickly the page responds to a real interaction. It replaced FID because it measures every interaction, not just the first.
Goodunder 200msNeeds work500msPoorover 500msUsual culprits- long tasks during hydration
- third-party tag managers
- heavy event handlers
- unsplit JS bundles
How much the layout moves while loading. Almost always fixable in a day, and almost always still broken.
Goodunder 0.1Needs work0.25Poorover 0.25Usual culprits- images without dimensions
- injected banners + consent bars
- web fonts swapping metrics
- late-loading ad slots
The deliverable
You get tickets, not a 90-page PDF
Every finding lands as a ticket with evidence, affected templates, acceptance criteria and an estimate your engineers can argue with. This is a real one, anonymised.
SEO-1042 · canonical-consolidationP13 ptsFaceted URLs self-canonicalise, splitting authority across 41,800 variants
Templates hitcategory, facetURLs affected41,812Evidencecrawl + logs + GSC
- <link rel=\"canonical\" href=\"{current_url}\">
+ <link rel=\"canonical\" href=\"{category.base_url}\">
# facet links: add rel=nofollow + noindex for
# combinations outside the curated allow-list
# robots.txt: Disallow: /*?*color=
Acceptance criteria
- Facet URLs outside the allow-list canonicalise to the parent category, verified on 20 sampled URLs.
- Curated facets (colour, size) remain indexable with unique titles and copy.
- Crawl of the category tree returns 3,000 or fewer indexable URLs, down from 41,812.
- Head-output regression test added to CI so the canonical rule cannot silently revert.
PrioritisationImpact × reach × confidence ÷ effort
Every ticket is scored, so your team can argue with the order instead of guessing at it.
EvidenceNothing without a receipt
Each finding links to the crawl row, log sample or GSC export that proves it.
ValidationWe retest after release
Affected templates are re-crawled post-deploy and the ticket only closes on evidence.
Migrations
The gates a migration has to pass
Most post-migration traffic losses are decided weeks before launch. These are the gates, the owner and the failure we are protecting against.
| Phase | Gate | If it is skipped | Owner |
|---|---|---|---|
| Pre | URL map signed off | Legacy URLs land on the homepage and years of equity is flattened. | SEO + dev |
| Pre | Template parity review | New templates quietly drop copy, links or metadata the old ones had. | SEO + design |
| Pre | Staging crawl + noindex check | Staging gets indexed, or production ships with noindex still on. | Dev |
| Launch | Redirect chain validation | Hops and loops dilute signals and slow reprocessing to weeks. | Dev |
| Launch | Analytics + GSC continuity | You lose the ability to prove what the migration did, in either direction. | Analytics |
| Post | Indexation reconciliation | Missing pages go unnoticed until a quarterly report surfaces them. | SEO |
| Post | Rollback readiness | A bad release has to be fixed forward under pressure instead of reverted. | Dev |
Measurement
Reported by template, because that is where fixes land
Four leading indicators, each tied to a template group so a regression is traceable to the release that caused it.
Index coverage94%of strategic URLs indexed
Tracked against a defined list of commercial URLs, with a named cause for every exclusion.
Crawl efficiency71%of hits on valuable templates
Log-derived. The inverse is crawl waste on parameters, duplicates and dead ends.
Template health0 P1open regressions
Canonical, status, depth and rendered-parity checks run per template on every release.
Field vitals82%of templates passing
75th-percentile field data by template group, not a single lab run on the homepage.
Decision guide
Technical SEO or on-page SEO?
The shortest diagnostic: if your pages are indexed but not ranking, it is on-page. If they are not indexed at all, or the site is fast in the lab and slow in the field, it is here.
- Large, JavaScript-heavy, ecommerce, marketplace or multi-location sites
- A redesign, replatform, domain move or international rollout is planned
- Search Console is showing crawl or indexation anomalies you cannot explain
- Your developers need precise, testable tickets rather than a PDF of warnings
| Factor | Technical | On-page |
|---|---|---|
| Focus | Crawling, rendering, indexing, speed | Content, intent match, internal links |
| Owner | Usually engineering | Usually marketing |
| Typical blocker | Pages not indexed at all | Pages indexed but not ranking |
| Unit of work | Template and release | Page and cluster |
| Cadence | Sprint-based, front-loaded | Continuous |
| Failure mode | Silent — looks fine in a browser | Visible — rankings plateau |
Commercial scope
Scoped by URL count, template variety and who ships the fix
Log-file analysis, international signals and implementation support are the three variables that move price most.
Diagnostic
Technical SEO audit
from$6,500Full diagnosis across crawl, render, index, architecture, performance and semantics.
- Crawl + GSC + rendered-DOM evidence
- Findings segmented by template
- Prioritised ticket backlog
- Two developer walkthrough calls
Specialist
JavaScript SEO review
from$4,200Source-versus-rendered testing across every template in a framework app.
- Parity testing per template
- Routing, metadata + status handling
- Rendering strategy recommendation
- PR-level implementation notes
High risk
Migration support
from$8,000Pre-launch requirements through post-launch reconciliation and recovery.
- URL map + redirect validation
- Template parity + staging QA
- Launch-day monitoring window
- Post-launch reconciliation report
Ongoing
Implementation retainer
from$4,500/moRelease reviews, ticket writing, staging QA and regression monitoring.
- Pre-release template review
- Standing crawler + index alerts
- Backlog grooming with your team
- Monthly technical health report
Free tools
Run the first checks yourself
Four of our free tools cover the opening moves of a technical audit. Real crawls and real output — no sample reports.
free + 3crFree SEO AuditA real crawl of your page — technical, on-page and content checks with evidence, plus Core Web Vitals.opening moverun →freeSchema Markup GeneratorValid JSON-LD for Organization, LocalBusiness, Service, Product, Article and FAQ — no code required.semanticsrun →freeMeta Title & Description CheckerFetch a page's real metadata, preview the SERP snippet and catch truncation before searchers do.head outputrun →freeLocal SEO AuditNAP consistency on the page, LocalBusiness schema validity and Google Business Profile health.multi-locationrun →
FAQs
What engineers and heads of growth ask us
Send the URL, the platform and what changed. You get a written technical read from the person who would run the audit.
Ask a questionWhat is the difference between an audit and ongoing technical SEO?
An audit is a point-in-time diagnosis with a prioritised backlog. Ongoing work supports implementation, reviews new templates before release, QAs staging, monitors regressions and keeps the backlog current as the site changes. Most teams need the audit once and the ongoing support during a period of heavy development.
Can you work with Next.js, Nuxt, Remix or a custom SPA?
Yes. We review source HTML, rendered output, routing, metadata timing, canonical behaviour, streaming, caching and deployment patterns. Recommendations differ depending on whether rendering is static, server-side, streamed or client-side — we scope after seeing how your app actually renders, not from the framework name.
Do you fix the issues or only report them?
Either. For client-controlled codebases we write implementation tickets with acceptance criteria and work alongside your developers, including PR review. Where the CMS and scope allow, we implement directly and validate the release ourselves.
How long does a technical SEO audit take?
A small marketing site is one to two weeks. Large ecommerce, marketplace or international sites take three to six, because findings have to be segmented by template, market and business impact — and because log-file analysis needs a full 30-day window to be meaningful.
Can technical SEO recover traffic lost after a migration?
It can resolve migration-caused problems: missing or chained redirects, changed canonicals, blocked resources, altered architecture, indexation errors. Recovery also depends on how long the problems persisted and whether content, links or demand changed at the same time. We tell you which of those we can see in the data before quoting.
Is Core Web Vitals a ranking factor?
Page experience is one input among many, and a weak one relative to relevance. We improve field vitals because speed and stability affect conversion and crawl efficiency as well as search — not because one score is a strategy. Anyone selling a 100 Lighthouse score as an SEO plan is selling the wrong thing.
How much does a technical SEO audit cost?
It moves with URL count, template variety, JavaScript complexity, international or ecommerce features, whether log-file analysis is in scope, and how much implementation support you need. Starting points are on this page; we review the site before proposing a scope.
Does technical SEO affect ChatGPT, Copilot and AI search?
At the eligibility level, yes. Search-connected AI systems depend on crawlable, indexable, well-structured content. Robots rules, CDN and WAF access, canonicalisation, internal links and clear textual HTML all affect whether your content can be retrieved at all. No agency can guarantee citation — but being unfetchable guarantees the opposite.
Send the URL and what changed. We will tell you what is broken.
Platform, recent releases, migration history and the Search Console or performance symptom you are seeing. You get a written technical read — the likely cause, the evidence we would pull to confirm it, and the right scope. No obligation.
Digital OttersWhat you get back
Likely cause, named — not a list of every warningThe evidence we would pull to confirm itWhether it is an audit, a review or a migration jobWritten scope and cost range, no obligation