Auditing an SEO checker against Google’s docs, and silent scoring bugs

An SEO checker that turns a page into a 0–100 score feels objective, but every number behind that score is a choice: 1% keyword density, 600 words, exactly one H1. This post walks through checking each rule of such a tool against Google's own documentation, what changed when a rule had no source, and the scoring bugs that never threw an error but quietly moved every score.

The tool is seo-analyzer, an open-source library I maintain. It is a fork of Yoast's yoastseo package: Yoast's language processing (stemming, sentence splitting, transition words for 20+ languages) plus an assessment set I wrote for e-commerce pages. Before the first npm release in October 2026, I reviewed every check. The full rule list with sources is in the repo's checklist.

How the score is built

Each check returns good, improve or bad, worth 100%, 50% or 0% of its weight. The weight comes from the check's priority:

export const POINT = 100;
export const priorityRate = { high: 1, medium: 0.75, low: 0.5, info: 0 };
export const statusRate = { good: 1, improve: 0.5, bad: 0 };

// one check's points
POINT * priorityRate[priority] * statusRate[status]

The overall score is the points earned as a percentage of the points available from the checks that apply to the page. Two consequences follow, and the rest of the post keeps coming back to them:

• Priority is a claim about importance. A high check moves the score twice as much as a low one. If the rule behind it is a guess, the score amplifies the guess.
• Every check is part of one sum. One check returning a bad value (a NaN, a missing score) changes the total for every page, not just its own line.

Most popular SEO numbers have no source at Google

The first pass was simple: for each threshold, find the Google page that says it. For most of the content checks there is none. Google's SEO starter guide says outright that "the length of the content alone doesn't matter for ranking purposes", that there is "no magical, ideal amount of headings", that heading order doesn't matter, and that keywords in the domain or URL path "alone have hardly any effect". The helpful content guidance even lists writing to a word count as a sign of search-engine-first content.

When Google gives no number, a checker still has to pick one. The rule I settled on: use the current Yoast default for the threshold, and lower the priority, so an unsourced number can't dominate the score. The changes:

The number-in-title check is the one I'm least proud of in hindsight. It came from listicle-style advice ("10 ways to…"), scored bad on every title without a digit, and had high priority. Reviewing it against a source made it obvious that it measured a style, not a quality.

Score what the page does, not what the user typed

One check gave full points for having a focus keyword set. With high priority, that was 100 points for typing something into a field, on every page, regardless of content. It is useful as a reminder, since most other checks need a keyword, but it says nothing about the page.

The fix was a fourth priority, info, with weight 0: the check still shows in the list, it just isn't scored. The test for any check is: would this result change if the page got better? If not, it's a hint, not a score.

Turn "more is better" into an upper bound

Several checks rewarded the keyword appearing somewhere and never asked how often. That nudges people toward exactly what Google calls spam. Where Google describes the failure mode, the check now looks for it:

• Keyword in the title: present once is good, repeated is improve. Google's title link guidance says there's "no reason to have the same words or phrases appear multiple times".
• Alt text: missing alt is still bad, but alt text that repeats the keyword is now improve. Google warns that filling alt attributes with keywords can be treated as spam.
• Keyword density: over 3% is bad, which is now the meaningful end of the range.

Add checks where Google is concrete

The review also turned up guidance that is specific and testable, so it became new checks:

• Link text (linkText): Google's link guidance asks for anchor text that is "descriptive, reasonably concise, and relevant" and gives "Click here" and "Read more" as bad examples. The check flags those and empty anchors; the phrase list is configurable.
• Old year in the title (titleYear): the title link guidance says Google may detect an outdated year in a <title> and show the date from the visible page instead. A "Best … 2025" page that got updated should update its title too.
• Product structured data (productSchema): merchant listings require name, image and an Offer with price and priceCurrency, and, unlike product snippets, "require a price greater than zero". Missing required properties is bad; missing recommended ones (brand, GTIN, availability, ratings) is improve.
• Generic image file names (imageFileNames, low): IMG_1234.jpg, image1.png, hashes. A small signal, so a small weight.

Scoring bugs don't throw

Re-checking thresholds was planned. The bugs weren't: they showed up when I installed the packed library into a fresh Vite + React app and used it exactly as the README said. None of them raised an error. Each one just produced a slightly wrong number.

A word counter that counted the gaps. The text length check split the text by the word pattern instead of matching it:

'one two three'.split(/[\p{L}\p{N}]+(?:-[\p{L}\p{N}]+)*/gu)
// ['', ' ', ' ', ''] → 2 non-empty pieces

What survives the split are the separators between words, so plain text always came out one word short, and anything with punctuation depended on where the commas fell. The fix was to use the word count research the language layer already had, which also counts Japanese by characters.

A score that was never set. In the URL length check:

assessmentResult.setStatus(calculatedResult.status);
assessmentResult.setStatus(calculatedResult.status); // meant: setScore(...)

The status said good, the line showed green, and the check contributed 0 points while its weight still counted in the maximum. Every page with a URL scored a little lower than it should. Another check had the same shape: it set a status but never a score.

An empty status becomes NaN. The H1 check started with let status = '' and only assigned a value in some branches; calling it without the optional h1Count option on a text with an <h1> hit none of them. statusRate[''] is undefined, 100 * 1 * undefined is NaN, and NaN plus anything is NaN, so one check turned the overall score into NaN. Defaulting status to a real value, and returning 0 when no scored check applies, closed both paths.

Operator precedence in a fallback. The suggested alt text was built like this:

shop?.name || 'Shop' + ' | ' + data?.title || 'Title'

+ binds tighter than ||, so with a shop name set this returns just the shop name. A template literal (`${shop?.name || 'Shop'} | ${data?.title || 'Title'}`) says what was meant.

Checks that disappeared in the browser. The link helpers import Node's url module, and the HTML parser needs events. Vite doesn't polyfill Node built-ins for browser code, so the internal and external link checks failed at runtime and dropped out of the results. Nothing on screen said a check was missing; the list was just shorter. Both polyfills are now real dependencies.

Test invariants, not just examples

The common thread: each bug broke a relationship that should always hold, while every individual result still looked plausible. So the tests assert the relationships:

// every check's score must follow from its status and priority
const mismatched = analyze(text, attributes).assessments.filter(
  (a) => a.score !== POINT * priorityRate[a.priority] * statusRate[a.status],
);
expect(mismatched).toEqual([]);

That single assertion catches the "status set, score forgotten" family on any sample page. Around it:

• The overall score is a finite number for every sample paper (Number.isFinite), which pins the NaN paths.
• Regression snapshots record each check's status and points on a set of sample papers. After an intended change, the diff shows exactly which pages moved and by how much; an unexpected change is a regression.
• Every "Learn more" link resolves: a test reads the checklist document and fails if an assessment's docUrl anchor has no section.
• An import check in lint fails on imports that don't resolve, packages missing from dependencies, and Node built-ins without a browser polyfill. It would have caught the disappearing link checks before any bundler did.

Summary

A score is useful when it points at something a reader would notice.

If you've traced the thresholds of your own checker or linter back to a source, or kept some without one, I'd love to compare notes. My email is on the home page.