Making a Site Findable Is a Decision, Not a Setting

For a while this site was easy to visit and impossible to find. Anyone who was given the link could read it. Every crawler that asked was told to leave, because the file that speaks to search engines said Disallow: / and each page carried its own instruction not to be indexed.

That was a choice rather than an oversight, and at the time it was a sensible one. The pages were still being rewritten, the photographs and documents had not been cleared for public distribution, and a page a search engine has never seen cannot be summarised in a result. Nothing was broken. The site had simply not been opened yet.

What opened it was one sentence from the owner, recorded in the project’s register at the time it was said. The sentence is short. What it required is the reason this piece exists, because almost none of the work that followed was about tags or keywords.

A property that has to be agreed on, not installed

Searchability is not a feature you add to a site. It is a small set of declarations that have to agree with each other, and each one is written somewhere different: a flag in the site’s configuration that says indexing is approved, a public file that asks crawlers in or keeps them out, the list of routes the site claims to have, and the manifest that records what will actually be served. The last of those carries the hashes of the files whose exact bytes matter.

Changing one declaration is easy, and changing one on its own is the mistake. The activation therefore went out as a single change covering all four, because a rebase that lands the flag without the public file leaves a site that is crawlable while its own tests still assert that it is not. The tests moved in the same change for a related reason. They existed to encode the previous posture, and a test that is confidently right about a state that no longer exists is worse than no test at all, since a green suite reads as agreement.

Two gates did most of that work. One compares the declared routes with the pages the build actually produced, in both directions, so a page cannot appear without being declared and a declaration cannot outlive its page. The other refuses to build when a file that will be served has not been declared. That sounds like bureaucracy until it refuses something you were sure was fine: a key file for a submission protocol, written during a test into the folder served to the public, stopped before it could ship. The gate was right. Adding a file to a public site is exactly the act the boundary exists to guard.

One host, declared the same way everywhere

The site answers on www.socraticblock.com, and the bare domain redirects there. For a while the pages named the bare domain as their canonical address. In plain terms, the site was telling every search engine that each page’s real address was a different address, the one that answers with a redirect.

The fix was small. The lesson was not. A declaration that points at a redirect is not a declaration; it is another hop for a crawler to follow. It also survived a sweep of the files, because each file looked correct by itself and only the comparison was wrong. What exposed it was asking the live site for each page’s declared address and comparing that, character for character, with the address that actually served the page.

Every gate was green

The build passed throughout. Then an audit fetched what the site actually served, rather than what the build had produced, and read every route’s title, description, canonical link, structured data and response headers.

It found three kinds of defect. Two article titles ran to 76 and 78 characters, in a place where a search result truncates around 60, so the ends of those headlines were being cut off exactly where the words matter most. The description window the project used ran from 75 to 180, which is not so much a window as an absence of one, and it let a 180-character description ship on a note and a 161-character one on two of the marketing pages. And the sitemap carried no modification dates at all, so the file whose purpose is to tell a crawler what changed never told it anything.

The gates had been green because they checked one end of each range rather than both, and because they read the build output rather than the bytes a crawler receives. Those are the same mistake in two coats: the check confirmed that a value existed, not that the value was right.

The same audit reads response headers, which is not fussiness. A single x-robots-tag header overrides every instruction on the page, is invisible to any check that only reads HTML, and is the kind of thing a host adds to a preview deployment without asking. A build cannot see it. Only a request can.

As the site is served now, titles measure 31 to 54 characters, descriptions 132 to 160, the sitemap carries modification dates on the articles, and no response carries an x-robots-tag. Titles and descriptions are now bounded at both ends, article pages trim their summary to fit the description window, and the audit is a script in the project that anyone can run against the live host.

The instrument was wrong and the site was right

One of the first audit runs reported that nine pages had defective heading structure. It counted heading text rather than heading elements, so one correctly written heading containing a line break and a span was counted as four headings. Every page had exactly one heading, as it should. The tool was broken.

That is the more dangerous direction for a check to fail in. A test that misses a defect leaves a defect in place, which can be found later. An instrument that reports a defect that is not there sends someone to edit markup that was already correct, and it does so with the authority of a tool. The habit that catches it costs one command: run the instrument against a case whose answer you already know, before believing what it says about a case you do not.

A key file in the wrong place

Submitting new pages to search engines uses IndexNow, a protocol with a detail that is easy to get wrong and hard to guess from the error. The key file belongs at the root of the host and is named after the key itself. Placed in the conventional .well-known directory, the submission was refused with a 422 and the words URLs are not related to your site verified through the keylocation parameter.

Moving it to the root fixed it. The file now sits in plain sight, by design, because that is what the protocol asks for: it is a public token, not a secret. Someone who has it can submit addresses on the site’s behalf, which is a nuisance, and it gives them no way to read the site or to impersonate it to a reader.

An automation that claimed an action it never took

Part of the work was automated, and the automation carried a defect no test would have caught, because it was a claim rather than a computation. A scheduled job called a search engine’s inspection interface and logged the call as requesting that a page be indexed. That interface is read-only. It reports what the search engine currently thinks about a page, and there is no public way to ask for indexing of an ordinary page; the control exists in a web interface for a person to press.

The call itself was useful and stayed. The wording changed, because a log line describing an action nobody took is how a project acquires a belief about itself that has no basis. What remains is the honest version of the automated path: a submitted sitemap, links from elsewhere, and time.

What is still not known

Nothing here reports an outcome. The site was opened to search engines on 13 October 2026, and this piece describes the work rather than a result, in part because there is no result yet to describe. The instruments that could show one belong to the owner: the crawl statistics inside Search Console, and the equivalent reports in the webmaster tools, both of which need a verified account rather than a crawler. Until someone reads them, the honest statement is that the site is findable in principle and unmeasured in practice.

One widely advised step was not taken, and its absence is a decision worth a sentence. A file called llms.txt has been proposed as a map for language models. An analysis of roughly 300,000 domains found no relationship between having that file and being cited in an AI-generated answer, and the search engine with the largest index has said it does not use the file. So the site does not have one. If it turns out to matter, adding it is a small change; claiming it matters now would be a claim without evidence.

What the work turned out to be about

Making a site findable is a decision with a public record, and the record is the part that has to be right. A file that speaks to crawlers is read, cached and quoted long after it is edited, so it belongs behind the same gates as a deployment: who authorised the change, against which exact bytes, and whether the tests that describe the old state moved with it.

The pages themselves needed less cleverness than the industry around them suggests. One host, named the same way everywhere. Titles that finish before a result cuts them off. Descriptions inside a stated window. Dates that are real. And the habit of checking what the network actually serves, because a green build is a statement about a folder, and being findable is a statement about the world.