Website Pre-Launch Checklist: Domain, robots.txt and Meta Tags

#Four checks that belong before launch

Pre-launch checks come down to four questions: is the domain still valid, can crawlers get in, does the live page render correctly, and do search results look right? Each is independent, and none of them will surface on your own machine.

The first two are the ones people skip, because they do not affect your own browsing — "it loads fine for me" tells you nothing about either.

#Domain expiry is not optional

An expired domain takes down your site, your email and every third-party service that verifies ownership — all at once.

Enter a domain into Whois Lookup and you get four things that matter:

• Registrar — who you renew through
• Creation date — how old the domain is
• Expiry date — the key figure, shown with days remaining
• Domain status — clientTransferProhibited is a normal anti-hijack lock

#robots.txt is the easiest way to block yourself

robots.txt is a set of rules for crawlers, not for visitors — so a mistake is invisible to you and simply shows up as traffic that never arrives.

The syntax is short, but each line can hide the whole site:

# Wrong: nothing is actually allowed after this
User-agent: *
Disallow: /

# Right: block the admin area and search pages, allow the rest
User-agent: *
Disallow: /admin/
Disallow: /search
Allow: /
Sitemap: https://example.com/sitemap.xml

robots.txt Checker does two jobs: it parses the rule groups, and it tests any URL as a specific crawler to see whether it may be fetched.

When something looks wrong, check these three in order: does User-agent: * cover Googlebot; is there a bare Disallow: /; is the Sitemap value an absolute URL. That covers almost every self-inflicted block.

#Verify how the live page actually renders

Local previews and production rendering routinely diverge, because of CDN injection, font loading order, path case sensitivity and extra server-side scripts.

Common sources of difference:

• A CDN injects an analytics script before </body>, which can change page height
• Font font-display settings make first-paint text briefly invisible
• Paths are case-sensitive on Linux — /Img/ and /img/ are different directories
• Server-side compression, cache headers or redirects alter the response

Capture one full-page shot at desktop width and one at mobile width with Web Screenshot, then compare against your local preview. Mobile is worth extra attention: overflow problems usually only appear on narrow screens.

#Meta tags decide what search results look like

The title and description in search results are taken directly from meta tags — search engines only write their own when the tags are missing.

Fill in the fields in Meta Tag Generator and it builds the full tag set live, with a search preview and length guidance:

#Which tool for which job

Run them in this order: domain first, because it is the least reversible; then robots.txt, because it has the widest impact; then screenshots; then meta tags, which you can always fix later.

#Frequently asked questions

#Can an expired domain be recovered?

Yes, within limits and at a cost. After expiry the domain usually enters a grace period where normal renewal still applies, then a redemption period where the fee is much higher, and finally deletion, after which anyone may register it. Exact windows depend on the registry. If the status reads redemptionPeriod, act immediately.

#Does a broken robots.txt affect human visitors?

No. robots.txt only binds crawlers that honour the protocol, so browsing is unaffected. That is exactly what makes it dangerous — the site looks fine, it just stops being indexed. After fixing it, re-check with the crawl testing tool in your search console, or resubmit the sitemap.

#What is the difference between Disallow and noindex?

Disallow lives in robots.txt and means "do not fetch this"; the page can still appear in results if other sites link to it. noindex is a meta tag on the page itself and means "do not index this if you do fetch it", so a crawler has to reach the page first. To remove a page completely, use both.

#Why do local and live screenshots differ?

Usually CDN-injected scripts, font loading order, or path case. One more common cause: opening the file directly over file:// breaks anything that depends on relative paths or network requests. Always preview over a local HTTP server before comparing.

#How long should a description be?

About 160 characters for English, roughly 78 CJK characters for Chinese. Anything longer is truncated, which wastes the effort; too short and the snippet looks thin. Put the key conclusion first, because mobile shows less. The preview in Meta Tag Generator shows exactly where truncation starts.