How to get the web to see your site

Indexing means a search engine has fetched the URL and may show it. The sitemap and robots.txt are already public. The remaining work is telling Google and Bing where those files are.

Google Search Console

1. Open Search Console and add a property. Choose the domain property, not a single URL prefix.
2. Google gives you a DNS TXT record. Add it at the DNS host for blackboxpersonalization.com. A domain property covers the apex host and www, and both http and https.
3. After the property verifies, open Sitemaps and submit https://www.blackboxpersonalization.com/sitemap.xml. The sitemap already uses the www origin.
4. Open URL Inspection, paste one www URL, and request indexing. Use it for a new or updated URL. Google limits how many requests a property can make.

This repository does not contain a verified property. The verification meta tag is omitted until GOOGLE_SITE_VERIFICATION is set, and that tag is a URL-prefix check, not the DNS domain check.

Bing Webmaster Tools

After Search Console verifies, Bing Webmaster Tools can import that property. Bing’s index feeds Microsoft Copilot. DuckDuckGo has long drawn on Bing. OpenAI has described ChatGPT search as using Bing among other sources. An import shares the sitemap with Bing. It does not place a sentence in an assistant’s answer.

IndexNow

IndexNow is a ping. Participating indexes, including Bing, can read it. Google does not use IndexNow. This site hosts the key at /b3c878e7001003a5d0d24925e7e49499.txt. After a deploy, node scripts/indexnow-ping.mjs posts the sitemap URLs to api.indexnow.org. A 200 or 202 means the ping was accepted. It does not mean the URL is indexed.

A crawler that fetches a URL still has to read a title, a description, headings, and links. This site sets those in the page, and the SEO lab shows the live tags.

• The title template is the page name, then the site name. This page’s title is “How to get the web to see your site”.
• The description is the sentence in the head of the page. Keep it specific to that page.
• One h1 states the page. The numbered sections are h2s.
• Person and WebSite structured data describe Neil Black and the site. Notes add Article.
• Internal links are the header, the footer, the lab trail, and the links in a guide like this one. They tell a crawler which URLs belong together.
• Speed is a ranking input beside the words and the links. The page speed test runs Lighthouse on a sitemap URL when Google’s API accepts the call.

The SEO lab reads the title, canonical, structured data, and Open Graph tags from a URL on this host.

Search engines discover URLs from links. A person has to publish the link. One useful post in a place you already take part in is the practical start. A blast of identical posts is easy to ignore.

Use a different UTM source for each place so GA4 can separate them. Campaign can stay get-found while you are pointing at this guide.

A LinkedIn post for this guide can use https://www.blackboxpersonalization.com/lab/get-found?utm_source=linkedin&utm_medium=social&utm_campaign=get-found. Build the next one in the UTM builder. Those parameters replace GA4’s organic medium, so use them on the posts you publish, and leave them off when you only want unpaid search counted as organic.

A backlink is a URL on someone else’s site. Search Console’s links report lists them after the property is verified. This page cannot see those links.

A Google Business Profile fits a business people visit at a public place. This site is a publication by Neil Black. It does not claim a Business Profile.

Assistants answer from indexes and from pages they fetch when someone asks. The practical steps are the same public files, plus a robots.txt that lets those crawlers fetch.

• Bing indexing is the path into Copilot, and it is one source ChatGPT search and DuckDuckGo have used. That starts with the Bing import in section 1.
• robots.txt allows / for every user agent. That includes search and user-fetch tokens such as Googlebot, bingbot, OAI-SearchBot, ChatGPT-User, PerplexityBot, Perplexity-User, and Claude-User. Training crawlers are allowed by the same rule. There is no separate block for them.
• Pages that state a fact plainly are easier to quote. The about page names Neil Black. Lab pages say what they measure.
• /llms.txt is a courtesy map from the 2024 proposal. Publishing it is speculative. It is not a confirmed way to be cited. What is llms.txt? walks the file.
• A citation on another site is how an assistant often meets a URL. The AI referrals page classifies assistant referrers and keeps a citation log you fill in.

Search Console’s performance report is impressions, clicks, and the queries that showed the site. Those rows appear after the property is verified and after Google has data. This lab does not invent them.

In GA4, unpaid search usually lands in Organic Search. Visits from assistant hosts are counted on the AI referrals page, which also shows the source pattern for a custom channel. Analytics storage stays denied until a visitor accepts, so GA4 sees the accepted visits.

A URL Inspection request can be the same day. A new URL often takes days, and sometimes a few weeks, before it appears. Some URLs are fetched and still not shown. Nothing on this page guarantees a ranking, a click, or a citation.

SEO · AI referrals · llms.txt · Page speed · UTM builder