Skip to content
Black BoxPersonalization

Understand

Sitemap changes and submission

This page keeps a timeline of the public URL list and tells Google and IndexNow when it changes. Google stopped accepting sitemap pings in June 2023. The sitemap line in robots.txt, Search Console, and IndexNow are the paths that still work.

History unavailable — database offline.

This sitemap

Compared with the snapshot bundled with this deploy. The check uses the sitemap fetched for this request.

  • Pass

    Sitemap HTTP 200

  • Pass

    URL set parsed

  • Pass

    66 URLs

  • Pass

    Absolute https URLs on this host

  • Pass

    lastmod is present and not in the future

  • Pass

    robots.txt names this sitemap

Sitemap: https://www.blackboxpersonalization.com/sitemap.xml

https://www.blackboxpersonalization.com/sitemap.xml

What changed

Compared with the snapshot bundled with this deploy.

Added
0
Removed
0
lastmod
0

No URL was added, removed, or given a new lastmod in this comparison.

Snapshot timeline

History unavailable — database offline.

Google Search Console

Not configured.

No successful or attempted submit is stored.

  1. In Google Cloud, create a service account and enable the Search Console API.
  2. In Search Console, open Settings, then Users and permissions, and add the service account email as an Owner. Restricted permission can read. Submit needs Owner or Full. https://support.google.com/webmasters/answer/7687615
  3. Set SEARCH_CONSOLE_SITE_URL or GSC_SITE_URL. When an account is set and the URL is empty, this lab uses sc-domain:blackboxpersonalization.com.
  4. Set SEARCH_CONSOLE_SERVICE_ACCOUNT_JSON, or SEARCH_CONSOLE_CLIENT_EMAIL with SEARCH_CONSOLE_PRIVATE_KEY. GSC_SERVICE_ACCOUNT_EMAIL and GSC_SERVICE_ACCOUNT_PRIVATE_KEY are accepted as the same pair. The private key stays on the server.

IndexNow

Not configured. Set INDEXNOW_KEY to the key this host serves at /{key}.txt. This lab does not invent a key.

66 URLs would be sent. Automatic IndexNow is off. Set INDEXNOW_AUTO=true to send changed URLs after a new snapshot. Google stays manual.

No IndexNow submit is stored.

Submit

Submit is limited to an admin session. The button runs a dry run: the exact request is shown here, nothing is sent, and the dry run is not stored as a submission.

Calls you can make yourself

IndexNow with curl

The key is public once the host serves it. A private key is not in this snippet.

curl -sS -X POST "https://api.indexnow.org/indexnow" \
  -H "Content-Type: application/json" \
  -d '{"host":"www.blackboxpersonalization.com","key":"YOUR_INDEXNOW_KEY","keyLocation":"https://www.blackboxpersonalization.com/YOUR_INDEXNOW_KEY.txt","urlList":["https://www.blackboxpersonalization.com/lab/sitemap"]}'

Search Console submit in Node

GoogleAuth reads the service account from the environment. The snippet does not contain the key.

import { GoogleAuth } from "google-auth-library"

const siteUrl = "sc-domain:blackboxpersonalization.com"
const feed = "https://www.blackboxpersonalization.com/sitemap.xml"
const auth = new GoogleAuth({
  scopes: ["https://www.googleapis.com/auth/webmasters"],
})
const client = await auth.getClient()
const response = await client.request({
  url: `https://www.googleapis.com/webmasters/v3/sites/${encodeURIComponent(siteUrl)}/sitemaps/${encodeURIComponent(feed)}`,
  method: "PUT",
})
console.log(response.status)

Search Console submit in Python

Point GOOGLE_APPLICATION_CREDENTIALS at the JSON file on the machine that runs this. Do not paste the key here.

import os
from urllib.parse import quote
from google.oauth2 import service_account
from google.auth.transport.requests import AuthorizedSession

site = "sc-domain:blackboxpersonalization.com"
feed = "https://www.blackboxpersonalization.com/sitemap.xml"
creds = service_account.Credentials.from_service_account_file(
    os.environ["GOOGLE_APPLICATION_CREDENTIALS"],
    scopes=["https://www.googleapis.com/auth/webmasters"],
)
session = AuthorizedSession(creds)
url = "https://www.googleapis.com/webmasters/v3/sites/" + quote(site, safe="") + "/sitemaps/" + quote(feed, safe="")
print(session.put(url).status_code)

Where this is documented

  • Google retired sitemap ping

    Since June 2023, google.com/ping?sitemap= does nothing. Google reads the sitemap from robots.txt and from Search Console.

  • Search Console sitemaps.submit

    A PUT to webmasters/v3 sites.sitemaps.submit tells Google the sitemap URL. The call needs the webmasters scope, not the readonly scope.

  • Add an owner in Search Console

    Settings, then Users and permissions. The service account has to be an Owner to submit. Restricted permission is enough to read.

  • How Google reads a sitemap

    lastmod has to be the real change time. Google ignores priority and changefreq.

  • Indexing API is not for this site

    That API accepts JobPosting and BroadcastEvent pages only. This lab does not call it.

  • IndexNow

    One POST to api.indexnow.org/indexnow reaches Bing, Yandex, Seznam, Naver, and Yep. Google does not participate.

  • Bing on IndexNow

    Send the URLs that changed. Repeating the whole site is not the point of the protocol.

  • Vercel cron

    A daily cron can record a snapshot. On Hobby, cron runs once a day and only in production. The page also records a snapshot on the first request after a deploy.

Related labs: Search Console, JSON-LD, and Get found.

Recommended next

Why these picks

Reading path transitions…