Bulk URL Submission with the Google Indexing API: Safe Limits and Scripts

This guide is for developers and site owners who need to submit many URLs through the Google Indexing API without triggering quota errors or harming crawl trust. You will learn safe batch sizes, how to build a clean canonical list, how to pace Python, Node, PHP, and cURL senders, and how to handle 429 with exponential backoff. The focus keyword is indexing api bulk submission. Every pattern here favors queues, throttling, and logging over speed, so large imports and job board feeds stay inside daily limits.

Key takeaways

• Bulk means paced queues with caps and retries, not parallel blasts against the publish endpoint.
• Google officially supports JobPosting and BroadcastEvent pages only, so bulk pings for other types are hints with no promised crawl.
• Normalize to canonical URLs, dedupe, and prioritize new and high value pages when backlog exceeds quota.
• Send 10 to 20 URLs per batch with pauses, back off on 429, and track daily usage to 80 percent of known quota.
• Log every attempt with code and notifyTime, keep sitemaps and internal links current, and spread imports over days.

• indexing api bulk submission: what it means in practice
• What Google supports and why it shapes bulk policy
• Quotas rate limits and 429 behavior
• Build a clean canonical URL list
• Prioritize when backlog exceeds quota
• Python bulk sender with throttle
• Node bulk sender with queue
• PHP and cURL bulk patterns
• Backoff retry and dead letter handling
• Logging monitoring and request IDs
• Recovery after quota exceeded
• FAQ
• Sources
• Further reading

<!-- IMAGE-PROMPT cover: 1200x630, DependsIt brand, deep charcoal #121212 or clean white background, vibrant mint #22E3B0 accent glow, thin node-network line art, Clash Display style bold heading space on left, General Sans clean labels, subject: bulk URL queue flowing through throttle into API, flat vector, high contrast, accessible, no photorealistic faces, no text smaller than 24px, no em dash in rendered text, export PNG then cwebp -q 82 to WEBP -->

indexing api bulk submission: what it means in practice

Bulk submission means sending tens, hundreds, or thousands of urlNotifications publish calls for changed URLs over a controlled period. In practice, bulk is a queue problem, not a speed problem. Each URL needs its own POST with url and type, each response needs logging, and each 429 needs a pause. Teams that treat bulk as a for loop without delays hit per minute limits within seconds, then spend hours in extended throttles. Teams that treat bulk as a paced worker with caps finish slower on the clock but with higher 200 rates and clearer logs.

A safe bulk system has five parts: a source list of canonical URLs with types, a queue table with status and retry counters, a worker that pops small batches with sleeps, a quota tracker that pauses at 80 percent of known daily budget, and a log that records code, notifyTime, and error excerpts. The worker runs on cron or a scheduler, claims rows before sending to avoid duplicates, and moves failures to retry or dead letter after three attempts. This design works the same whether the sender is Python on a server, Node in a container, PHP in WordPress, or cURL from a shell script. It also keeps operators calm during large imports.

Volume determines design. A blog that publishes 20 posts per month can bulk with a CSV and a 10 line script run manually. A job board that adds 300 jobs per day needs a persistent queue, priority for new jobs, daily caps, and dead letter review. A marketplace migration that changes 50,000 URLs should not attempt API submission for all 50,000 at once. Instead, select fresh and high value URLs for API hints, rely on sitemaps and internal links for the rest, and spread API sends over weeks. Careful scoping at this stage saves days of quota troubleshooting later. For context on manual limits that bulk replaces, see the comparison of Indexing API versus Request Indexing.

Bulk also needs ownership discipline. Submit only URLs on properties where your service account holds Owner in Search Console. Normalize hosts, protocols, and trailing slashes to canonical form before queueing. Exclude staging, preview, and parameter variants that canonicalize elsewhere. These filters prevent 400 errors and keep quota focused on URLs that can actually benefit from a crawl hint.

When you bulk submit urls google for a large property, start with a reviewed allowlist so every call targets an owned canonical. Handling indexing api multiple urls in one controlled run works best when the list is deduped, split by priority, and logged with batch IDs.

Do not attempt to submit 1000 urls google in one sitting, since daily caps and per minute throttles will stop the run early. A safer mass url submission plan splits the list into priority slices, sends each bulk urlnotification with a sleep between calls, and spreads the backlog over days with sitemaps covering the rest.

Keep an audit trail from the first bulk run. Record who approved the list, which property was targeted, which service account was used, and which batch IDs were sent on which dates. Store the source CSV, the cleaned queue export, and the results log together for 90 days. When a stakeholder asks why a URL was not sent, the trail shows whether it was filtered as duplicate, deferred as low priority, or failed with a specific code. This record also helps new team members learn the pacing rules without repeating early mistakes that burned quota.

What Google supports and why it shapes bulk policy

Google documents the Indexing API for JobPosting and BroadcastEvent pages only. JobPosting covers job listings with valid structured data. BroadcastEvent covers livestream pages inside VideoObject markup. For those types, URL_UPDATED may prompt a fresh crawl and URL_DELETED helps with removals. Normal posts, product pages, category pages, and archives sit outside documented support. Bulk submitters often see 200 responses for those types, but 200 means receipt, not a promise to crawl or index. Shape bulk policy around that fact to avoid overpromising results to stakeholders.

Google does not support IndexNow. IndexNow serves Bing, Yandex, Naver, Seznam, and other participants through a key file at the root. It never reaches Google. If you need bulk coverage for Bing, run a separate IndexNow batch with its own 10,000 URL per request format and logs. For Google bulk, the inputs are the Indexing API publish endpoint, accurate sitemaps, and strong internal links. The official scope is defined in the Indexing API prerequisites, which should settle scope debates when tool docs claim broader support.

Bulk policy should encode content fit. For job boards with valid markup, bulk on publish and on expiry is aligned with docs. For livestream schedules, bulk on schedule change is aligned. For standard blogs and stores, bulk is off label, so limit to new and materially updated canonicals, monitor coverage for effect, and invest remaining effort in sitemaps and hubs. A short policy doc that lists allowed post types, allowed triggers, daily caps, and review cadence prevents ad hoc bulk runs after every template change. For risk background on normal pages, share the honest answer on normal pages with content teams.

Quotas rate limits and 429 behavior

Quota has two layers: daily publish budget and per minute rate. New projects often report around 200 publish calls per day, with per minute limits that return 429 on bursts. Quota can differ by project age, property trust, and API usage history, so treat published numbers as starting points and confirm your own quota in Cloud Console under IAM and Quotas. Track usage per day in your queue table or a counter file, and pause routine sends at 80 percent to reserve room for urgent fixes. Per minute behavior matters more for bulk design, since a tight loop of 50 sends in 10 seconds triggers 429 even when daily budget remains.

HTTP 429 means slow down. The response may include a Retry After hint in seconds. Respect it, then add exponential backoff such as 60 seconds after first throttle, 300 seconds after second, 900 seconds after third, plus small jitter so parallel workers do not retry together. Do not retry immediately in a loop, since that extends the throttle window and burns quota accounting. Log 429 with timestamp, batch size, and worker ID to tune future pacing. Separate 429 from 403 in code, since 403 permission errors need config fixes, not pauses.

Design pacing from the start. A reliable default is 1 request per 2 to 5 seconds, batches of 10 to 20, then a 60 to 120 second pause between batches. That yields roughly 300 to 600 sends per hour without bursts, which fits many daily quotas when spread across the day. For larger daily budgets, increase batch count, not concurrency. Avoid parallel threads hammering the endpoint from several workers at once unless you coordinate through a shared claim mechanism. Respecting bulk indexing limits means coding daily caps and per minute sleeps rather than trusting manual discipline. A simple batch indexing api worker that sends 10 to 20 URLs then pauses keeps throughput steady and avoids extended throttles.

For quota fundamentals that inform these numbers, review quota limits explained.

Open Cloud Console quotas for your project before each large bulk and note the current daily limit, the usage reset time, and any per minute ceiling shown for urlNotifications publish. Screenshot or copy those numbers into your run notes, since limits can change with project history and you want the worker cap to reflect reality. If Console shows a lower limit than your plan assumes, split the backlog across more days rather than hoping the API will allow extra sends. This two minute check prevents the common failure where a plan built on old quota numbers stalls halfway with 429.

Build a clean canonical URL list

Bulk quality starts with list hygiene. Build source lists with columns for url, type, lastmod, priority, and source. Pull from the CMS database or sitemap rather than from crawl exports that include parameter variants. Normalize each URL: force https, lowercase the host, remove default ports, collapse duplicate slashes, and match trailing slash policy to canonical tags. Resolve redirects before queueing, so the list holds final destination URLs, not 301 hops. Dedupe case insensitively on the normalized form and drop rows that canonicalize to another URL in the set.

Filter aggressively. Keep only public indexable URLs with 200 status, self referencing canonicals, and no noindex. Drop staging hosts, preview query strings, paginated duplicates that canonicalize to page one, and faceted URLs blocked by robots. For deletions, keep only URLs that now return 404 or 410, since URL_DELETED for live URLs creates confusion. Validate a 5 percent sample with header checks before queueing the full file. A 30 minute cleaning pass often prevents hundreds of 400 errors and saves quota for URLs that matter. Repeat this filter after every import, since source feeds often reintroduce parameter variants and expired rows that were removed in prior cycles.

Split lists by type and priority. Keep URL_UPDATED and URL_DELETED in separate files or queue partitions so review and retry stay clean. Tag priority such as P1 new jobs, P2 major updates, P3 minor refreshes. When quota is tight, send P1 first and defer P3 to sitemap coverage. Record source such as publish hook, import job, or manual audit, so later analysis can compare 200 rates and crawl outcomes by source. Store raw source files for 90 days to support audits and quarterly reviews.

# Normalize and dedupe canonical URLs before queueing. Keeps quota focused.
from urllib.parse import urlparse, urlunparse
def canonicalize(u):
    p = urlparse(u.strip())
    scheme = 'https'
    host = (p.hostname or '').lower()
    path = p.path or '/'
    if path != '/' and path.endswith('/'):
        pass
    q = ''  # drop query for canonical list unless params are canonical
    return urlunparse((scheme, host, path, '', q, ''))
urls = ['https://Example.com/Jobs/123/?utm=x', 'https://example.com/jobs/123/']
seen = set()
clean = []
for u in urls:
    c = canonicalize(u)
    if c not in seen:
        seen.add(c)
        clean.append(c)
print(clean)

Verify list size against quota before sending. If clean P1 holds 400 URLs and daily budget is 200, plan two days with P1 split by value, not one burst that hits 429 halfway. Communicate the plan to stakeholders with dates, so expectations match the queue rather than the raw file size.

Add a validation pass with header checks for the full P1 slice when tools allow it. Request headers for each URL and confirm 200 for updates and 404 or 410 for deletions, then confirm content type is HTML for pages you expect Google to crawl. Flag soft 404s where the server returns 200 with thin no results content, since those waste quota and confuse reporting. Keep the validation output with the batch record so reviewers see that list quality was checked before any send.

Prioritize when backlog exceeds quota

Backlog exceeds quota after imports, migrations, template changes, and feed resyncs. The fix is triage, not speed. Rank URLs by business value and freshness, send the top slice through the API, and let sitemaps plus internal links carry the rest. For job boards, rank by publish date descending, then by application conversion, then by location coverage. For news with livestreams, rank by stream start time ascending. For blogs, rank by publish date and hub linkage. Document the ranking so future runs reuse it.

Set daily caps per priority. For example, with a 200 daily budget, allocate 120 to P1 new pages, 50 to P2 major updates, 30 reserve for urgent fixes, and zero to P3 minor tweaks until backlog clears. Process in that order each day. When P1 clears, promote P2. This simple allocation prevents low value refreshes from crowding out new pages that need discovery. Review allocation weekly during backlog periods and adjust based on 200 rates and Googlebot log evidence. Keep the allocation table visible to content and engineering so both teams plan submissions against the same budget.

Use sitemaps and links as the second lane. Update sitemap lastmod to true modification times, split sitemaps by type and priority, and reference them in robots.txt and Search Console. Link new and updated URLs from homepage modules, category hubs, and related lists that Google crawls often. Fix orphan pages with no inlinks before spending API quota on them, since isolated pages crawl slower even after a hint. These steps cost no quota and often move more URLs than extra pings. A persistent indexing queue with status, tries, and next retry fields makes triage visible to the whole team. The queue holds P1, P2, and reserve slices separately so operators always know which URLs will send next.

Coordinate bulk sends with deploy and content calendars so API quota is available when it matters. Pause routine bulk during site migrations, theme launches, or feed reimports that generate thousands of low value changes, then resume with a reviewed P1 list once the site is stable. Inform editors of the bulk window and the daily cap so manual dashboard sends do not collide with the worker and trigger avoidable 429. This coordination keeps bulk predictable even during busy release weeks and holiday content freezes.

<!-- IMAGE-PROMPT diagram-01: 1600px max, DependsIt brand mint #22E3B0 on charcoal #121212 or white, node-network line art, subject: bulk URL triage to priority queue diagram, flat vector, accessible, no em dash, Clash Display style headings, General Sans clean labels -->

Python bulk sender with throttle

Python suits bulk because requests, time, and sqlite or CSV handling are straightforward. The sender below reads a CSV with url and type columns, sends one POST at a time with Bearer auth, sleeps between sends, and writes results to a log CSV. It tracks daily count in a small file and stops at a configured cap. It treats 429 with backoff and jitter, treats 403 as fatal for the run, and retries 5xx once after a pause. Replace token handling with your existing OAuth flow that caches for 55 minutes. Keep the script single threaded to avoid bursts.

Structure the run for auditability. Use batch IDs such as 2026-10-14-P1-01, log batch ID per row, and store raw responses for non 200 codes. Validate URLs before sending with a canonical check and a live header check for a sample. Add command line flags for input file, batch size, sleep seconds, and dry run. Dry run prints what would send without calling the API, which catches list errors before quota spend. Run first with 5 URLs, then 20, then full P1 slice.

Handle auth outside the send loop. Fetch the access token once, reuse it for the batch, and refresh on invalid token errors. Do not fetch a new token per URL, since that adds latency and auth load. If token fetch fails, exit without sending and log the auth error separately. This separation keeps auth issues from looking like URL issues in logs.

# Bulk sender sketch with pacing and backoff. Single threaded by design.
import csv, time, random
def send_bulk(rows, token, sleep_s=3, cap=180):
    sent = 0
    for url, typ in rows:
        if sent >= cap:
            break
        code = post_one(url, typ, token)  # wrapper around publish endpoint
        log_row(url, typ, code)
        sent += 1
        if code == 429:
            time.sleep(300 + random.randint(0, 60))
        else:
            time.sleep(sleep_s)

Tune sleeps from evidence. Start with 3 seconds between sends and 90 seconds between batches of 15. If 429 appears, double sleeps and halve batch size for the next run. If 200 rate stays above 95 percent for a week, hold settings rather than pushing faster. Bulk success is stable throughput over days, not peak speed for an hour. A reusable bulk indexing script saves time because the same file handles CSV input, token reuse, and logging across properties. Keep the indexing api loop single threaded with explicit sleep calls so bursts never slip through during late night runs.

Keep run flags consistent across environments so staging tests predict production behavior. Use the same input columns, the same sleep defaults, and the same cap logic in every run, with only the property and token differing. Document the exact command used per batch ID in your log header, including input filename, batch size, sleep seconds, and dry run setting. This consistency makes it simple to compare 200 rates across days and to spot when a list change rather than a pacing change caused a new error pattern. Archive flag sets with each batch record for later review.

Node bulk sender with queue

Node fits teams that already run JavaScript workers or deploy pipelines. Use a simple file or table queue, a worker that shifts one item, sends with fetch, awaits a sleep, and repeats. Keep concurrency at one to avoid bursts. Store attempts and next retry timestamp per row, and update status to sent, retry, or dead after three tries. Use environment variables for key path and property, never hardcode secrets in the repo. Log JSON lines with timestamp, url, type, code, and batch ID for easy filtering.

Add scheduler control. Run the worker on cron every 10 minutes with a max per run such as 15 sends. Check daily cap at start and exit early if reached. Claim rows with a worker timestamp before sending so overlapping runs do not duplicate. On 429, set a global pause until timestamp and exit, so the next cron run resumes after the window. On 403, exit and alert, since config needs fixing.

// Node worker sketch. One at a time with sleep. No parallel blasts.
async function runQueue(items, token) {
  for (const it of items) {
    const code = await postOne(it.url, it.type, token);
    logResult(it, code);
    if (code === 429) {
      await sleep(300000);
    } else {
      await sleep(3000);
    }
  }
}
function sleep(ms) { return new Promise(r => setTimeout(r, ms)); }

Test with a staging property first. Send 5 test URLs, confirm 200 and notifyTime, then trash one test URL and confirm URL_DELETED handling. Only then point the worker at production lists. Keep staging and production queues separate to avoid cross posting. Version the worker and list schema together so rollbacks stay consistent.

PHP and cURL bulk patterns

PHP fits WordPress and custom CMS hosts where Python or Node is unavailable. The pattern mirrors the others: read a CSV from a private directory, loop one at a time with wp_remote_post or curl, sleep between sends, and append results to a log file. Use transients or options for daily counters and token caching. Keep batch size small per web request to avoid timeouts, and prefer WP CLI or cron for larger lists. Never run bulk from an admin page load without caps, since browser timeouts create partial sends that are hard to resume.

cURL fits one off fixes and audits. Store URLs in a text file, one per line, then loop in bash with sleep and per URL logging. Read the token from an environment variable, not from command history. Send URL_UPDATED for live pages and URL_DELETED for 404 pages in separate passes. Keep passes small and labeled, and append codes to a results file for sheet import. This shell approach is transparent and easy to review, but it lacks automatic retries, so keep batches under 30 and handle 429 manually with a pause.

# cURL bulk loop with pacing. Token from env, one URL per line in urls.txt.
# while read -r U; do
#   curl -s -o /tmp/resp.json -w "%{http_code} $U\n" -X POST "https://indexing.googleapis.com/v3/urlNotifications:publish" \
#     -H "Content-Type: application/json" -H "Authorization: Bearer $TOKEN" \
#     -d "{\"url\":\"$U\",\"type\":\"URL_UPDATED\"}";
#   sleep 3;
# done < urls.txt

// PHP sketch for WP CLI or cron. Reads private CSV, sends with pauses.
function bulk_send_from_csv($path, $token) {
    $rows = array_map('str_getcsv', file($path));
    $sent = 0;
    foreach ($rows as $r) {
        if ($sent >= 15) { break; }
        list($url, $type) = $r;
        $code = post_single_url($url, $type, $token);
        error_log('[bulk] ' . $type . ' ' . $url . ' code=' . $code);
        $sent++;
        sleep(3);
    }
}

Choose the tool that matches hosting and skill. Python for data heavy cleaning, Node for JS teams, PHP for CMS hosts, cURL for quick audits. All four must share the same list format, pacing defaults, and log fields so results stay comparable across runs.

<!-- IMAGE-PROMPT workflow-02: 1600px max, DependsIt brand, subject: paced bulk sender workflow with backoff and dead letter, flat vector, accessible, no em dash, mint #22E3B0 on charcoal #121212, node-network line art, Clash Display style headings, General Sans clean labels -->

Backoff retry and dead letter handling

Retries need rules, not loops. Retry 429 with exponential backoff and jitter. Retry 5xx once after a few minutes, then queue for next window. Do not retry 400, since malformed URLs fail the same way until fixed. Do not retry 403 without a config change, since permission errors persist. Do not retry 404 on metadata as a failure, since no history is normal for new URLs. Encode these rules in the worker so operators do not decide per row under pressure.

Use next retry timestamps rather than immediate loops. On 429, set next retry to now plus 600 seconds with jitter. On 5xx, set now plus 300 seconds. On success, mark sent with notifyTime. After three attempts, move the row to dead letter with last code and message for human review. Review dead letter weekly: fix canonical issues, remove stale URLs, confirm property access, then requeue valid rows once. This cycle prevents poison rows from blocking fresh URLs.

Coordinate multiple workers through claims. Before sending, update the row to claimed with worker ID and timestamp. Only the claiming worker sends that row. If a worker crashes, a sweeper releases claims older than 30 minutes back to pending. This simple lock avoids duplicates without complex infrastructure. Keep worker count at one for most sites. Add a second worker only with shared caps and shared pause flags, so throttles propagate instantly.

Logging monitoring and request IDs

Logs make bulk safe over months. Log every attempt as one JSON line with timestamp, batch ID, url, type, code, tries, notifyTime when present, and error excerpt for non 200. Keep success excerpts short and error excerpts up to 2000 chars. Store logs outside the web root, rotate weekly, and keep 90 days for trend review. Add a daily summary with sends, 200 rate, 429 count, 403 count, and oldest pending age. Alert when 429 spikes, when dead letter grows, or when oldest pending exceeds 24 hours.

Correlate with Cloud Console metrics and server logs. Note request IDs when the API returns them, since they help trace quota accounting and error bursts. Compare submitted URLs against Googlebot hits in access logs to estimate time to first crawl for samples. Compare submitted versus non submitted cohorts in Search Console coverage to judge effect without overclaiming. If 200 rate is high but crawl lag grows, shift effort to sitemaps and internal links rather than more sends.

Build a minimal dashboard for operators. Show queue depth by priority, sends today versus cap, last 20 log lines, and dead letter count. Restrict access to admins, escape output, and never display full tokens or private keys. For WP or small teams, a CLI status command that prints the same numbers fits version control and avoids extra UI. Review the dashboard each morning during backlog periods and weekly otherwise.

Close the loop each week with a short written summary that names the batches sent, the 200 rate, the 429 and 403 counts, and the oldest pending item age. Attach the summary to the same folder as the source lists so future audits can trace decisions. If crawl evidence from server logs shows submitted P1 URLs receiving Googlebot hits faster than non submitted controls, note the observation with dates and sample sizes without claiming causation. This habit builds trust in the bulk process and keeps pacing decisions grounded in observed data across teams and quarters.

Recovery after quota exceeded

Quota exceeded means pause and triage, not force retries. When 429 persists across hours, stop the worker, record daily usage, and note the reset window. Keep the queue intact with priorities preserved. Update sitemaps with accurate lastmod, strengthen internal links to P1 URLs, and fix any 403 or 400 rows that would waste budget on resume. Communicate the pause with a short note: usage today, top priorities queued, resume time, and expected daily throughput. This clarity prevents duplicate manual sends from teammates during the pause.

Resume with smaller batches. On the next window, send half the normal batch size with longer sleeps for the first hour, then restore if 200 rate holds. Requeue valid dead letter rows once after fixing causes. Do not resubmit already sent 200 rows to catch up, since duplicates waste quota. If backlog still exceeds several days of budget, reprioritize and defer P3 indefinitely to sitemap coverage.

Prevent repeat exhaustion with caps and schedules. Set daily caps in code, not just in docs, so the worker exits at 80 percent automatically. Spread bulk across days with cron windows rather than manual bursts. Require list review for any batch over 100 URLs, with sign off on priority and property match. These guardrails turn bulk from a risky event into routine throughput.

FAQ

How do I bulk submit urls google safely without hitting quota?

To bulk submit urls google safely, start by confirming ownership in Search Console and cleaning your list to canonical 200 URLs only. Split the backlog by priority, then send 10 to 20 URLs per batch with 3 second sleeps and 60 to 120 second pauses between batches. Track daily usage and pause routine sends at 80 percent of known quota so urgent fixes still have room. Respecting bulk indexing limits in code prevents extended 429 throttles that stall the whole queue. Treat a large mass url submission as a multi day plan with sitemaps and internal links covering lower priority URLs while the API handles P1.

How should I handle indexing api multiple urls in one run?

Handling indexing api multiple urls in one run works best with a persistent queue table that stores url, type, priority, tries, and next retry timestamps. Claim rows before sending so overlapping workers never duplicate, then send one POST at a time with a shared token and paced sleeps. A simple batch indexing api worker that processes 15 URLs per cron run with 3 second gaps keeps per minute rates safe. Log every attempt with batch ID, code, and notifyTime, and move failures to retry or dead letter after three tries. Keep an indexing queue dashboard showing depth by priority, sends today versus cap, and oldest pending age for operators.

What should a reliable bulk indexing script include?

A reliable bulk indexing script reads a CSV with url and type columns, normalizes to canonical form, and validates status codes before any send. It fetches one access token, reuses it for the whole batch, and runs a single threaded indexing api loop with sleep calls between sends. On 429 it backs off for 5 minutes with jitter, on 403 it exits with an alert, and on 5xx it retries once after a pause. It writes JSON line logs with timestamp, batch ID, code, and notifyTime, tracks daily caps in code, and supports dry run flags. Start with 5 URLs, then 20, then full P1 slices for safe scaling.

Why does an indexing queue prevent quota burnout?

An indexing queue prevents burnout by turning bulk sends into paced, auditable work instead of tight loops. Each row holds url, type, priority, tries, and next retry time, so the worker knows exactly what to send next and when to pause. Daily caps are enforced in code, with P1 new pages first, P2 major updates second, and a 15 percent reserve for urgent fixes. When 429 appears, the worker sets a global pause timestamp and exits, letting the next cron run resume after the window. Operators see depth, sends today, and dead letter counts at a glance, which stops duplicate manual sends during pauses.

What batch indexing api pacing works for beginners?

Beginners should run a batch indexing api worker with one thread, 10 to 20 URLs per batch, 3 seconds between sends, and 90 seconds between batches. That yields roughly 300 to 600 sends per hour without bursts, which fits many daily quotas when spread across the day. Confirm your own bulk indexing limits in Cloud Console before the first run, since new projects often start near 200 publishes per day plus per minute throttles. If 200 rates stay above 95 percent for a week, hold settings rather than pushing faster. If 429 appears, halve batch size and double sleeps for the next run.

What bulk indexing limits should I code into the worker?

Code both daily and per minute bulk indexing limits so the worker exits automatically instead of relying on memory. For example, cap routine sends at 80 percent of known daily budget, allow 10 to 15 sends per batch with 2 to 5 second sleeps, and pause 60 to 120 seconds between batches. Do not try to submit 1000 urls google in one day, since caps and throttles will stop the run early and extend the throttle window. Send each bulk urlnotification with logging, spread the backlog over days, and let sitemaps cover lower priority URLs. Review Console quotas before large imports and adjust caps to reality.

How do I write a safe indexing api loop without triggering 429?

A safe indexing api loop sends one URL at a time, sleeps 2 to 5 seconds between sends, and pauses 60 to 120 seconds between batches of 10 to 20. Reuse one access token for the whole batch, log code and notifyTime per URL, and check daily caps before each send. On first 429, pause 5 minutes with jitter, halve batch size, and double sleeps on resume. Do not add parallel workers to catch up, since concurrency extends throttles. Plan any mass url submission over days with priority order, so P1 new pages clear first while sitemaps carry the rest without quota pressure.

Sources

• https://developers.google.com/search/apis/indexing-api/v3/prereqs
• https://developers.google.com/search/apis/indexing-api/v3/quickstart
• https://support.google.com/webmasters/answer/7440203
• https://schema.org/JobPosting

Further reading

• Quota exceeded recovery playbook
• No code Indexing API use