Skip to content

Crawl a Shopify website to extract all URLs and submit them to IndexNow

Notifications You must be signed in to change notification settings

vallieres/crawl-n-indexnow

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

21 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Crawl n' IndexNow
Crawl n' IndexNow!

Get the goods and ship 'em to the indexes!

Crawl n' IndexNow is a simple CLI that pulls your Shopify or regular website sitemap, and submits the URLs to IndexNow to speed up the indexing process.

⚡️ Quickstart

Install the CLI:

go install github.com/vallieres/crawl-n-indexnow@latest

🎯 Commands

Submit All Shopify URLs to Index

crawl-n-indexnow shopify --domain legacygoods.co --key a1b3c34d

This will crawl the Legacy Goods Co. domain, list the found URLs from its sitemaps and submit them all to IndexNow.

Submit All Sitemap URLs to Index

crawl-n-indexnow sitemap --domain hipsterpixel.co --key a1b3c34d

This will crawl the Hipster Pixel domain, list the found URLs from its sitemap and submit them all to IndexNow.