就绪检查 · 启用通道 · 权重 5
已发布 robots.txt
/robots.txt 返回纯文本文件。
- 已观察到
- 79%
- 前 1K 网站
- 76%
- 前 1M 区间
- 81%
- 未测量
- 10,885
在 175,171 个已测量网站中
已测量 240 个
已测量 90,782 个
被阻止、遇到验证挑战或不适用
为何重要
抓取程序首先会在 robots.txt 中查找抓取规则和站点地图位置。如果没有该文件，它们会假定所有内容均允许抓取。
如何修复
发布 /robots.txt，其中包含您所需的抓取规则，以及指向 XML 站点地图的 Sitemap: 行。
我们如何检查
/robots.txt 返回纯文本文件。
已观察到：已找到 robots.txt。
未观察到：未找到 robots.txt。
局限：仅在检查时对标准位置请求了一次。
按受欢迎程度分档
| Chrome UX Report 区间 | 已观察到 | 已测量 |
|---|---|---|
| 前 1 千名 | 76% | 240 |
| 前 5 千名 | 75% | 1,628 |
| 前 1 万名 | 75% | 2,419 |
| 前 5 万名 | 74% | 17,917 |
| 前 10 万名 | 76% | 15,669 |
| 前 50 万名 | 78% | 46,516 |
| 前 100 万名 | 81% | 90,782 |
最可能这样做的类别
按技术栈
- PrestaShop on Cloudflare>99%
- BigCommerce>99%
- WordPress on Fastly99%
- Drupal on Cloudflare98%
- Drupal on Fastly98%
- Next.js on Fastly98%
- Tilda98%
- Next.js on Netlify98%
- Wix98%
- PrestaShop97%
- WordPress on Cloudflare97%
- Sitecore on Azure Front Door/CDN97%
- Squarespace97%
- Drupal97%
- Magento / Adobe Commerce on Cloudflare97%
- Drupal on Amazon CloudFront97%
- Bitrix97%
- WordPress on Google edge96%
- WordPress on Vercel96%
- Drupal on Akamai96%
- Astro on Cloudflare96%
- WordPress96%
- Sitecore on Cloudflare96%
- Gatsby on Amazon CloudFront95%
- Shopify95%
- Adobe Experience Manager on Cloudflare95%
- WordPress on Amazon CloudFront95%
- HubSpot CMS95%
- Next.js on Akamai95%
- WordPress on Sucuri95%
- Adobe Experience Manager on Akamai94%
- Angular on Cloudflare94%
- Magento / Adobe Commerce94%
- Sitecore94%
- Next.js on Azure Front Door/CDN94%
- Adobe Experience Manager on Amazon CloudFront94%
- Adobe Experience Manager93%
- Next.js on Cloudflare93%
- Astro93%
- Angular on Amazon CloudFront92%
- Next.js on Amazon CloudFront92%
- Joomla91%
- Next.js on Vercel91%
- Next.js on Google edge91%
- Angular91%
- Webflow90%
- Next.js90%
- WordPress on Imperva89%
- Astro on Amazon CloudFront89%
- Nuxt on Google edge89%
- Nuxt on Cloudflare87%
- Blogger87%
- Gatsby86%
- Nuxt86%
- Nuxt on Amazon CloudFront85%
- GoDaddy Website Builder83%
按提供商
- Bunny CDN93%
- Cloudflare92%
- Akamai91%
- Fastly89%
- Google edge88%
- Azure Front Door/CDN87%
- Amazon CloudFront86%
- Sucuri85%
- Netlify83%
- Vercel81%
- Alibaba Cloud CDN72%
- Imperva70%