Screaming Frog SEO Spider

Screaming Frog SEO Spider is a website crawler software available for Windows, macOS, and Linux. It is designed for SEO professionals and agencies to conduct technical SEO site audits. The tool can crawl a website to identify over 300 SEO issues, warnings, and opportunities related to site health and user experience. Its key capabilities include finding broken links such as 404 errors and server errors, auditing temporary and permanent redirects along with redirect chains and loops, and analyzing page titles and meta descriptions for length, duplication, or missing elements. It can discover duplicate content using an md5 algorithmic check and identify low-content pages. The tool also allows users to extract data from HTML using CSS Path, XPath, or regex, review URLs blocked by robots.txt or meta robots directives, and generate XML Sitemaps and Image XML Sitemaps. Users can download and crawl up to 500 URLs for free, or purchase a license for €245 per year to remove the URL limit and access advanced features.