Screaming Frog is probably the single most useful tool for performing website audits. The team behind Screaming Frog continues to add new features and functionality that keep it at the top of the market when it comes to audit tools.
The new features are amazing, but I find that a lot of people are not aware of many of the features that are already a part of this versatile tool.
Today I’m going to cover 3 features in Screaming Frog you probably didn’t know existed.
Change your storage mode to database instead of RAM for crawling large sites
By default, Screaming Frog uses your devices RAM for storing your crawl data, which makes its crawling really fast.
But what happens when you have to crawl a really large website?
This option will bring most desktops and laptops to a screeching halt.
Screaming Frog offers an option to deal with this. You can simply change the storage mode from RAM to database storage instead.
To engage Database Storage in Screaming Frog, simply go to Screaming Frog >> Settings >> Storage Mode. Then select Database Storage. Hit the browse button to select the location you want to use.
I use a folder I setup on an SSD drive.
I would only do this with an SSD drive, and as Screaming Frog’s tooltip suggests avoid using any drive that is synced to a cloud service or a network drive.
I use this all the time and honestly cannot see a difference in crawl speeds between this and the RAM storage option.
Crawl excluding URL parameters
This option is especially useful for crawling large ecommerce sites. Not excluding URL parameters for product sizes, colors, pricing, and other variations can result in a crawl with hundreds of thousands or even millions of extra URLs.
You could use Screaming Frog’s exclude feature to prevent crawling of these, but there is already a simple toggle baked in for you.
Simply go to Configuration >> URL Rewriting and then under Remove Parameters check the box that says Remove all.
Audit your image sizes and sizing
You can use Screaming Frog to find all of your images over 100 KB by simply selecting the Images tab. Then go to the Overview and select the filter Over 100 KB.
This will show you all of your images over 100 KB and their sizes.
You can also find incorrectly sized images. Web browsers will take an image at whatever dimensions it exists at and resize it to what the CSS code calls for before displaying it.
Web developers and website owners often upload images at much larger sizes than what they are displayed at. This creates images with file sizes that are much larger than they need to be.
Screaming Frog gives you an easy way to identify these across your entire website.
In the same Overview tab, you can also choose Incorrectly Sized Images.
