We have just changed our website over to using ssl everywhere, but this is resulting in some broken embedded content in browsers.

Is there a tool I can use to crawl the whole site looking for this kind of broken content (mostly video) so I can fix/replace? There's enough content that doing this manually would be a slog. I already use linkchecker occasionally to find 404ing links and such, but this doesn't find broken embeds (afaik).


I suggest to try crawling the site with Screamingfrog.


I'd check your webserver log files, depending on what format (ie, Apache common log format) you should be able to see status code (look for 404), the ask-for-but-unfound resource, and the referring URL (what page teh link/embed was on)


If all of your content is supposed to be https or ssl then you should be able to crawl the site with Scrapy and check for links to on site but not the correct type with Beautiful Soup 4.

Just use Scrapy to find all of the pages, parse with BS4 and check the embed elements.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.