2

We have just changed our website over to using ssl everywhere, but this is resulting in some broken embedded content in browsers.

Is there a tool I can use to crawl the whole site looking for this kind of broken content (mostly video) so I can fix/replace? There's enough content that doing this manually would be a slog. I already use linkchecker occasionally to find 404ing links and such, but this doesn't find broken embeds (afaik).

5
  • 1
    Can you expand on what you mean by broken embedded content in browsers?
    – Tom
    Commented Sep 10, 2016 at 22:57
  • Videos for the most part, that the browser won't load due to being from a non ssl url. Ends up leaving a blank area on the page.
    – feroxy
    Commented Sep 10, 2016 at 23:48
  • Di dyou try Xenu's link sleuth? It has ben around for decades and works well fo rme - home.snafu.de/tilman/xenulink.html
    – Mawg
    Commented Sep 14, 2016 at 8:26
  • Are you using a CMS to power the website? For example, WordPress, If you are maybe look for a tool written for that CMS to scan the database and change the protocol for you? E.g. https://interconnectit.com/products/search-and-replace-for-wordpress-databases/. Commented May 14, 2017 at 4:34
  • Did you ever find a solution?
    – Mawg
    Commented Nov 13, 2018 at 12:59

3 Answers 3

1

I suggest to try crawling the site with Screamingfrog.

1

I'd check your webserver log files, depending on what format (ie, Apache common log format) you should be able to see status code (look for 404), the ask-for-but-unfound resource, and the referring URL (what page teh link/embed was on)

0

If all of your content is supposed to be https or ssl then you should be able to crawl the site with Scrapy and check for links to on site but not the correct type with Beautiful Soup 4.

Just use Scrapy to find all of the pages, parse with BS4 and check the embed elements.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.