1

There's plenty of recommendations for website crawler software which can archive entire websites. However in recent years the nature of web development has advanced to the point where old tools simply don't work well any more.

For example, the following web dev techniques will not be captured by "traditional" scrapers such as wget or HTTRACK:

  • Lazy loading of images (i.e. images load only when scrolled into the browser viewport)
  • AJAX loading of whole pages or parts of a page
  • Infinite scrolling instead of traditional pagination to display multi-page content

Is there a modern equivalent software which can archive an entire website into an offline-browsable copy including all images, attachments, etc. even when the website uses such modern techniques?

The software would ideally:

  • Have a GUI and be easy to use without requiring knowledge of the command line etc.
  • Be free and/or open source

The software can either be downloadable or provided as a web service.

2
  • Any preferences to the OS they should support?
    – Izzy
    Apr 4 '20 at 19:34
  • 1
    @Izzy Not really. Open to anything that works well. Thanks.
    – WackGet
    Apr 5 '20 at 20:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.