8

Back in the 90s I was using Teleport Pro.

These days what would be the right way to save a complete copy of a website?

(including files from CDN / fonts / source maps / media)

One option would be to write a simple Node.js app using npm modules such as request or cheerio, but I don't want to reinvent the wheel.

(hence asking)

I'm primarily on Mac, can use Windows if software is worth it.

Improve this question
3
  • 1
    What's the issue with Teleport Pro?
    – Franck Dernoncourt
    Jun 27, 2014 at 17:50
  • Not sure what is the intent of the question. I actually upgraded from Word 2.0 on Windows 3.11 and I was hoping that there are modern apps serving my need... (I remember I had some issues)
    – Mars Robertson
    Jun 27, 2014 at 18:56
  • I was just curious to know how Teleport Pro is doing nowadays (e.g. maybe issues with new OS or new web technologies?) as I also used to use it one decade ago :) wget and HTTrack Website Copier are fairly old too btw.
    – Franck Dernoncourt
    Jun 27, 2014 at 19:06

3 Answers 3

Reset to default
10

You can use GNU wget:

  • free and open source
  • Linux, Windows, Mac (brew install wget)
  • CLI

To create an offline copy of the website (aka. mirror it), use 

wget --mirror --page-requisites --convert-links http://stackexchange.com
  • --mirror: This option turns on options suitable for mirroring. This option turns on recursion and time-stamping, sets infinite recursion depth and keeps FTP directory listings. It is currently equivalent to -r -N -l inf --no-remove-listing.
  • --page-requisites: This option causes Wget to download all the files that are necessary to properly display a given HTML page. This includes such things as inlined images, sounds, and referenced stylesheets.
  • --convert-links: After the download is complete, convert the links in the document to make them suitable for local viewing. This affects not only the visible hyperlinks, but any part of the document that links to external content, such as embedded images, links to style sheets, hyperlinks to non-HTML content, etc.

FYI: JavaScript Functionality For Wget.

Improve this answer
1
  • The Windows wget link is now outdated - that version doesn't support SSL correctly. More recent versions can be obtained at eternallybored.org/misc/wget
    – Simon East
    Oct 10, 2019 at 5:44
7

You can use HTTrack Website Copier:

  • free and open source (GNU General Public License Version 3)
  • Linux, Windows, Mac
  • GUI

enter image description here

enter image description here

Improve this answer
3
  • 1
    Hey that's what I was going to recommend when I saw this question ;)
    – Nick Dickinson-Wilde
    Jun 28, 2014 at 0:43
  • @NickWilde Yeah that's a pretty nice application, but I've always being pissed about the ridiculous bandwidth limitation by default so I tend to recommend wget instead :)
    – Franck Dernoncourt
    Jun 28, 2014 at 0:48
  • Unfortunately HTTrack doesn't download CSS, JS or images by default. If you attempt to include them with an include rule like +*.css, this sometimes leads the app into trying to download the entire internet! Argh. You need to specify complex include/exclude rules to work around this. Wish they didn't make this so painful. 😒
    – Simon East
    Oct 10, 2019 at 4:41
4

...what would be the right way to save a complete copy of a website?

The short answer is that this is not generally possible (in the sense of "complete") except for static HTML sites.

Today's modern "responsive" sites rely on JS that dynamically updates the DOM with AJAX, and unless you are prepared to execute that JS and re-render the DOM you won't get the complete site.

To see what I mean, consider what it would mean to "save a complete copy" of, say, maps.google.com.

Improve this answer
4
  • Indeed, e.g. wget has issues with JS: wget.addictivecode.org/FeatureSpecifications/JavaScript
    – Franck Dernoncourt
    Jun 28, 2014 at 4:29
  • Precisely. That is why I'm looking for a piece of software that can do that for me... :) #lazy
    – Mars Robertson
    Jun 30, 2014 at 7:18
  • You missed my point. What is an "offline copy" of a website with dynamic content that changes every few minutes (like cnn.com, for instance)?
    – Jim Garrison
    Jun 30, 2014 at 15:27
  • I don't expect realtime updates on the offline copy. Current snapshot would suffice.
    – Mars Robertson
    Jul 1, 2014 at 0:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.