You can use GNU wget:
free and open source
Linux, Windows, Mac (brew install wget)
To create an offline copy of the website (aka. mirror it), use
wget --mirror --page-requisites --convert-links http://stackexchange.com
--mirror: This option turns on options suitable for mirroring. This option turns on recursion and time-stamping, sets infinite ...
...what would be the right way to save a complete copy of a website?
The short answer is that this is not generally possible (in the sense of "complete") except for static HTML sites.
Today's modern "responsive" sites rely on JS that dynamically updates the DOM with AJAX, and unless you are prepared to execute that JS and re-render the DOM you won't get ...
There are some addons for Firefox that come very close to your goal:
Open in Wayback Machine
Right-click link, chose "open in wayback" – and if it works, copy over the URL:
Screenshot from the addon's page at AMO
As the name suggests, this supports the "Wayback Machine" (aka "Internet Archive") only.
This is a quite fresh addon, not ...
This extension is quite famous, it has even got a Wikipedia entry. The following description is from Firefox add-ons:
ScrapBook is a Firefox extension, which helps you to save Web pages
and easily manage collections. Key features are lightness, speed,
accuracy and multi-language support. Major features are:
Save Web page
One possibility, which I have to admit I haven't tried, is to use just about any compression software that can create a ISO or ZIP file, some of the mount tools can do this for you and some support other formats as well, to compress/save the directory tree to a single file and then use a tool that will allow you to "mount" that file as if it were a separate ...
Judging from the title (i.e. ignoring the "windows" tag and the "port 25" you want to connect to), I'd recommend either Archivemail or offlineimap.
You can read about Archivemail in it's man page. From the synopsis:
archivemail is a tool for archiving and compressing old email in
mailboxes. By default it will read the mailbox MAILBOX, moving
I have finally decided to settle for Git and Markdown. For Git hosting I chose GitLab because it is basically GitHub with addition of free private repos. The idea is to have a single repo for all logs and then have a single dir for each log or entry:
I also plan to ...
I’m using a wiki for a similar use case.
The good thing: You can build the site architecture/organization/navigation yourself.
The bad thing: You have to build the site architecture/organization/navigation yourself.
Almost any wiki should be suitable for your case, but it’s worth looking into details you care about.
Wikipedia has a list and comparison of ...
Facebook has an option to download most of your Facebook data, including sent and received messages. Go to your Settings and click on “Download Your Information” :
It is gratis, no app is required and it grabs your whole chat history. One issue is that you cannot select what you want to retrieve, so if you have 100 GB of videos I hope you have a decent ...
tl;dr: Use digiKam, edit the settings so it saves metadata to image files themselves, use the caption field.
I researched this a little a few months ago. Here is what I came up with.
There are several metadata formats available for images, but the most widely used is EXIF. This is important, because I found that most programs that can edit photos or ...
The wayback machine is open source:
It is implemented in Java and uses hadoop. It looks like both the web crawler and the web front end (the very one you cite in your question) are there.
However, the wayback machine is a website, not a project that has been reused by many people and carefully honed to be an ...
I would suggest python + scrapy to scrape the web sites on a regular basis - you may also need the requests library for some pages - and storing the results in a directory structure that is version controlled with Mercurial or possibly using MongoDB.
Python is Free, Open Source and fast to develop with.
Scrapy is very good at scraping static pages, (...
The request is quite specific, I'm not sure if something with that functionality exists out-of-the-box.
My suggestion would be to use something like a custom script, which would
also give you good flexibility.
Since you are using Windows, I quickly developed a simple PowerShell script for you.
Create a .ps1 text file (e.g. Create-MultiZip.ps1) with the ...
You want to use a relational database. For "basic user" friendliness I would start with MS Access. If you want to use something that is more programmer friendly start with MySQL.
You will find that there are many options out there for relational database, so to choose the road you are going to travel, look ahead at the places you might want to go with ...
Have you tried parchive (short for parity archive) checksum files (https://en.wikipedia.org/wiki/Parchive). There are several implementations of generators around for different OSs.
In this model the files under consideration are cut into blocks and parity information is stored in separate files with choosable redundancy.
Let's check your points:
Well If you want to resize or use lossy compression there is Handbrake, You might like it If you want to compress raw videos, like screen capture. Otherwise modern video codecs are already capable and there is high chance your video is as compressed as it could be, without loss of quality.
Hand brake can compress, i.e. either resize or use better codecs, ....
They Wayback Machine has an JSON based API available as well as Memento protocol and some deeper capture data - https://archive.org/help/wayback_api.php
PHP is trivial to learn, does GET and POST requests with native functions or curl support, and has easy json parsing to associative arrays.
I just want to finish this Q&A with an actual accepted solution.
I have written POSIX shell scripts available on GitHub using openssl for both encryption and decryption of my files.
So far they are very simple, I intend to make a few enhancements this year.
You can use LogicalDOC for this, it is a complete document management platform, and can do the most simple to the most complicated tasks, however I think you wish for a more simple solution. I would then ask you: are these file with important/classified content which you would be concerned?
If the answer is yes, then I would again suggest LogicalDOC. If the ...
After reading a lot of manual pages, I can recommend the following:
xz (man page)
As it has in-built integrity check adjustable to SHA-256 using the following switch:
You may specify the number of CPU threads using the following switch:
The default compression level switch is set to 6, you may change that to ...
I read the features description of bup (interesting) on GitHub and maybe you should look into using good-ol' DAR, if I understood correctly what you need.
Actively developed software, incremental/differential backups, good encryption, data protection through redundancy (using Parchive, just as bup if I'm not mistaken), and multi OS support (with official ...
Maybe you should consider Owncloud? Here is a quote from it:
Access your data from all your devices, on an open platform you can extend and modify.
The way I often introduce OwnCloud to somebody new to it is like "Think of it as your 'private' version of Dropbox, somewhere in the cloud. Where 'you' set the terms and conditions for anybody you authorize ...
You can do the same without any email client. I have experienced lagging and other issues with Outlook. Mail Clients often skip the attachments and the contents of the original message. That results in an incomplete download of emails to local storage. Relying on them for a local backup is mere foolishness.
I may suggest you a paid tool that archives the ...
Personally I would look at using python and either its built in email library or the gmail api.
This would have a number of advantages:
Free, gratis & open source
Cross platform Windows, Linux & OS-X
You can structure the storage as you choose
You can choose the storage formats
with the Gmail API you should be able to preserve tags
python can ...
If you can get your mailbox into an MS Outlook PST format on your local machine, X1 Search, http://www.x1.com/, works well locally. I am using it to manage my current slice of mail, going back to 2008, about 4.2 GB of mail, about 10K of messages, and the existing PST structure is kept. I have an offline mail store that has been moderately curated with my X1, ...
i'm writing a simple Email Viewer for Linux/GTK. you may check it out: gtk email viewer
It can open raw rfc822 format files (they often have .eml extension), show headers, plain text content, html/pdf/rtf content (with converter helper tools: lynx, w3m, ...), image attachments, can extract/save attachments, compose new email, attach files by drag-n-drop, ...
I’m partial to the Nero product line. It’s not free, but, worth the money (currently $30 US) considering the product offerings. In your case, I’d suggest taking a look at the Nero Burning Rom 2016. While I’ve not used it to burn Blu-ray discs, I’ve used it to burn lots of DVDs and to make copies of data. The software also provides the ability to protect ...
This would be quite simple to achieve with python scripts and a very little work.
The trick would be to read in from either the command line or a file a list of the items to be duplicated and a destination then use the brilliant library item os.walk to find the duplicate the items in the destination.
Free, Gratis, Open Source & Cross Platform
You could ...