7

TL;DR: I am looking for a web tool, API, program, script can check whether a URL exists in any of the Internet archives. I don't want to have to manually check Web archiving initiatives one at the time.

Background: I'm trying to mitigate link rot (e.g., 10% of the links posted here are dead on stack overflow. How do we deal with them?). I would love to have a web browser extension that can automatically add mirror links to the question/answer/comment being edited. The more of the Internet Archives listed behind the initial links are supported, the better – but I'm already happy if at least on of the major Archives is covered.

I know of this related question: API or library to obtain a mirror of a link, but it's about APIs. I prefer something ready-to-use for the end user. Any price, programming language, operating system, web browser or license is fine.

Reverse question: Archiving an URL on several Web archiving initiatives at once

Improve this question
8
  • 1
    Though you might have the entire park of MIT machines at your hands (any OS etc ;), there's one difficulty: Some of the archives are "members only" with access only from inside their intranet (e.g. Deutsche Nationalbibliothek). So I guess it's OK if those special cases are not covered, right? All the better if they are, but that might not be possible. // Besides: Didn't know there are so many! Always felt 80% of all website visitors are just crawlers...
    – Izzy
    Commented May 20, 2017 at 17:08
  • 1
    @Izzy Yes I am OK if those special cases are not covered :-)
    – Franck Dernoncourt
    Commented May 20, 2017 at 17:17
  • 2
    @Izzy I use computers most of the day and therefore need programs for various use cases :-) for this question I'm trying to mitigate link rot (e.g., 10% of the links posted here are dead on stack overflow. How do we deal with them?). I would love to have a web browser extension that can do github.com/soscripted/sox/issues/285
    – Franck Dernoncourt
    Commented May 20, 2017 at 17:51
  • 2
    Thanks for these additional details! Give a much clearer image of what you're after – so I took the freedom to integrate that with your original question (feel free to edit again if I overdid it). As for my original request: I'm not a native English speaker, so I obviously didn't get the literal "any" versus "each". So you're happy as long as long as at least one of those archives (with enough coverage) is covered, right?
    – Izzy
    Commented May 20, 2017 at 18:23
  • 2
    Not fully doing the job, but making it much easier: Open in Wayback Machine addon for Firefox. Right-click link, chose "open in wayback" – and if it works, copy over the URL. Also, MementoWeb Check sounds close to your goal: Check in one click if the page you are viewing has previous versions archived on popular archives websites – supports "archive.org, archive.is, etc". Not tried, though.
    – Izzy
    Commented May 20, 2017 at 22:49

2 Answers 2

Reset to default
4

There are some addons for Firefox that come very close to your goal:

Open in Wayback Machine

Right-click link, chose "open in wayback" – and if it works, copy over the URL:

OpenInWayback
Screenshot from the addon's page at AMO

As the name suggests, this supports the "Wayback Machine" (aka "Internet Archive") only.

MementoWeb Check

This is a quite fresh addon, not having any screenshots or reviews available yet (at AMO). Its description tells you to Check in one click if the page you are viewing has previous versions archived on popular archives websites. Note the plural: this one promises to support multiple archives. Explicitly mentioned are Archive.org (aka "Wayback Machine" or "Internet Archive") and Archive.is. As backend it uses timetravel.mementoweb.org – and on that page you can also find the other archives supported – quite a list I'm sure you will like!

Details provided by Franck:

The extension is also available in Google Chrome: Memento Time Travel:

Memento Menu Memento Preferences
Memento context menu / preferences (click images for larger variants)

It has already been useful to me by saving my time looking around for copies of a page when unavailable on the Wayback Machine (the extension I was using previously could only check the Wayback Machine).

The lookup time just take a few seconds, typically less than five seconds. There is no option other than choosing the archiving services. By default, it overwrites the page you're currently viewing, and as far as I can tell this cannot be changed.

Note by Izzy:

At least for now, Franck's descriptions only apply to the Chrome addon. On Firefox, all you get is an icon in your toolbar that you can click to "view archived versions and history on mementoweb" (for the currently open page) – no settings, no context-menu. Hopefully, there will be an equivalent available for Firefox soon.

Improve this answer
1
  • 2
    Thanks for the updates, Franck! That Memento thingy looks amazing, especially with its aggregator functionality. Love to hear how it works out in "daily business" (hoping for your update on that :) – Funny how my trying to improve your question turned out in finding myself something new and even meeting your requirements at the same time, I like that!
    – Izzy
    Commented May 21, 2017 at 16:20
2

They Wayback Machine has an JSON based API available as well as Memento protocol and some deeper capture data - https://archive.org/help/wayback_api.php

PHP is trivial to learn, does GET and POST requests with native functions or curl support, and has easy json parsing to associative arrays.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.