What Mac program or browser plugin can I use to download all hyperlinks within a text file?

Question

Here is an example of the .rtf file containing numerous hyperlinks:

I just want to know if there’s any app that will batch download all of these images so I won't have to manually save each image one-by-one.

Thank you!

Answer 1

Use a simple python script with the requests library.

First get requests, or update it if you already have it, with pip install -U requests

Then open a python console in a directory containing your URL list and type:

from __future__ import print_function # Not needed if python 3.x
import requests

with open('url_file.rtf') as infile:
    for line in infile:
        url = line.strip() # See Note 1
        res = requests.get(url)
        if  res.status_code == requests.codes.ok:
            outname = res.url.decode().split('/')[-1] # Note 2
            with open(outname, "wb") as outfile:
                 for chunk in res.iter_content(100000):
                     outfile.write(chunk)
            print (outname, "written")
        else:
            print("PROBLEM WITH:", url)

Note 1: This assumes a each line contains only a valid URL - you could use the python re, or regex, libraries to separate URLs from other text.

Note 2: This line is making the filename from the URL. If your URLs have some URL encoded characters such as %20 you may need to use the res.url.decode() method to convert these so as to get a valid filename.

Note 3: If you need to do this regularly you can store the above, possibly with some minor modifications, in a file ending with .py and then run it when needed.

Answer 2

Mac OS X contains the bash shell, and you could use Bash Shell scripts, along with your text file as input, invoking wget or curl, or you might even be able to do this with OS X built in utilities like Automator.