I'm a little distressed, for my job I need to get some info from a website, specifically http://www.museumsusa.org/museums/ , I need for every museum in America to get the name, address, phone number and e-mail address, if they exist.
I would prefer not to copy-paste 20000 strings; does anyone know of a tool to help me?
The answer depends on your skills.
I would tell the problem to my friend ruby. She probably will use mechanize to do the job.
If you need some guidance, consult the search engine of your choice for a book or video with the term "web scraping" in it's title. I checked Safari, and guides for JavaScript, Python, Java turned up in the search results.
If you go the coding route, remember to be nice with the server operators.
If coding is not your thing, there used to be software for clicking together bots in a GUI like Kapow out there, but I am not up to date regarding the state of the art here. Kapow seems to have turned into this.
