Does anybody know if there's any program/library able to extract "main contents" (like the Readability experiment) from a forum-like HTML page?
- should be able to extract:
- posts contents (and properly separate each post),
- "prev/next page" link,
- optionally also: authors, title, date(s);
- at minimum, should be able to properly process "most phpBB sites" (fuzzy, I know), and any Stack Exchange Q+A page;
- bonus points for a Stack Exchange "hot"/"new"/… listing page;
- bonus points for handling http://news.ycombinator.com comments pages, or main ("listing") page, or reddit main/comments pages;
- as to point above, should still be "generic", i.e. not specially-tailored to parse exact HTML layout of Stack Exchange/phpBB;
?mode=printis allowed, but solutions working without this get more "points".
edit: just to make it clear, please note I don't need help with the initial step of downloading the HTML, so answers like "use wget/curl" won't help me a bit.