Get difference of two regular literature texts (not source code), with markup on changes

Question

How to get difference text of two ebooks converted to plain text? They are both the same book but different editions, so much of text is exactly the same to within hyphenation and punctuation changes.

I have read previous edition so I would like now only skim over changes of new edition, and not reread the whole new edition. And this could be helpful not only for one book but for many!

Things I found, but they do not solve the task:

• 'diff' utility and diff-type GUI utilities are supposed for use with software source codes, since their unit of comparison is a single line of text, which is not the case for text in natural language, which flows naturally not limited by lines,
• 'wdiff' and 'dwdiff' utilities are supposed for use with ordinary text, not source codes, since they perform comparison by 'words' considered as a result of text splitting by some specific delimiters. But in practice due to some unknown reason they both fail to find any similarities in identical parts of books, like same sentences, even entire same chapters
• anti-plagiarism software simply doesn't allow such functionality, they are big complex specialized software, working interactive, without ability to batch produce the "diff of natural text" output

Any local operating system or remote online service will do.

Answer 1

Assuming you can use a console program for this, I'd suggestvimdiff. it's a tool packaged with the Vim editor that takes two files as input, and presents them side-by-side with synchronized scrolling and special highlighting to mark which lines are different and what has actually changed on a line. It's smart enough to handle blocks of text moving (but not necessarily complex reorderings of text), and will also automatically fold (hide) sections of the two files that are identical.

Dependent on how the text actually flows in these two ebooks though, you may have issues finding any differences. For example, if a bunch of the paragraphs in one section are in a completely different order but are otherwise identical, than many tools won't recognize that they are the same (maybe they'll recognize one of them, but not all). This is simply a side effect of how comparison tools work, and you're not likely to find a tool that can properly automate finding such differences (and if you do, it's likely to be extremely specific in what it does).

Also, slight side note, but diff is not 'only for software source code'. It simply has difficulties handling naturally flowed text (like you will find in most ebooks, even if converted to plain text) because it's line oriented, not word oriented.