2

How to get difference text of two ebooks converted to plain text? They are both the same book but different editions, so much of text is exactly the same to within hyphenation and punctuation changes.

I have read previous edition so I would like now only skim over changes of new edition, and not reread the whole new edition. And this could be helpful not only for one book but for many!

Things I found, but they do not solve the task:

  • 'diff' utility and diff-type GUI utilities are supposed for use with software source codes, since their unit of comparison is a single line of text, which is not the case for text in natural language, which flows naturally not limited by lines,
  • 'wdiff' and 'dwdiff' utilities are supposed for use with ordinary text, not source codes, since they perform comparison by 'words' considered as a result of text splitting by some specific delimiters. But in practice due to some unknown reason they both fail to find any similarities in identical parts of books, like same sentences, even entire same chapters
  • anti-plagiarism software simply doesn't allow such functionality, they are big complex specialized software, working interactive, without ability to batch produce the "diff of natural text" output

Any local operating system or remote online service will do.

  • For which operating system? – unor Mar 25 '18 at 13:50
  • No limits on that, any local operating system or remote online service will do – Mna Mar 25 '18 at 14:15
  • Did you try WinMerge? If you want the matches not the similarities it does the job. – onurcano Mar 25 '18 at 20:03
  • Yes, WinMerge JP and all in the list of diff alternatives are for line-oriented source code-type file comparison – Mna Mar 26 '18 at 1:55
1

Assuming you can use a console program for this, I'd suggestvimdiff. it's a tool packaged with the Vim editor that takes two files as input, and presents them side-by-side with synchronized scrolling and special highlighting to mark which lines are different and what has actually changed on a line. It's smart enough to handle blocks of text moving (but not necessarily complex reorderings of text), and will also automatically fold (hide) sections of the two files that are identical.

Dependent on how the text actually flows in these two ebooks though, you may have issues finding any differences. For example, if a bunch of the paragraphs in one section are in a completely different order but are otherwise identical, than many tools won't recognize that they are the same (maybe they'll recognize one of them, but not all). This is simply a side effect of how comparison tools work, and you're not likely to find a tool that can properly automate finding such differences (and if you do, it's likely to be extremely specific in what it does).

Also, slight side note, but diff is not 'only for software source code'. It simply has difficulties handling naturally flowed text (like you will find in most ebooks, even if converted to plain text) because it's line oriented, not word oriented.

  • I can use console programs, and I tried all mentioned 'diff', 'wdiff' and 'dwdiff'. Like vim has GUI counterpart, vimdiff has its version too: "gvim -d file1.txt file2.txt". While the presence of GUI is proper feature, the core problem remains. The problem is, as you mentioned, in the line-based comparison algorithm, which is not useful for comparison of naturally flowing text. dwdiff utility is better in this regard but yet not even close to satisfactory. I guess problem needs some more sofisticated approach, like using Natural Language Processing (NLP) algorithms to split sentences and words – Mna Mar 25 '18 at 12:06
  • Updated original question to describe work of dwdiff – Mna Mar 25 '18 at 12:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.