Local Plagarism Detection Software

Question

I am looking for software that does plagiarism detection, and has the following attributes:

1. Free (as in beer)
2. Windows (7, 8, 8.1, 2012 Server R2)
3. Command line interface
4. Gives matching text snippets

Nice to have:

1. 3-way comparisons (text in all, in pairwise documents, in only one document)
2. XML output

Totally willing to compile from source, any reasonable languages, or just use a Python library instead.

In case it was not clear, I only want it to detect plagiarism from the sources I pass to it. Does anyone know of software like this?

Answer 1

This is a tool that does all this: http://ssdeep.sourceforge.net

It's mostly designed for computer forensics - detecting similar, modified, tweaked or truncated files.

I have successfully used it for spamfighting by detecting emails that are modified in a degree from slight to significant when compared to other emails and spam samples.

I think it will fit your purpose because it has tunable "sensitivity" to changes and it is capable of detecting similar fragments, see here for demonstration:

http://ssdeep.sourceforge.net/usage.html#needles

Warning: I have used the shared library directly via Python cffi (https://cffi.readthedocs.org/en/latest/), not the commandline tool itself, although ssdeep does have commandline utility. Still, using cffi to make ssdeep library calls was more powerful than commandline and very easy too.

Answer 2

I was thinking that you could probably use Python and the SciPy NLTK to build a list of sentences from each of your texts that you are considering that there might be plagiarism from using tokenize - I would suggest removing punctuation from them and excluding any with less than 5 or 6 words and storing them, possibly with some sort of references to where they occur in the originating text.

You could then apply the same algorithm to your suspect texts and check for matches.

• Free both as in beer and as a bird.
• Just about any platform that can have python installed which does include Windows.
• Command line interface - yes unless you give it a GUI.
• The outputs could then be just about anything you like.
• Of your nice to haves you could add either or both.