SVN is actually worth a shot. TortoiseSVN has facilities to diff Office files (at least, doc(x) and xls(x)), it looks like this:
Though there are easy to install SVN servers with GUI like VisualSVN, an HTTP-based server is not even needed because TortoiseSVN can create and use repositories on the file system.
The only catch is the 3-way merge: I couldn't find any information about whether and how it's possible.
Git is even more convenient in the long run since it e.g. 1)allows to save incomplete work locally in stages and only upload it to the central server once it's complete and 2)allows to work offline. But it has a steeper learning curve.