2

I have a question about programs for syncing large amounts of small files (on Windows). I have two 1 tb drives on two different computers (e.g. one at work and one at home). I want to keep these hard drives in sync.

Note that not only do I have a lot of data, but there are a lot of small files which seems to cause problems for many programs. They crash, use a lot of memory, or are very slow.

I have tried many programs (Google Drive, Dropbox, Microsoft OneDrive, Bittorrent Sync, and GoodSync), but all of them either continuously crash or use up all of the RAM on my computer.

So far the best program I have tried is oDrive. However, it is quite slow for my task, and keeps crashing. Therefore, I need to continually restart it. ODrive seems to actually be getting the job done though.

Note that I have an extremely fast network connection (fiberoptic with 100 Gb/s upload and 100 Gb/s download) so network speed is not the problem.

Improve this question
2
  • 2
    Your issue is not the file (drive) size or the total amount of memory, it is that you have so many small files. I am assuming that you use Windows (please add that to your question), it will handle the memory allocation for you. But even Windows Explorer is notoriously slow in handling a large number of (small) files. I have edited the question accordingly, if you disagree you can revert the edit.
    – user416
    Mar 27, 2016 at 10:01
  • Unison cis.upenn.edu/~bcpierce/unison Or rsync, but if you're not on a *nix that'll be harder to get going.
    – user23366
    Apr 30, 2016 at 4:03

2 Answers 2

Reset to default
3

Well, you don't specify the tag, so ...

I tried all the free synch programs. I even recommended some of them on this site (the best being http://www.freefilesync.org/).

But, in the end, I went with Bvckup2 at US $19.11 for a single license and discounts from the second license onward.

It, like DropBox, etc, mirrors in real-time, every time a file is changed.

From the web site:

Complete backups faster

The app is very fast at bulk copying and it is even faster at updating.

Extensively optimized across the board Bvckup 2 goes out of its way to process every backup as quickly as possible. Focus

Do just one simple thing...

The app does not compress, encrypt or FTP files. It's not a two-way sync nor is it a full system backup software.

It does just one thing - it mirrors A to B.

...and do it really well

It's better to excel at one thing than to be average in many.

From the nuanced user interface to the high-performance core, the app is built to always favor quality over quantity.

There are nice features , which you can see on the web site.

Since there is a free trial, it will cost you nothing to see if it works for you.

Personally, I have stopped searching and am happy to highly recommend this excellent software (with which I am in no way affiliated, other than being a highly satisfied user).

Improve this answer
2
  • Don't thank me, just upvote (wink). Seriously, though try it first, then upvote and /or accept as answer. That will show others with the same question whether it is good or not. Since you are new, I will remind you always to leave feedback. In fact, if you solve your own question the worst thing you can do is delete it, thinking that you no longer need an answer. Other people might, so you are encouraged to answer your won question, if no one else can, to help others with a similar problem. I do hope that this will help you; it is a great application.
    – Mawg says reinstate Monica
    Apr 1, 2016 at 9:32
  • 1
    bvckup 2 is awesome. I was running it since the beta days and I was pretty impressed.
    – Journeyman Geek
    Apr 30, 2016 at 11:32
0

Give Resilio Sync at try. I found it to work really well with syncing a large number of small files across my 3 devices (desktop, laptop, and NAS server). The only drawback I've seen so far is that it does tend to grow RAM memory over time; apparently the database it uses grows and never deletes old entries. The work around is to either a) don't use so many directories or b) rebuild the database on each device from time to time.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.