I am looking for a solution to regularly transferring large amounts of data from one computer to a network of multiple computers via an external drive.

It currently works like this: The source computer runs OpenVMS and the files are transfered via FTP to the external drive using a windows computer. The OpenVMS systems does not organize or name the files in a usable way so they are renamed and reorganized using an automatic script running on windows. The external drive is then transported to another location where I have a system which contains dozens of TB of data distributed across multiple windows computers which are connected in a network (any computer in the network has access to all relevant drives on all computers). The files are then manually sorted into the folder structure. In an average week 5 - 50 GB of data (2000 - 30000 individual files of varying size) usually distributed across dozens of folders are transported this way.

As you can imagine this system is rather time consuming and error prone. Due to size limitations I have to regularly remove old data from the OpenVMS system so I have to be certain that all necessary data has been transfered to the other system before doing so.

I am imagining a program which does the following: Creates a database or mock folder structure with empty files of the target system on the external drive. I can then transfer new files from the OpenVMS computer to the external drive, setting the desired location in the database/mock file structure. At the target system, the program would then transfer the newly added files to the desired location, update the database/mock file structure and remove the transfered files from the external drive.

Does any program exist that does that or is there any other solution to this problem?

  • If you external drive is not a SSD and connecting via USB 3 or 3.1 or 10 gigabit it will at least speed up the transfer of files. – cybernard Apr 15 '17 at 16:23
  • If you have scripting to re-organize the files, why does that script organize them according to the file destination? For example say you at the remote site, and the removable drive is N. Is it possible to have folders like computer1,computer2,computer3, or etc. Then when you get to your destination, net use r: \\computer1 xcopy n:\computer1*.* r:\ net use del r: net use r:\computer2 xcopy n:\computer2*.* r:\ ?? – cybernard Apr 15 '17 at 16:31
  • Also do the systems have to remain disconnected? Potentially, lets say it takes an hour each way to get to 2nd site, plus $.57 per mile for gas and wear and tear. At $20/hour 2 journeys *4 weeks = $80 plus say 10miles so $5.70. The cost is now around $100+/month which you might get an afford cellular connection and/or standard cable internet for that cost or less. – cybernard Apr 15 '17 at 16:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.