I am looking for a solution to regularly transferring large amounts of data from one computer to a network of multiple computers via an external drive.
It currently works like this: The source computer runs OpenVMS and the files are transfered via FTP to the external drive using a windows computer. The OpenVMS systems does not organize or name the files in a usable way so they are renamed and reorganized using an automatic script running on windows. The external drive is then transported to another location where I have a system which contains dozens of TB of data distributed across multiple windows computers which are connected in a network (any computer in the network has access to all relevant drives on all computers). The files are then manually sorted into the folder structure. In an average week 5 - 50 GB of data (2000 - 30000 individual files of varying size) usually distributed across dozens of folders are transported this way.
As you can imagine this system is rather time consuming and error prone. Due to size limitations I have to regularly remove old data from the OpenVMS system so I have to be certain that all necessary data has been transfered to the other system before doing so.
I am imagining a program which does the following: Creates a database or mock folder structure with empty files of the target system on the external drive. I can then transfer new files from the OpenVMS computer to the external drive, setting the desired location in the database/mock file structure. At the target system, the program would then transfer the newly added files to the desired location, update the database/mock file structure and remove the transfered files from the external drive.
Does any program exist that does that or is there any other solution to this problem?