I am in need of operating system (which probably will be Linux) either bootable from USB or safely integrated with Windows boot system.
I need it to be able to mount ntfs disk, mount RamDisk, extract zip packages from ntfs to RamDisk, perform crazy data diffs, pack result to zip, write it to ntfs (or pendrive).
GUI is not necessary, I can google stuff up on phone. However I would prefer OS to be fully stand-alone for this task. I just want to plug it in and start diffing.
x64 solution would be great, as it should boost operations somewhat. (And use all my RAM)
I got 20 or so data sets, averaging around 10GB per set. So yeah, over 300GB of data. All but two (first and last) data sets will be used twice. Data sets in ~90% are composed of zip packages of which I need to diff content. I will be diffing corresponding zip packages between data sets, and they should easily fit into RamDisk.
As I am in need of processing (diff check) huge amount of data (to produce cascading diff subsets from series of sets) I thought about using RamDisk, and to further boost performance I want to remove Windows(with AV and other typical performance murdering stuff) overhead from both memory and computing speed.
I need to generate diff between each pair of following data sets, so in batch diffing I can leave one set from processed pair in memory to use in next diff check.
Resulting diffs will be at less than 10% size of full data set.