I'm building an application in C# that works on 64 bit only and requires a huge storage for binary data. Typically one would just allocate a buffer like
byte buffer = new byte; // 10 GB
Unfortunately, arrays can only be 2 GB or perhaps a bit larger, but far away from what I'm looking for.
The linked article suggests that one can "easily" use a list of arrays and then work on it. It turns out that it's not so easy, especially when it comes to performance of copying partial data from one array to another. I'd appreciate if there's already a stable and tested version of such a "large array" where someone has already put some thoughts into it.
Such a library
- should be gratis and open source
- available as a permissive license, e.g. MIT or Apache or Code Project Open License which allows making changes without the need of publishing the new source
- should support generics, e.g.
- works on .NET 4, 64 bit
- works on Windows, Mono support not required