I am currently working on a machine learning application that requires users to upload datasets, so naturally I want a database that has very fast batch-reads. Writing is only a one time process, but the learning model can be iteratively changed, and most optimization algorithms like mini-batch gradient descent use batch data for input.
The data will be in the form of tables. The dataset can involve any numerical set (floating and integer), like image sets. That would mean 10,000 columns for a dataset of 100*100 images like the CIFAR-100 dataset. So upto a few millions of columns is an upper bound.
MySQL is my preferred option here. But I am ready to work with any NoSQL database if it outperforms SQL database for this use case. Memory should not be a problem, as I am only expecting to do batch reads of upto 1000 rows at a time (and 100 on average).