For such amount of data and such size will be good to use kind of NoSQL key-value database. This will give you option for better scalability in sense of disk space, paralleling search operations. One possible option is **MongoDB** with very powerful select type command and very good scalability. Other option is **etcd** which support extreme write speed (1000 records/s per host).