Around 150K of records daily, expect to grow 100% in a year. Average record size is below 200B. One day of records is around 30MB.
(That would be 11 GB of data at the end of the year, or 22 GB with the 100% growth). Max QPS is around 6.
Records will be added by either Go or PHP written app.
I'm concerned with data corruption.
What I would like:
- validation checksum
- or correction codes
I was considering HBase but it's like taking a sledgehammer to crack a nut.
I'm looking either for a file format which allows to skip/recover corrupted data or a minimalistic storage solution.