It is well recognized that at very large scale (tera to peta and over bytes of data), column stores or something specifically custom-built are the default choices.
What are off-the-shelf SQL based choices before one gets to petabyte scale? In particular, considering a multi master setup. Or, is sharding the real answer to these kind of scalability questions regardless of database choice?
In particular, some of the main requirements are:
- easy to start with
- time tested solution
- supports sharding (scalable horizontally across tens to thousands of servers) out of the box or with off-the-shelf tools
- good built in text search or easily integrable with lucene, solr, etc.
- ACID compliant
- fast read
- concurrent read-writes
- good support for variety of indexing techniques
- open source and decent community support