0

I am in need for a database that can store very long tables.

E.g.

MYTAB
-----
name STRING
date DATETIME
mark INTEGER
value DOUBLE


"A.112.akizo", "2023-01-01 01:00", 0, 1234.54
"A.112.akizx", "2023-01-01 01:00", 0, -33.00
"A.113.boxrw", "2023-01-01 02:00", 0, 45.32

name and date would be my primary-key.

I need to store millions of rows per year. I will select by name/date and also update. Do not need to have references to other tables. No need for SQL but addressing by name/date is mandatory.

My 1st approach would be a normal table in a DBMS (like MariaDB or PostgreSql) and maybe have it for each year if it gets too long (e.g. MYTAB_2022, MYTAB_2023).

Not sure if there are better ways for handling this.

2 Answers 2

1

Since you don't need relations or SQL, have you considered NoSQL? Such as DynamoDB that definitely comes with scalability. And if you need a more local and free solution, since you didn't state, then e.g. Elasticsearch could work.

1

You could try a Timeseries Database. Or a database for logging data such as RRDtool which is mature.

Third-party tools couldhelp visualize these datasets.

For nanosecond-resolution you could try Prometheus which is a monitoring system with a time series database. Which product you choose depends on your requirements and preferences.

If you need fast loading times and your focus is on in-memory analytics, you could try Apache Arrow or DuckDb for instance

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.