There are so many open source frameworks that help doing analysis by SQL-like query on distributed system. We are currently looking for good solutions for our case such as Apache Hive, Tajo, Tez, Impala, PrestoDB and Spark(Shark) etc.
The main job we need to do is analyze the access log data with our user data. For example, we want to aggregate user's traffic based on our marketing campaign or analyze user's activity based on their tracking log.
Here are some priority we concerns about:
- Robustness and Stability
- We hope the solutions that are robust on the situation of OutOfMemory so it rarely be crashed.
- Can use JOIN functionality
- Easily interfaced with rails application
- We are looking for the solution for background batch job not real-time.
By now, we think PrestoDB is the one we are looking for but I want to here another opinions from experts.
Is there anyone who can help me?
Thanks in advance.