I am working on a macro benchmark (the Train Benchmark) in Java to evaluate the performance of various database management systems. I took some steps for eliminating potential threats to its validity: the benchmark is executed multiple times in the same JVM to allow for warmup and JIT compilation. However, I think this could be improved with a benchmark library.
I tried both caliper and jmh but these libraries are geared towards with micro benchmarks, i.e. benchmarks that can be implemented in a single method. Is there a Java library for macro benchmarks which
- allows the definition of complex workflows, e.g. (1) loading the data (2) querying the data (3) repeatedly updating and querying the data,
- takes care of warming up the JVM,
- provides classes to gather the benchmark results for later analysis?