I am looking for an Open Source Java Solution that can be used in a small (2-4) cluster of linux machines. You can think on this like a farm of processing worker servers that are just listening to a message from a JMS endpoint to start processing.
This library/solution/whatever must be able to trigger around 10~20 processes in each machine from the cluster (each one is a JVM). Each process will consume a message from a centralized JMS instance and will save the job results in a centalized DBMS instance. Each process takes several minutes (5 to 50 mins) to complete and has little footprint in terms of network, disk IO, CPU and memory usage. Each job is independent. The library must just help to manage this allocation/deallocation of JVM processes and provide some minimal statistics and control. It's not necessary to pause/resume/cancel jobs. I just need to know when they're running or not, and if it completed successfully or not. Keeping idle worker servers is not a problem.
Important: I am not looking for PaaS or any cloud-based solution.
What do I know
Initially I've considered the idea of just start a bunch of tomcat instances, but it seems overkill and I'd have to provide each one of them different ports. It's not a divide and conquer problem, so I am not looking for map-reduce solutions. It's also not something to be solved using hadoop (I guess). But I confess I know little about this kind of solutions. I've read a little about JavaSpaces and RMI, but it seems these are building blocks for distributed solutions. I've also heard about microservices, but they just look like something more useful for orchestration of different parts of a whole process. I've also checked memcache, hazelcast, terracota, but they're intended to solve a different class of problem.
is that this is some sort of a well known problem with several insteresting solutions, but I just don't know exactly how it's called (and then I can't google it properly).