How to handle concurrent requests in R

Question

I apologize if this is a neophyte's question. I have an AWS box running R and responding to requests to run an R function via a socket connection from another AWS box. Currently my R box runs just one instance of R, listening on only one port, but I need to be able to handle more requests from the other server than the single R process can handle. So questions come from a fixed IP and go to a fixed IP and I don't think I can get concurrency on a unique IP/port connection.

What's the best way to handle this? I could run several R processes on the R box each listening on a different port but then the client side would have to figure out which port wasn't busy.

Any suggestions? Thanks in advance.

Answer 1

I don't know if HTTP is an option for you, but you've asked for any suggestions. You could investigate the Rapache Project - R embedded inside Apache. (github) - Paper

Apache scales perfectly well and can handle with many incoming requests.

However -

• Rapache was designed for Apache 2.0 , a really old version. Current is 2.4.
• RApache might not be suitable for long-running processes, you need to keep the connection to the client open and display a progressbar or something
• You probably need an expert sysadmin to set everything up properly. I couldn't do it ad-hoc

I have played with it in ~2010. It worked in principle it but I've never really used it in production.