This isn't really the answer your looking for, feel free to down vote me for this if you would like, but as a programmer I want to suggest you be very...cautious implementing this approach, even if you find a tool to make it possible. It is very easy to end up writing test questions that focus on irrelevant details that could exclude great programers who haven't memorized meaningless details; while not doing much to filter out bad programers. You want to focus more on critical thinking and understanding of high level concepts then raw knowledge.
You don't want to find a programer that has memorized which methods are available for an object or how one configures colors in an application. Their response to any question like that should be "I would google that", and it's the right answer. These things can be figured on on the job without much effort, and so there is no need to know them from memory.
A good programer can come on to a team without more then a cursory knowledge of the programing language at all and still contribute within a few weeks, and be at 100% by the end of the month, because the concepts of programing are shared regardless of specific language. However, if you started asking this same programer specifics about the language he may bomb horrible since he hasn't memorized that languages syntax and methods yet, despite knowing what is required to be a great developer.
If you want an automated testing platform I would instead suggest some sort of fizz-buzz approach. Get a half dozen or so fizz-buzz style questions. Fizzbuzz questions are simple programs that don't take long to write and have rather straight-forward requirements, used to see if a programer can actually utilize his knowledge to make a computer do something, and are a mandatory first step for screening programers.
Write very detailed requirements (ambiguity is unfair if it's automated) for what you want. Then let the programer write up code to do what he wants and when he submits it have the code run through some automated unit tests to verify it meets the requirements. This could quickly filter out bad programers. Unfortunately, it would be hard to provide him an enviroment rquired to do this fairly, since he would pretty much need a full IDE he knows how to use with the ability to do his own sanity checks on his code before submitting it if your going to automate it.
To get around the need of handing him a full development enviroment, enough time to program, and hoping he knows the language your testing, I would instead suggest a take-home variant. Give him an (only slightly) harder take home 'project' to write and send back within the next day or two, something that can still be done with only 30-45 minutes effort at most. It could be automatically submitted when done and go through some unit tests to verify it does as required, and since it's take home the developer can use his preferred IDE and development enviroment. It wouldn't be too hard to set up, so long as you know ahead of time what language the developer will use so you can write unit tests for that language. In this case it's even fair to specify a language to write the code in, even if it's one the developer is less familiar with, since your allowing him time to research syntax on his own and a good developer should be able to figure out syntax of a new language quickly, especially if he is doing something as simple as a fizzbuzz.
Yes some people could go home and get others to help them with the problem, even if you specifically ask them not to, but that's fine. This whole process would (hopefully!!!!) not be the sole step in interviewing. It will weed out most of the duds prior to the real interview for minor expense, and if someone cheats you can still catch him in the real interview later. I have seen companies do something similar to this in the past.
If you do use automated test as you suggested make sure you are not asking for specifics of a language or configuration, unless you also give them access to google. Focus on higher level concepts like "what design mistake was made in this code snippet" or "what is a datarace and what techniques are used to avoid them" etc.