I am looking for a software than can manage practical test sessions for software engineering interviews, something like what Red Hat is doing in their certification grading, it will get you a virtual environment with a pre-configured desktop, i'm targeting to provide a very simple test, let's say you'll get an environment with a running Java application, you'll have few questions about changing colors, adding simplistic features or even changing some configuration, the system must be automated to provide random questions and make the grading without any human intervention.

HR people will get a result like:

  • Java rating: 7 / 10
  • programming: 5 / 10
  • Network: 4 / 10
  • Stress management: 8 / 10
  • ...
  • ...

Any idea about a software that can provide such testing platform ?

4 Answers 4


This isn't really the answer your looking for, feel free to down vote me for this if you would like, but as a programmer I want to suggest you be very...cautious implementing this approach, even if you find a tool to make it possible. It is very easy to end up writing test questions that focus on irrelevant details that could exclude great programers who haven't memorized meaningless details; while not doing much to filter out bad programers. You want to focus more on critical thinking and understanding of high level concepts then raw knowledge.

You don't want to find a programer that has memorized which methods are available for an object or how one configures colors in an application. Their response to any question like that should be "I would google that", and it's the right answer. These things can be figured on on the job without much effort, and so there is no need to know them from memory.

A good programer can come on to a team without more then a cursory knowledge of the programing language at all and still contribute within a few weeks, and be at 100% by the end of the month, because the concepts of programing are shared regardless of specific language. However, if you started asking this same programer specifics about the language he may bomb horrible since he hasn't memorized that languages syntax and methods yet, despite knowing what is required to be a great developer.

If you want an automated testing platform I would instead suggest some sort of fizz-buzz approach. Get a half dozen or so fizz-buzz style questions. Fizzbuzz questions are simple programs that don't take long to write and have rather straight-forward requirements, used to see if a programer can actually utilize his knowledge to make a computer do something, and are a mandatory first step for screening programers.

Write very detailed requirements (ambiguity is unfair if it's automated) for what you want. Then let the programer write up code to do what he wants and when he submits it have the code run through some automated unit tests to verify it meets the requirements. This could quickly filter out bad programers. Unfortunately, it would be hard to provide him an enviroment rquired to do this fairly, since he would pretty much need a full IDE he knows how to use with the ability to do his own sanity checks on his code before submitting it if your going to automate it.

To get around the need of handing him a full development enviroment, enough time to program, and hoping he knows the language your testing, I would instead suggest a take-home variant. Give him an (only slightly) harder take home 'project' to write and send back within the next day or two, something that can still be done with only 30-45 minutes effort at most. It could be automatically submitted when done and go through some unit tests to verify it does as required, and since it's take home the developer can use his preferred IDE and development enviroment. It wouldn't be too hard to set up, so long as you know ahead of time what language the developer will use so you can write unit tests for that language. In this case it's even fair to specify a language to write the code in, even if it's one the developer is less familiar with, since your allowing him time to research syntax on his own and a good developer should be able to figure out syntax of a new language quickly, especially if he is doing something as simple as a fizzbuzz.

Yes some people could go home and get others to help them with the problem, even if you specifically ask them not to, but that's fine. This whole process would (hopefully!!!!) not be the sole step in interviewing. It will weed out most of the duds prior to the real interview for minor expense, and if someone cheats you can still catch him in the real interview later. I have seen companies do something similar to this in the past.

If you do use automated test as you suggested make sure you are not asking for specifics of a language or configuration, unless you also give them access to google. Focus on higher level concepts like "what design mistake was made in this code snippet" or "what is a datarace and what techniques are used to avoid them" etc.

  • 1
    +1 (and wish it could be more). After more decades than I care to admit to, nothing annoys me more than "coding on paper" interviews. When I am interviewing, I would never, ever give them. "Tell me about your current/previous/favo(u)rite project works much better. I wouldn't even use this sort of a thing as a filter (I'm looking at you, Cisco!)
    – Mawg
    Mar 12, 2015 at 12:53

Do you mean "Real Coding Test" in which candidates write, compile and run real code and the software gives them score according to their code writing / performance skills, Or online tests which contains multiple choice questions which contains code snippets.

In case you referred to the latter, You can try Codelect.net, a low-cost & flexible tool which allows you to create your own quiz from multiple programming languages and difficult levels, for instance here is a score report for a quiz which contains Java JEE & JBoss enter image description here

If you prefer to select your own questions and have better control on the quiz content you can try LuGo-Test.com this service is more robust, although it cost more and has an annual subscription fee.

  • Am looking for a "Real Coding Test" solution.
    – zfou
    Nov 24, 2014 at 12:27
  • 1
    In such case, I recommend to invite the applicant and give him a programming assignment, can't see how it can be done by a service.
    – Uri Lukach
    Nov 24, 2014 at 13:41

I strongly agree with @dsollen's answer, especially as there are many ways to tackle most programming questions unless the scope is very narrowly defined and there are limited answers, especially as you want it to be automatically graded, scored for points, and that is the result they get. To answer your specific question...

The topics you have Java, Programming (general???) Networking (do you mean client server or do you mean configuring a Firewall and a VPN?), and Stress Management are four completely different areas and there are established norms for testing knowledge of each except JAVA & Programming which is more open.

What you are suggesting is a Programmer / Carpenter test be administered by a generic test system that will fit multiple candidates and skills and create some intelligle report that actually means something..which getting exactly that I think will be difficult and implies the hiring staff (manager/HR/etc) do not have the background themselves and are relying a generic test instead. I would alternatively suggest the manager/HR screen for personality and then have a programming team interview with the candidates if they get past that. That is what we do in combination with other non-technical skills tests.

Stress Management and other reasoning skills are offered through SHL-on-demand, it is adaptive testing that covers many topics including starting to expand into technical areas and is partnered with Brainbench.com.

I took the test for a job and found the results fairly in line with how I saw myself. I do not follow how puzzles pattern (boxes and circles) problems cover logical reasoning of technical people though.

On coding: I have seen DataCamp.com use something like what you are referring to for their "R" CBT's where you have to complete a code segment. It does look for some specific things but also provides some flexibility in my experience of taking them. They use an rFiddle interface and have some free tutorials if you want to check it out, but not sure what the validation engine is or if that is something just built into the fiddle platform.

Knowing it uses the Fiddle platform, I looked around and found this which doesn't have everything you are looking for...but this is supposedly a JSfiddle one. http://jsfiddle.net/73sz6o82/light/

There are others like TestGeeks that while still are multiple choice but their choices are lines of code which has to be placed in code segments.

Alternatively you can use the above as a guide for building your own test, if you can get it approved by HR, to use something like Lime Survey to build your own tests. It supports creating question pools, multiple surveys, and the surveys open on invite only, but does not support interactive coding.. which as previously mentioned is perhaps trying to use a stapler to test for the presence of water.


Try Oxcoder. https://www.oxcoder.net

  • A huge programming task library including Java, iOS, Android... at all difficulty levels.
  • Candidates can write, compile and run code on the test platform.
  • Solutions submitted by candidates will be automatically scored and analyzed.
  • Charge on each test session at very low cost.
  • 1
    Please disclose any affiliation you have with oxcoder. Oct 17, 2017 at 6:33
  • Thank you for your comment! Yes, I'm one of the oxcoder team members. We are a startup and trying to be known and supported. If my answer has offended you, please accept my apology. If you have any chances to use our product, please let me know. We will be very grateful if you can give us suggestions, advice, comments or remarks. Thank you again for leaving a comment!
    – Jasper777
    Oct 18, 2017 at 5:35
  • 1
    No offense taken at all. Please just update all your answers to include the fact that you are one of the team members. Oct 18, 2017 at 6:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.