Believe it or not, but the last companies and projects I worked for still managed their system tests as simple Word documents that describe the steps and the tester fills in the result.
As you can imagine, evaluating the results is tedius, since you need to look at each Word document and copy/paste the results into a "management" system (typically Excel).
I don't want to elaborate other deficiencies of that approach but I'd like to improve this situation by providing the same advantages of the approach:
- anyone, really anyone, understands the concept: you get a Word document, follow the steps, write down the results. That's it. No training required.
- zero impact on the test system, as long as Word is installed on another machine to fill out the results.
The software must support writing tests
- provide a basic concept of defining steps and expected results in textual form. No images required.
- provide "comments" for information that is not a test step, e.g. to link to a bug tracking tool (or a requirement for the seldom case such a thing exists)
- if the software has way more features than needed, I'd like to be able to customize the UI to reduce it to the minimum of defining steps and expected results.
and executing tests
- show the steps to the user
- go forward / backward through the steps if they are not displayed all at once
- provide a human readable report (e.g. PDF)
- provide a machine readable report. Should be a format which is widely supported by other tools/libraries, e.g. XML.
- run at least on Windows 7
- (like before) if the software has way more features than needed, I'd like to be able to customize the UI to reduce it to the minimum of showing steps and defining results.
- cost less than 300 € per user (not a per machine license)
- provide good keyboard input, perhaps customizable
- is not a web application (due to delays, connectivity issues, keyboard support, ...)
- no or little impact on the test system if the software needs to run on the test system itself (e.g. must not need Java or .NET since installing that may be part of the test itself)
- simple administration: no servers, no fat database etc. It should be really simple.
The software needn't
- manage requirements. Just focus on testing. In 8/10 projects I worked on, the requirements weren't specified anyway.
- manage users and access rights. I just want to give someone the test to execute.
- do complex reporting / trending etc. That will be done based on the machine readable format in a 3rd party (proprietary) application.
- test scheduling, test agents
- integrate with anything. Just run standalone.
I have already tried: