1

I wonder if anyone has used or at least if there is any Jenkins plugin that can enable the management of a complete test cycle for a particular build.

Maybe a first question would be, is there any Jenkins plugin that can offer a high level interface to testing embedded software through a node connected to some hardware equipment?

In this case, the test itself would be performed by some specific tool on the node. Jenkins would act only as a manager, starting the testing on the node, collecting results, updating statistics about the build quality. Everything in an iterative manner, allowing the test team involved to debug any issues and to update if needed the test items (test cases if you like, but also environment specifics if required). The test execution cycles would be performed during the night leaving the equipment available during the day to be used for one-off executions and debug.

Such plugin would have to be able to checkpoint the test project if required before any regression run and to allow manual input as well (e.g. to force stopping the testing)

The whole idea involves some other aspects as well, but for now I'm only interested about the plugin itself.

|improve this question
1

Basically if you can set up your regression testing so that it is able to be triggered from the command line and the results captured & possibly post processed then a Jenkins node can trigger it and, given a return value from the process of 0 for a pass and non-zero for a fail, you will get a success or failure, (just as you do with make). All the command line output is captured and available for debugging, etc.

In the past I have set this sort of thing up with python being used to invoke the test tools, parse the relevant logs and set the return value.

|improve this answer
  • 1
    This is already done at the moment. I've implemented a tool to do exactly that and it is used for one-off executions. This tool would be used during the day when everybody is in the office. It makes very well use of the queuing mechanism that Jenkins provides and if there is more than one node servicing a particular job, it's a bliss. The Jenkins plugin (or whatever is there) I'm looking for would have to be a bit smarter than that. Would need to start testing, wait for testing to finish in the allowed time slot, wait for the next available time slot, and so on. – uulescu Dec 13 '17 at 16:35
  • 1
    Most probably I will need to implement it myself based on this pipelining mechanism, I would suspect. Any test statistics would have to be processed on the node and given to Jenkins as junit artifacts. I'm only speculating at the moment. Any thoughts on this? Or suggestions? Thank you very much! – uulescu Dec 13 '17 at 16:40
  • 1
    From the command line is the only way that I have seen it done, at several organizations. – Mawg Dec 14 '17 at 7:57
  • 1
    @Mawg too many developers these days, including tool developers unfortunately, have either forgotten about the command line or buried the command line interface documentation too well which makes automatic testing a pain. With Jenkins if you can do it on the command line then Jenkins can do it for you and record the results. – Steve Barnes Dec 15 '17 at 6:26
  • 1
    I certainly agree, but while skills like bash/dos/powershell scripting have declined, the popularity of Python has risen, so most of your Jenkins "command line" stuff generally just invokes some Python scripts – Mawg Dec 15 '17 at 7:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.