1

I am looking for an open source tool that can run a (scientific) simulation model many times. It should vary the input parameters in a study and evaluate the results. It should at least support:

  • Parameter sweep: specify several parameter ranges e,g. x=1,2,3 y=10,20,30,40 and execute every possible combination z=f(1,10), z=f(1,20), ... z=f(3,40)) The result could be visualized for example as contour plot or a set of curves.

  • Sensitivity analysis: use a "working point" as a reference, e.g. {x=4, y=6} and vary several input parameters to identify which parameter is the most important one. The result could be visualized for example as tornado chart.

It would be nice if the software allows for parallel model executions and for batch processing of the results (e.g. create the above mentioned plots).

I could imagine a master-slave pattern where the model ("slave") is given as executable model.exe, accepting an input file "input.txt" and writes output to another text file "output.txt". The batch software (master) would generate the input files, execute model.exe several times and collect the output from output.txt.

A more advanced tool would also allow to use databases instead of plain text files and might support more kinds of studies, e.g. scenario/probe analysis (specify a set of distinct input tubles instead of parameter ranges) and monte carlo simulations (use probability distributions for the input parameters).

1

This is exactly the sort of thing that Python, Jupyter + the SciPy Stack are regularly used for.

  • Free, Gratis & Open Source
  • Cross Platform: OS-X, Windows, Linux, Raspberry Pi though to CERNs super computer clusters.
  • Batch processing & Parallel Processing. You can even run your tests across multiple machines and collate the results later.
  • You can rapidly implement your algorithms in python with a full numeric & scientific stack available and then explore the behaviour.
  • If software under test is command line based it can be invoked with different input parameters and the output to standard out be captured and parsed or if it takes an input file and outputs to another you can create and parse most file formats from python.
  • Algorithms can also be implemented in C, C++ and can be called directly from your python using the ctypes library - you can even call FORTRAN functions from python, see here for details. (Always assuming that you can the appropriate compilers availalbe)
  • With Jupyter you can produce combined software, documentation in markdown with MathJax formulae & charts, etc., in your browser.
  • Charts can be displayed &/or output in a number of formats including publication ready.
  • Python can connect to most, if not all, databases for your advanced use.
  • 1
    Thanks for your suggestion to use Python as a programming language. The mentioned libraries seem to be a good basis to develop the software I am looking for. However, there does not seem to be kind of a "finished solution". In fact I already started to develop a tool for batch processing in Java/Eclipse. But it feels a lot like reinventing the wheel. Therefore I am still looking for alternatives to Treez: github.com/stefaneidelloth/treez. I am going to add some answers to share what exists so far. – Stefan Nov 21 '16 at 8:01
0

SimLab provides a free development framework for Sensitivity and Uncertainty Analysis.

https://ec.europa.eu/jrc/en/samo/simlab

0

The PSUADE Uncertainty Quantification Project

https://computation.llnl.gov/casc/uncertainty_quantification/

0

There are some Matlab tool boxes:

http://www.bris.ac.uk/cabot/resources/safe-toolbox/

http://www.gui-hdmr.de/

0

Treez: Eclipse plugins for tree based graphical user interfaces and parametric simulations (no stable version yet):

https://github.com/stefaneidelloth/treez

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.