I am aware of keras, 'block' and a few others Python libraries for neural networks which do Reinforcement Learning among others. But is there a library than can make the task of visualisations easy? Particularly in 3D modelling of agents/environment,Seeing the simulations etc... I can see a few reinforcement learning videos online that show the simulated agent/environment but either they have made visual models from the ground up or used some other language/technology...(or they are very old). Open AI gym was one such platform where you can place and train your agents but I believe it's more for creating the learning algorithm than for visual simulation and environment.
You could also take a look at PyBrain, which implements some of the standard benchmark tasks and it allows for the experiments 3D visualization.
Here you can find some examples: https://www.youtube.com/watch?v=fEM7YDNonSE
See this link for testimonials about Psychopy, including one about the ease with which links to visualisation software can be made.
You could look at RL Toolkit, which gives examples of visualizations for mountain car and grid worlds.