I am looking for a quick way to do AR (augmented reality). I think the correct term is camera matching.

Given a photograph which contains several squares of a known size, and several 3D models, I would like to compute the position and orientation in 3D of the camera and the objects so that the models render with a correct perspective with respect to the perspective of the image. If this can't be done inside a single software, two different softwares are acceptable.

So i'm looking for a way to detect the features in the original image (can be done by hand if needed), to calibrate the camera and compute the projection matrix, and to place the objects in 3D, and then render the scene.

Here is a sample photo showing some actual data, and the kind of result I'm looking for: enter image description here


If you are happy with doing the alignment yourself you should take a look at Blender you can:

  • Generate your 3D model
  • Align your "camera"
  • Overlay onto the image, (or vice-versa)
  • Rendering
  • Lots more.
  • it is free in both senses of the word.

You could probably use OpenCV to compute your alignment with your model as well - also free - a similar exercise, drawing a cube about a moving head, is detailed here.


I would recommend OpenCV pose estimation. OpenCV is an opensource software that has had roughly 10,000 contributors. I am using various OpenCV functions, including pose estimation, for surgical device prototypes that I am producing at a world's leading research institution.

OpenCV is available free from https://www.opencv.org . There are many installation techniques. I use CMAKE. An example of using CMake to install opencv can be found at https://www.learnopencv.com . There are detailed instructions for many operating systems on learnopencv.com . Just search "install [your OS])" on learnopencv.com . Follow the instructions exactly, or it may not work. If you are ok with python and you are using linux just do: 'pip install opencv-contrib-python' in the terminal.Actually, that works on Ubuntu. Google the correct installation procedure for you OS. I recommend using a virtual environment. But it isn't required.

Pose estimation requires you to first perform a camera calibration. But follow these three tutorials, and you should have no problem (one of them skips camera calibration and can be used for prototyping.): Camera Calibration Pose Estimation (Requires camera calibration) Pose Estimation (without camera calibration)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.