In order to complete a personal project I need to know what the "tools" (libraries) will be in order to do so.

I need a library that would be capable of detecting images in a 3D game (so that I can make a GUI giving me various statistics) - some of said images have some levels of transparency so the library needs to be able to handle "tolerance" (as in "mmmhyeaah, I'm pretty sure that logo's there").

So far I stumbled upon openCV but I have no idea if it applies to "live" windows or just images, I've seen people experimenting with their webcams, but then the software they have created just processes the webcam feed image by image to create an output, yet what I want is not to create an output but to analyze one.

The only workaround I can find (if it's one and not simply the way it is supposed to be done) would be figuring out how to have the code to take a screenshot (I don't think tutorials to do this would be that hard to find, hours to understand 'em is another thing), analyze it + update values, and then proceed to take a new one every 100ms. Is it how I should do thing? Would sound pretty CPU-intensive.

  • I'm currently developing a screenshot app, since many tools don't suport multiple screens or have issue with negative coordinates. While the CPU usage is pretty low (5% for 3 screens), I only get 1.6 frames per second. I'm not sure if it would capture DirectX 3D graphics. Anything that manipulates the graphics card memory directly (might also apply to videos) may result in just a black rectangle. So maybe want to ask another question on how to grab screenshots in "realtime" including 3D and video support. – Thomas Weller Oct 13 '16 at 7:49

detecting images in a 3D game (so that I can make a GUI giving me various statistics)

Sikuli automates anything you see on the screen. It uses image recognition to identify and control GUI components. It is useful when there is no easy access to a GUI's internal or source code - and you can use it to find elements inside the game. If the game is web-browser based, then you can use Kantu, which is the same as Sikuli, but inside a web browser.

From my tests... the screenshot-check rate in boths apps is around 100ms-1s, depending on the image size and PC. Both apps use OpenCV under the hood.

You need to grab some, or all of the screen, then pass it to OpenCV for processing - isolating a specific icon against a background can be done relatively easily, just remember to search your image for an area that matches a smaller sub-region of the icon due to anti-aliasing. Spotting anything that looks like an icon would require machine learning which is a very big subject.

If you are determined to work in C++ you can look at the wxWidgets GUI library which includes both GUI components for your capture and the ability to take shots of some or all of the screen for processing. Personally I would look at doing things in python as in this question.

  • 1
    Thanks :), i'll head to python then - Tried to get openCV to run properly for C++ for 7h straight and it was hell on earth for my lil' newbie brain. Not planning on giving up anytime soon but if python is "easier" for newbies to learn and is able to achieve what I want it to then fine :) Thanks a lot for the help ^^ – FatBee Oct 13 '16 at 9:18

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.