I play a webpage based game in which a picture is shown alongwith some jumbled alphabets below the image. The player is supposed to click on the alphabets in an orderly manner to form the name of the thing shown in the image. Instead of using the mouse to click the alphabets I want to use the keyboard. e.g If I press S key on keyboard the mouse should click on the S on the screen. I have tried macro recorder softwares with 'if image exists' and 'if then else' commands and also tried to make a python script with openCV. But couldn't find a way through any of those.

  • 1
    Link to the game, and mention your OS, since programmatic control of the mouse and keyboard can depend a lot on OS. – Kodiologist Dec 20 '18 at 15:11
  • I have windows 10 and kali Linux. The link wouldn't be helpful to you as it requires a sign up process for which you should have an Indian mobile number. I will try to post a screenshot instead – baba Dec 20 '18 at 15:19
  • I admittedly don't have an Indian phone number, but another user might and be interested in digging into your question. Failing direct access to the game, it would be helpful to know whether it's implemented with HTML canvas with JavaScript, SVG with JavaScript, Flash, Java, or something else. – Kodiologist Dec 20 '18 at 15:21
  • Ok I will post the link. I forgot to mention in the question that there are 10 questions with 10 seconds to answer each. So it would be hard to find out all these things. – baba Dec 20 '18 at 15:26

The simplest way to do this would be using AutoHotKey. A script can be written using the ImageSearch command and setting the alphabets on keyboard as hotkeys to activate different commands.

| improve this answer | |

Well, the standard tool for web automation is Selenium.

And articles like this can help you with image recognition, so that's the way that I would go.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.