I play a webpage based game in which a picture is shown alongwith some jumbled alphabets below the image. The player is supposed to click on the alphabets in an orderly manner to form the name of the thing shown in the image. Instead of using the mouse to click the alphabets I want to use the keyboard. e.g If I press S key on keyboard the mouse should click on the S on the screen. I have tried macro recorder softwares with 'if image exists' and 'if then else' commands and also tried to make a python script with openCV. But couldn't find a way through any of those.

  • 1
    Link to the game, and mention your OS, since programmatic control of the mouse and keyboard can depend a lot on OS. Dec 20, 2018 at 15:11
  • I have windows 10 and kali Linux. The link wouldn't be helpful to you as it requires a sign up process for which you should have an Indian mobile number. I will try to post a screenshot instead
    – baba
    Dec 20, 2018 at 15:19
  • I admittedly don't have an Indian phone number, but another user might and be interested in digging into your question. Failing direct access to the game, it would be helpful to know whether it's implemented with HTML canvas with JavaScript, SVG with JavaScript, Flash, Java, or something else. Dec 20, 2018 at 15:21
  • Ok I will post the link. I forgot to mention in the question that there are 10 questions with 10 seconds to answer each. So it would be hard to find out all these things.
    – baba
    Dec 20, 2018 at 15:26

2 Answers 2


The simplest way to do this would be using AutoHotKey. A script can be written using the ImageSearch command and setting the alphabets on keyboard as hotkeys to activate different commands.


Well, the standard tool for web automation is Selenium.

And articles like this can help you with image recognition, so that's the way that I would go.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.