Is there an open source application that automates GUI operations? I'd like to automate GUI applications that are not web-based. I want the application to possibly recognize screenshots and locations. But I'd settle for an application that issues tab and enter key in succession. I want an application to automate operations of diverse Windows applications including RDP sessions to other servers.


SikuliX has a motto of "Automate Anything You See" - it is a Java based testing framework that uses Image Processing to search the screen for specific elements and perform actions on them driven by python/ruby/JavaScript scripting.

  • Free, Gratis & Open Source
  • Linux, OS-X & Windows
  • Comes with its own IDE
  • Works even with problems like windows, buttons, etc, not being in a consistent position - as long as they look the same they can be identified.
  • Does not need to know resource IDs, etc.
  • Can issue:
    • Click
    • Double Click
    • Delays
    • Type keyboard input, etc.
  • conditional actions e.g. if this icon is found click it, etc.

There are dozens, here are a few:

  • AutoIt & AutoHotKey are the grandparents of the type
  • If Python is your poison, how about pywinauto
  • There are many more, starting with those, you will surely find the others
  • This list may give you more clues

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.