I'm looking for screen recording software that writes a file with the keyboard and mouse events that is easy to parse, alongside the video. The time from the key/mouse events should be synced with the video, so given a mouse click, I can find the corresponding frame in the video. So for example, the ideal software would write files video.mp4, events.json, cursor_position.json. I'm interested in all OSes.

Screenflick (OSX) seems to almost do what I want, but it saves the key/mouse data in an undocumented binary format.

In case it helps, my use case is to train a model (with machine learning) that predicts click locations during computer usage.

  • 1
    It probably can't be done with screen recording. You want to log not display events, but input events (from keyboard/mouse). What you need, is called keylogger. – peterh Oct 9 '20 at 9:13
  • 1
    Do you just want keyboard and mouse events, or also what is shown on the screen? Can you update your question to add clarity? Thanks! – RockPaperLz- Mask it or Casket Oct 9 '20 at 13:54
  • I also want to log what's on the screen, with time synced to the events. Will edit. – John Oct 9 '20 at 17:18
  • I don't know software that everything, but software does exist for the individual components. A keylogger (of which there are many) can record mouse & keyboard events to JSON whilst a conventional screen recorder such as ShareX can record what's shown on the screen. You can start them at the same time to achieve what you want, or write a short script to do that. – Huey Oct 11 '20 at 13:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.