Video player with external control?

Question

i'm looking for a video player, which is able to smoothly shuttle back and forth at high speeds and can be controlled externally - eg. by a poti connected to an arduino with which one can shuttle around the video. this may sound trivial at first but turns out, it's not.

first: i'm perfectly fine to encode to intraframe codecs which are fast to decode eg. prores, cineform, hap or even photo-jpg if i have to. as this will run of fast ssds, data rate is not an issue. playback platform is windows.

now, there are not that many programs which provide smooth shuttling in the first place (even with said codecs). the ones that come to my mind are editing systems like premiere pro, vj tools like resolume and a few players like mpv and quicktime x (vlc is really bad at shuttling).

however, all these solutions are either costly (editing and vj tools) or do not offer the openess to interface with something at low latency (eg. serial or osc), like an arduino (afaik).

so, any hints? thanks a lot for any help!!!

ps: one more thing: i need to be able to shuttle to absolute framenumbers/timecodes. so mpv keybindings are not an option...

Answer 1

it took a while, but i solved this. i tried a bunch of different solutions (html5, processing, max/msp) - but all performed terribly slow and choppy. fluent shutteling is a lot about caching and keyframes, you really do want to have a dedicated mediaplayer to start with.

the solution i ended up with uses mpv player which can create an ipc socket, which recieves commands in json format. these i send from a node.js script, which reads the arduino via serialport.

the following won't work out of the box, you will need to adjust ports and stuff (also on windows the socket works with pipes, see mpv documentation) but it hopefully can guide you:

requirements: https://mpv.io/ , https://nodejs.org/en/ , https://serialport.io/en/

on macos you start the mpv from terminal with:

#!/bin/sh 
/Applications/mpv.app/Contents/MacOS/mpv mymovie.mp4 --input-ipc-server=/tmp/mpvsocket --keep-open=yes --keep-open-pause=no

then, from a second terminal, you start your node.js script:

node mynodescript.js

the node.js script itself looks like this:

// stuff for reading arduino over serialport
const SerialPort = require('serialport');
const Readline = SerialPort.parsers.Readline;
const port = new SerialPort("/dev/cu.wchusbserial1420",{
    baudRate: 9600,
    parser: new Readline("\n")
});
const parser = port.pipe(new Readline({ delimiter: '\r\n' }));

// stuff for connecting to mpv player over ipc socket
var net = require('net');
var socket = net.createConnection("/tmp/mpvsocket");

socket.on("connect", function() {   // do something when you connect
  console.info("mpv socket connection established.")
});

// fusing the two together
parser.on('data', function (data) {
  socket.write('{ "command": ["seek", "' + data + '", "absolute-percent+keyframes"] }\r\n');
});

and last but not least the arduino code (in this case a simple poti, delivering values between 0 and 1023, which are converted to percent):

int potPin = 2;    // select the input pin for the potentiometer
int val = 0;       // variable to store the value coming from the sensor
int oldval = 0;
float mappedval = 0;

// since normal mapping can't deal with ints
float mapfloat(float x, float in_min, float in_max, float out_min, float out_max) {
  return (x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min;
}

void setup() {
  Serial.begin(9600);
}

void loop() {
  val = analogRead(potPin);    // read the value from the sensor

  if (val < (oldval * 0.98) || val > (oldval * 1.02)) {
    oldval = val;  // save the changed value
    mappedval = mapfloat(val, 0.0, 1023.0, 0.0, 100.0); // maps the poti values to percent 
    Serial.println(mappedval);
  }

  delay(20);   
}