Live-session shell daemon

Question

TLDR

I need software that runs a daemon server and hosts an interactive shell program as a subprocess. The software should also include a client program that connects to the daemon server and provides PTY access to the hosted subprocess. I also need to be able to send lines to the hosted subprocess's STDIN and read it's STDOUT from scripts(JSON/XML RPC is preferable, but I can also work with standard HTTP(e.g. REST?) or a command line util that comes with the software).

Usecase

I work in Linux and use Vim as my development environment. The project uses Waf as a build system(which means the noise/signal ratio in the output is skyrocketing). The project needs a specific Linux configuration to build and run - which means either a virtual machine on my local computer or a remote compilation server.

This means I need SSH, and since I'm using Vim writing shell scripts to build and run projects is a second nature.

Attempt #1: SSH with command

This is the straightforward solution: run ssh with a compound command that cds to the project's folder and runs waf to configure/build/clean the project, or the whatever is required to run it.

The problem

Waf creates a ridiculously large amount of noise, making it hard to read the error messages when build fails. Due to Vim's poor terminal emulation, I can't copy the result to a Vim buffer(to parse it my own way), and I can't continue using Vim until I close the shell command's result.

The output is also too long to display all at once on the screen, so I need to continue scrolling tons of useless Waf output every time I build.

Reading the result to a buffer directly(r !ssh...) or using a third party shell plugin will be too cumbersome for regular use, since I don't need all that info when the build succeeds.

Attempt #2: Abusing tmux

This is what I'm currently using. I run an SSH session in a tmux window, and use tmux send-keys to build and run from a script. I'm using a dual monitor setup where the laptop's small monitor shows Konsole with the tmux session that runs ssh, and the larger external screen shows Vim with the code. It's hard to read the exact output, but I can easily see that the build is running and notice when it fails, and I can copy the output to a Vim buffer where I can parse it.

The problem

The main problem is that I have to copy the output to a Vim buffer manually. Reading the output of the last command is something I would like to automate, and use in scripts that can parse it.

Another problem is that I can't connect to the tmux session from inside Vim. Neither the built-in terminal emulator nor any of the terminal emulator plugins available can run a TUI application like tmux.

Attempt #3: Abusing Supervisor

Supervisor allows me to run and control other programs as daemons. It runs these programs as it's own subprocesses, so it has full access to their STDIN, STDOUT and STDERR. It has the fg command for opening an interactive session into these programs, and has an XML RPC service that allows me to send stuff to these programs' STDIN and read their STDOUT and STDERR.

Supervisor could have been perfect for me, but there is one problem that makes it unusable for my case:

The problem

Supervisor was never meant for running interactive shell programs - it does not support PTY at all! When I try to run Bash I can run commands but I don't get the prompt(which means Bash can't tell that STDIN is pending - an PTY feature AFAIK). ssh does not work at all - not when I invoke it directly and not when I invoke it from Bash - I simply get an error Pseudo-terminal will not be allocated because stdin is not a terminal. - and then my commands do nothing and I have to stop the process.

So....

Does anyone know a program that can do what I want? Something like Supervisor with PTY support - or a way to make Supervisor itself support PTY - would be great.

Answer 1

Maybe GNU Screen (similar to tmux) could be of use to you.

It allows you to write a running log of a window (by either pressing CTRL-a followed byH in an attached session, or by invoking screen with -L). You could then use that screenlog.n to get the waf output into a VIM buffer in an (semi-)automated fashion.

Alternatively you can dump the output of a running, but detached session to STDOUT using screen -X hardcopy /some/file. I'm not sure if the scrollback buffer will be included in that dump though.

If you want to review the output interactively, you can also scroll inside a screen session: In an attached session, access the scrollback buffer by entering copy mode with CTRL-A, followed by [. You can then scroll up/down using CTRL-u and CTRL-d. To exit copy mode again, press ESC.