diff --git a/README.md b/README.md index e25c801..8282a5d 100644 --- a/README.md +++ b/README.md @@ -1,36 +1,153 @@ -Setting up the extra display using sway's hidden [`create_output`](https://github.com/swaywm/sway/blob/master/sway/commands/create_output.c) command, and starting wayvnc +# CG Virtual Monitor -``` -% swaymsg create_output -% swaymsg output "HEADLESS-1" resolution 396x224 -% wayvnc -o HEADLESS-1 +This demo showcases how an fx-CG calculator can be used as an external monitor +by forwarding VNC framebuffers through USB. It was made as a test of the new +USB driver features introduced in gint 2.10. The program also supports multiple +parallel monitors and can transmit key presses on calculators back to the PC. -# to remove the output later -% swaymsg output "HEADLESS-1" unplug -``` +![](/images/triple-monitor-setup.jpg) -Compiling and starting the VNC client +## How it works -``` +In this demo, the desktop manager ([sway](https://swaywm.org/)) has two virtual +outputs not connected to any monitor. The calculators don't act as a physical +monitors since there is no standard device for that in USB 2 (it's too slow). +Instead, sway's outputs are exposed by a VNC server running locally +([wayvnc](https://github.com/any1/wayvnc)). + +The VNC client in this repository's `vnc-client` folder is connected to the +VNC server, while at the same time using the communications library _libfxlink_ +from the [fxSDK](https://gitea.planet-casio.com/Lephenixnoir/fxsdk) to discover +calculators and exchange messages. It sends framebuffer updates and receives +keyboard updates (which are, unsurprisingly, forwarded to the VNC server). + +The calculators themselves run [gint](https://gitea.planet-casio.com/Lephenixnoir/gint) +2.10 using newly-added support for host → calculator USB communication. They +"simply" show framebuffers they receive and send back keyboard updates so that +the calculator keyboards can control the PC. Obviously the USB driver is not +that simple... but this is invisible from the application code. + +The entire setup runs at about 30 FPS, which each frame taking about: + +- 20 ms to receive framebuffers via USB into the VRAM, including + * 14 ms of raw reading from the USB buffer; + * 6 ms of waiting for data transfer from the host. +- 11 ms to send the VRAM update to the display. + +There are many ways to optimize this further if the need ever arises... I'd be +surprised if it couldn't run at 60 FPS with double-buffer mode, frame streaming +and/or parallelizing both bottlenecks by reading directly to the display. + +## Download + +TODO: Direct link to the .g3a + +You'll need to build the VNC client, though. + +## Building + +Here are the dependencies: + +- The [fxSDK](https://gitea.planet-casio.com/Lephenixnoir/fxsdk). The fxSDK + repository alone is needed for the VNC client, while the entire tool suite + including gint and the cross-compiler is needed to build the add-in. +- [libvncserver](https://github.com/LibVNC/libvncserver) for the RFB protocol. +- SDL2 for debug display on the PC. + +Build the VNC client with: + +```bash % cd vnc-client % cmake -B build # options... % make -C build -% ./cgvm_vnc --calc ``` -If you installed native fxSDK tools outside of the default `$HOME/.local`, add `-DFXSDK_PATH=` to CMake options. (GiteaPC uses the default path.) +If you installed the fxSDK tools outside of the default `$HOME/.local`, add +`-DFXSDK_PATH=` to CMake options. (GiteaPC uses the default path.) -Compiling and installing the virtual monitor add-in +Build the virtual monitor add-in with: -``` +```bash % cd cgvm-addin +% fxsdk build-cg +# Send .g3a file to calculator using fxlink and the LINK app: % fxsdk build-cg -s ``` ---- +## Using virtual monitors -This entire thing was coded in a few hours with limited regard for exceptions. Ideally, you should -1. Open the Virtual Monitor add-in on the calculator -2. Start `cgvm_vnc` -3. Not disconnect the calculator while `cgvm_vnc` is running -4. After closing `cgvm_vnc`, disconnect the calculator +**Step 1: Creating the virtual output** + +This step depends on the desktop manager. I'm using sway, which has a hidden +[`create_output`](https://github.com/swaywm/sway/blob/master/sway/commands/create_output.c) +command to create and configure headless outputs: + +```bash +# Creating outputs under sway specifically +% swaymsg create_output +% swaymsg output "HEADLESS-1" resolution 396x224 # position 1920 0 + +# For a second monitor: +% swaymsg create_output +% swaymsg output "HEADLESS-2" resolution 396x224 # position 1920 224 +``` + +Other Wayland desktops might have their own way of doing this, though there +doesn't seem to be a unified method. + +On X, [Xvfb](https://www.x.org/releases/X11R7.6/doc/man/man1/Xvfb.1.xhtml) +seems to allow this (but I'm not sure whether it can run alongside the normal +display). + +**Step 2: Exposing the virtual output on a VNC server** + +The VNC client will connect to VNC servers at addresses `127.0.0.1:5900` and +`127.0.0.1:5910`, whichever is up. This is because wayvnc only exposes a single +output, so I run two instances to get both monitors. + +```bash +% wayvnc -o "HEADLESS-1" 127.0.0.1 5900 + +# For a second monitor: +% wayvnc -o "HEADLESS-2" 127.0.0.1 5910 +``` + +As the name suggests, wayvnc is for Wayland, but here I'm not worried about +X-based alternatives. + +**Step 3: Run the VNC client and calculator add-in** + +Transfer the add-in to the calculator and run it from the main menu. It will +show "USB connected!" as soon as the PC establishes a connection. Then, start +the VNC client with: + +```bash +% ./cgvm_vnc --calc # --sdl +``` + +You can add `--sdl` to have a copy of the outputs in SDL windows for debugging. +The VNC client will automatically find calculators exposing the proper USB +interface and send them the frames. + +**Step 4: Stop servers and unplug calculators** + +To take the setup down, interrupt the VNC client and close the add-in on the +calculator with EXIT or MENU (in any order). Then stop the VNC servers, and +finally remove the outputs. + +```bash +# Removing the outputs under sway specifically +% swaymsg output "HEADLESS-1" unplug + +# For a second monitor: +% swaymsg output "HEADLESS-2" unplug +``` + +## Known issues + +- The first frame is often incorrect. +- Frames from the VNC server are dropped if the calculator is busy processing a + previous frame. +- The VNC client seems to lost connection with the calculator sometimes (libusb + returning "No such device"). Not sure why yet, as it's hard to reproduce. In + that case restart the VNC client. diff --git a/images/dual-monitor-setup.jpg b/images/triples-monitor-setup.jpg similarity index 100% rename from images/dual-monitor-setup.jpg rename to images/triples-monitor-setup.jpg