||2 months ago|
|cgvm-addin||2 months ago|
|images||2 months ago|
|vnc-client||2 months ago|
|.gitignore||2 months ago|
|README.md||2 months ago|
CG Virtual Monitor
This demo showcases how an fx-CG calculator can be used as an external monitor by forwarding VNC framebuffers through USB. It was made as a test of the new USB driver features introduced in gint 2.10. The program also supports multiple parallel monitors and can transmit key presses on calculators back to the PC.
How it works
In this demo, the desktop manager (sway) has two virtual outputs not connected to any monitor. The calculators don't act as a physical monitors since there is no standard device for that in USB 2 (it's too slow). Instead, sway's outputs are exposed by a VNC server running locally (wayvnc).
The VNC client in this repository's
vnc-client folder is connected to the
VNC server, while at the same time using the communications library libfxlink
from the fxSDK to discover
calculators and exchange messages. It sends framebuffer updates and receives
keyboard updates (which are, unsurprisingly, forwarded to the VNC server).
The calculators themselves run the addin in the
cgvm-addin folder, which is
based on gint 2.10 with
newly-added support for host → calculator USB communication. They "simply" show
framebuffers they receive and send back keyboard updates so that the calculator
keyboards can control the PC. Obviously the USB driver is not that simple...
but this is invisible from the application code.
The entire setup runs at about 30 FPS, which each frame taking about:
- 20 ms to receive framebuffers via USB into the VRAM, including
- 14 ms of raw reading from the USB buffer;
- 6 ms of waiting for data transfer from the host.
- 11 ms to send the VRAM update to the display.
There are many ways to optimize this further if the need ever arises... I'd be surprised if it couldn't run at 60 FPS with double-buffer mode, frame streaming and/or parallelizing both bottlenecks by reading directly to the display.
You can download the calculator add-in here: CGVirtMo.g3a
You'll need to build the VNC client, though.
Here are the dependencies:
- The fxSDK. The fxSDK repository alone is needed for the VNC client, while the entire tool suite including gint and the cross-compiler is needed to build the add-in.
- libvncserver for the RFB protocol.
- SDL2 for debug display on the PC.
Build the VNC client with:
% cd vnc-client % cmake -B build # options... % make -C build
If you installed the fxSDK tools outside of the default
-DFXSDK_PATH=<the path> to CMake options. (GiteaPC uses the default path.)
Build the virtual monitor add-in with:
% cd cgvm-addin % fxsdk build-cg # Send .g3a file to calculator using fxlink and the LINK app: % fxsdk build-cg -s
Using virtual monitors
Step 1: Creating the virtual output
This step depends on the desktop manager. I'm using sway, which has a hidden
command to create and configure headless outputs:
# Creating outputs under sway specifically % swaymsg create_output % swaymsg output "HEADLESS-1" resolution 396x224 # position 1920 0 # For a second monitor: % swaymsg create_output % swaymsg output "HEADLESS-2" resolution 396x224 # position 1920 224
Other Wayland desktops might have their own way of doing this, though there doesn't seem to be a unified method.
On X, Xvfb seems to allow this (but I'm not sure whether it can run alongside the normal display). Or you can use a section of your primary display (see below).
Step 2: Exposing the virtual output on a VNC server
The VNC client will connect to VNC servers at addresses
127.0.0.1:5910, whichever is up. This is because wayvnc only exposes a single
output, so I run two instances to get both monitors.
% wayvnc -o "HEADLESS-1" 127.0.0.1 5900 # For a second monitor: % wayvnc -o "HEADLESS-2" -S /tmp/second-wayvnc.socket 127.0.0.1 5910
As the name suggests, wayvnc is for Wayland. There are alternatives on X, such as TigerVNC. The following setup (courtesy of Tituya) grabs the top-left corner of the primary display:
% vncpasswd % x0vncserver -rfbauth ~/.vnc/passwd -rfbport 5910 -Geometry 394x224+0+0
Step 3: Run the VNC client and calculator add-in
Transfer the add-in to the calculator and run it from the main menu. It will show "USB connected!" as soon as the PC establishes a connection. Then, start the VNC client with:
% ./cgvm_vnc --calc # --sdl
You can add
--sdl to have a copy of the outputs in SDL windows for debugging.
The VNC client will automatically find calculators exposing the proper USB
interface and send them the frames.
Step 4: Stop servers and unplug calculators
To take the setup down, interrupt the VNC client and close the add-in on the calculator with EXIT or MENU (in any order). Then stop the VNC servers, and finally remove the outputs.
# Removing the outputs under sway specifically % swaymsg output "HEADLESS-1" unplug # For a second monitor: % swaymsg output "HEADLESS-2" unplug
- The VNC client seems to lose connection with the calculator sometimes (libusb returning "No such device"). Not sure why yet, as it's hard to reproduce. In that case restart the VNC client.