Setting up USB multiseat with DisplayLink on Linux (GDM up to 2.20)
Posted on 16. Nov, 2009 by Bernie Thompson in UT-1
Soon, we’ll be able to plug inexpensive zero-state USB docks/terminals into new Linux systems, and a new graphical login will pop up in a completely plug and play fashion. Many users can then simultaneously share any single Linux PC. This is great for education, libraries, internet cafes, etc — anywhere where you have clusters of client machines with light 3D/video app demands, and want the simplicity and savings of just one server machine with many terminals connected.
[Update August 8, 2010 - Further work on this has moved to the Google Summer of Code 2010 project for USB Multiseat]
Until this is in the distros, we have to assemble a few pieces ourselves to get everything installed for this scenario. Some major components (GDM/ConsoleKit) are changing the way this kind of thing will be done, so for now the instructions here are designed only for distributions with GDM 2.20 or earlier (Ubuntu < 9.10; Fedora < F12; Debian < 5.0; etc). The instructions have been tested most extensively on Ubuntu 9.04.
gdm –version
will show what version you have on your current system. Any tweaks to make things work with your specific distro are definitely welcome in the comments.
The first step is hardware. You need a USB hub with a DisplayLink USB device and free USB ports for keyboard and mouse. You can get this in one package in the form of devices like the Plugable DC-125, or buy independent USB devices like the UGA-125 combined with any USB hub. USB Audio and other devices can be made to work also, but these instructions just cover basic display, keyboard, and mouse.
With that, you're ready to start configuring your Linux setup ...
1. DisplayLink framebuffer driver
udlfb is in the staging tree of Linux kernels 2.6.32 and later. We'll install the latest version here. First, make sure you have git installed with "sudo apt-get install git-core", and create a directory which will host all the source code you download with git (e.g. ~/git/).
sudo apt-get install module-assistant sudo module-assistant prepare git clone http://git.plugable.com/webdav/udlfb/ cd udlfb make sudo make install sudo depmod -a
Now, when you plug in a DisplayLink device, you should see a "green screen" as the driver successfully loads and sets the graphics mode to match your monitor.
2. DisplayLink X server
This will get the X server installed, ready for use by later scripts.
sudo apt-get install pkg-config xorg-dev cd ~git git clone http://git.plugable.com/webdav/xf-video-udlfb/ cd xf-video-udlfb ./configure make sudo make install
We now need to create or modify a few scripts and configuration files. You'll need to use sudo to edit files in these system directories. You can cut/paste the text below, or download the files with "git clone http://git.plugable.com/webdav/misc-udlfb/" and copy each to the right location, and fix up ownership and permissions on the files.
3. udev script
Create a file called /lib/udev/rules.d/50-usbseat.rules owned by user root, with the following contents.
# set all DisplayLink devices to configuration 1 # see http://libdlo.freedesktop.org/wiki/DeviceQuirks for more info ATTR{idVendor}=="17e9", ATTR{bConfigurationValue}=="2", RUN="/bin/echo 1 > /sys%p/bConfigurationValue" # aliases for display, kbd, mouse attached to specific hubs KERNEL=="fb*",SUBSYSTEMS=="usb",PROGRAM="/bin/cat /sys/%p/../../../devnum",SYMLINK+="usbseat/%c/display",RUN+="usbseat.sh %c" KERNEL=="mouse*", SUBSYSTEMS=="usb", ATTRS{bInterfaceClass}=="03", ATTRS{bInterfaceProtocol}=="02", PROGRAM="/bin/cat /sys/%p/../../../../../devnum",SYMLINK+="usbseat/%c/mouse",RUN+="usbseat.sh %c" KERNEL=="event*", SUBSYSTEM=="input", ATTRS{bInterfaceClass}=="03", ATTRS{bInterfaceProtocol}=="01",PROGRAM="/bin/cat /sys/%p/../../../../../devnum",SYMLINK+="usbseat/%c/keyboard",RUN+="usbseat.sh %c" KERNEL=="control*", SUBSYSTEM=="sound", SUBSYSTEMS=="usb", PROGRAM="/bin/cat /sys/%p/../../../../../devnum", SYMLINK+="usbseat/%c/sound" # Handle when keyboard and mouse are one more hub downstream. Relying on pnp order to have already set up mouse, keyboard on upstream hub if we're daisy-chaining KERNEL=="event*", SUBSYSTEM=="input", ATTRS{bInterfaceClass}=="03", ATTRS{bInterfaceProtocol}=="01",PROGRAM="/bin/cat /sys/%p/../../../../../../devnum",SYMLINK+="usbseat/%c/keyboard",RUN+="usbseat.sh %c" KERNEL=="mouse*", SUBSYSTEMS=="usb", ATTRS{bInterfaceClass}=="03", ATTRS{bInterfaceProtocol}=="02", PROGRAM="/bin/cat /sys/%p/../../../../../../devnum",SYMLINK+="usbseat/%c/mouse",RUN+="usbseat.sh %c"
The udev subsystem will run this script automatically each time a USB device is attached.
4. usbseat.sh script
Create a file /lib/udev/usbseat.sh owned by user root with the following contents. Make sure to "sudo chmod a+x" to make the file executable.
#!/bin/bash # takes the "seat number" as parameter $1 # the seat number is the kernel device id of the hub the seat's devices are sitting off of # called once for every usb device that MIGHT be part of a seat, when they arrive or remove if [[ !(-n `/bin/pidof gdm`) ]]; then exit 0 fi seat_running=`/usr/bin/gdmdynamic -l | /bin/sed -n -e "/:$1,/p"` # $ACTION environment variable is set by udev subsystem case "$ACTION" in 'remove') if [[ -n "{$seat_running}" ]]; then /usr/bin/gdmdynamic -v -d $1 fi ;; *) # A device which might be part of a seat has been added # if we already have a running seat for this #, exit if [[ -n "${seat_running}" ]]; then exit 0 fi if [[ -e /dev/usbseat/$1/keyboard && -e /dev/usbseat/$1/mouse && -e /dev/usbseat/$1/display ]]; then # We have a newly complete seat. Start it. TMPFILE=`/bin/mktemp` || exit 1 /bin/sed "s/%ID_SEAT%/$1/g" < /lib/udev/usbseat-xf86.conf.sed > $TMPFILE /usr/bin/gdmdynamic -v -t 2 -s 1 -a "$1=/usr/X11R6/bin/X -br :$1 -audit 0 -nolisten tcp vt07 -config $TMPFILE" /usr/bin/gdmdynamic -v -r $1 fi ;; esac exit 0
5. X config file
Create file /lib/udev/usbseat-xf86.conf.sed with contents
Section "ServerFlags" Option "AutoEnableDevices" "false" Option "AutoAddDevices" "false" Option "DefaultLayout" "seat" Option "DontZoom" "true" Option "DontZap" "true" Option "AllowMouseOpenFail" "yes" EndSection Section "Module" Load "ddc" EndSection Section "Files" ModulePath "/usr/lib/xorg/modules" ModulePath "/usr/local/lib/xorg/modules" EndSection Section "Device" Identifier "dl" driver "displaylink" Option "fbdev" "/dev/usbseat/%ID_SEAT%/display" EndSection Section "InputDevice" Identifier "keyboard" Driver "evdev" Option "CoreKeyboard" Option "Device" "/dev/usbseat/%ID_SEAT%/keyboard" Option "XkbModel" "evdev" Option "XkbLayout" "us" EndSection Section "InputDevice" Identifier "mouse" Driver "mouse" Option "CorePointer" Option "Protocol" "ImPS/2" Option "Device" "/dev/usbseat/%ID_SEAT%/mouse" Option "Buttons" "5" Option "ZAxisMapping" "4 5" EndSection Section "Monitor" Identifier "monitor" EndSection Section "Screen" Identifier "screen" Device "dl" Monitor "monitor" EndSection Section "ServerLayout" Identifier "seat" Screen 0 "screen" 0 0 InputDevice "keyboard" "CoreKeyboard" InputDevice "mouse" "CorePointer" EndSection
6. fbcon workaround
[update: this step is no longer needed with the latest udlfb from git or in kernels 2.6.37+]
fbcon is a standard Linux kernel module, which aggressively assumes it can open any framebuffer device and take it over for use as a text terminal. Unfortunately, that's not what we want if we're going to be using that framebuffer to run its own X server. So we need to add a file /etc/modprobe.d/fbcon.conf to disable fbcon and leave our framebuffers free for other uses.
blacklist font blacklist tileblit blacklist bitblit blacklist fbcon
This file will not take effect until you run
sudo update-initramfs -u
Now, when you reboot and run "lsmod" you should not see fbcon in the loaded modules. And in /sys/class/graphics you should see fb0, instead of fbcon.
7. xrandr workaround
[update: this workaround is not needed when running xf86-video-fbdev or with recent displaylink X servers with this patch]
The DisplayLink X server currently has limited RANDR support, but later versions of GDM assume better. So for the time being, a workaround is required to get GDM applications (including gdmlogin) to display properly within the actual screen area -- otherwise they tend to think the screen has a strange rotation, and display themselves completely off it.
Add these lines into /etc/gdm/Init/Default, right after the definition of gdmwhich()
XRANDR=`gdmwhich xrandr` if [ "x$XRANDR" != "x" ]; then $XRANDR -o 0 fi
8. /etc/rc.local script
Add the following lines to your /etc/rc.local script to check for attached usb terminals that were attached (at boot), and udev found them before GDM was running.
oldIFS=$IFS IFS=/ for seat in /dev/usbseat/*; do set $seat /lib/udev/usbseat.sh $4 done IFS=$oldIFS
9. /etc/init.d/gdm patch
In recent versions of X, the system largely assumes that you'll only run one X server, and it will own all devices. So to support multiseat with multiple X servers easily, we need to have two configurations: normal (using only your primary graphics), and multiseat (where your primary graphics isn't used). We do this by detecting whether you have a USB terminal attached and configured, and if so coming up with a different gdm configuration.
Add these lines to your /etc/init.d/gdm script, just after the section to "Allow cdd to override the config" (around line 35).
# Allow usbseat to override the config if [ -f /etc/gdm/gdm-usbseat.conf ]; then for usbseat in /dev/usbseat/*; do seatid=${usbseat##*/} if [ -e "/dev/usbseat/$seatid/keyboard" -a -e "/dev/usbseat/$seatid/mouse" -a -e "/dev/usbseat/$seatid/display" ]; then CONFIG_FILE="--config=/etc/gdm/gdm-usbseat.conf" fi done fi
10. Create /etc/gdm/gdm-usbseat.conf
This is the alternative gdm.conf that will be used when a USB terminal is present at boot:
[daemon] DynamicXServers=true FlexibleXServers=0 Greeter=/usr/lib/gdm/gdmgreeter [security] [xdmcp] [gui] [greeter] [chooser] [debug] [servers] 0=inactive
Now, when you boot with USB terminal(s) attached, graphical logins will come up on all of those, while your primary display will remain a text console.
See my video from Linux Plumbers 2009 for more thoughts and background on USB multiseat.
Unsolved Problems
Beyond reworking for the new GDM/ConsoleKit versions, and getting all this just "built in" to the distributions, there are at least several major problems yet:
input duplication
[Update Feb 19, 2010 - fixed by removing "-sharevts -novtswitch" from the X start line and substituting a specific vt "vt07". Listing above now has problem resolved]
Input to any of the terminals will be duplicated to your primary console. This has a bunch of very nasty effects, including often duplicating your login to both your session and the console, and possibly causing strange effects like ctrl-c within any terminal will often cause that whole X session to close. This seems to be http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=525736, which at the bottom reports a workaround involving faketty, which unfortunately didn't work for me. You can also see a short discussion of this bug at https://fedorahosted.org/pipermail/multiseat/2009-April/date.html There ought to be a simple solution here -- any comments on that welcome.
mouse scroll wheel doesn't work
[Update Feb 19, 2010 - fixed by changing Protocol from "auto" to "ImPS/2" in the xorg.conf]
This is a problem with some other ways of doing multiseat: https://fedorahosted.org/pipermail/multiseat/2009-February/000004.html Again, probably a small tweak might resolve this.
unplug terminal doesn't tear down (or reconnect)
Currently, when you physically disconnect a terminal, everything remains running. By default, each user can only be logged in once, which means that you must do gdmdynamic -l to find the zombie instance and sudo gdmdynamic -d $seat to kill it. But the behavior users would really like to see is disconnecting the terminal leaves all apps running, but logging in from that terminal (or any other) reconnects and just transparently starts where the user left off.
Comments on other distros (e.g. yum vs. apt-get), better solutions, and other problems are very much appreciated.
udlfb 0.4.0
Posted on 11. Nov, 2009 by Bernie Thompson in USB-VGA-165
[Update Dec 29, 2011: udlfb was promoted from the staging to the mainline kernel tree in 2.6.38. And in kernel 3.3 pagefault detection and console are enabled by default. See all our udlfb posts for the latest news.]
[Update March 14, 2010: udlfb versions have moved to being released with the Linux kernel. Update on udlfb support in Linux kernel 2.6.34]
[Update Feb 6, 2010: additional features and fixes post-0.4 are available at the plugable git page. Background is in later posts. One major udlfb patch with these changes has made it into linux-next (slated for 2.6.34), and additional patches will be coming as they're ready.]
This is a new release of the DisplayLink kernel framebuffer driver, udlfb.
udlfb was accepted into the Linux kernel staging tree of 2.6 a few months ago. It needs some work to add key features and get it moving from the staging tree, into mainline. Roberto De Iorio, the author of udlfb and displaylink-mod, is focusing on X server work, and is happy with this work happening in parallel to move udlfb forward.
This first release intentionally adds no fundamentally new features. It only gets udlfb up to sync with the displaylink-mod branch (up to Roberto’s last 0.3 release in July 2009) that has been in use the past few months. With this update, displaylink-mod users should be able to switch to this version of udlfb transparently.
Bug reports are very welcome, especially regressions or problems that would stand in the way of moving this driver forward out of staging (comments here are fine for bug reports).
New in 0.4.0 (since 0.2.3 currently in the Linux kernel staging tree)
- Add dynamic modeset support (from displaylink-mod 0.3 and libdlo)
- udlfb uses EDID to find the monitor’s preferred mode
- udlfb no longer has fixed mode tables – it’s able to set any mode (within the capabilities of the chip) dynamically, from the standard VESA timing characteristics of the mode
- Fix teardown synchronization issues (from displaylink-mod 0.3)
- Other minor changes related to probe/modeset (from displaylink-mod 0.3)
- Functionally identical to displaylink-mod 0.3
- Retains basic layout of udlfb to make diffs more transparent and understandable
Download
See the git project summary page at http://git.plugable.com/gitphp/index.php?p=udlfb&a=summary for information.
Switching from displaylink-mod to udlfb.
Both these drivers match against all displaylink devices. So you don’t want both loading on your system. To clear out displaylink-mod:
sudo rmmod displaylink sudo rm /lib/modules/`uname -r`/extra/displaylink.ko sudo depmod -a
Then download, compile, install udlfb 0.4
./configure make sudo make install sudo depmod -a
Todo
- Merge in enhancements from Jaya Kumar’s displaylinkfb branch (defio support)
- Merge in enhancements from Bernie’s displaylink-mod branch (performance)
- Clear up remaining endian issues, to make sure it works on ARM and others
- Add performance metrics, and sysfs attributes to read/reset them
- Make allocation/use of backbuffer a runtime option, with param and sysfs switch
- Figure out what KMS (Kernel Mode Setting) means to framebuffer drivers
- Move from single URB with synchronous dispatch to ring of USB URBs, with asynchronous dispatch
- Enhance probe() to better handle chip type detection
- Enahance mode selection to better handle limits of DisplayLink chip
- Add simulated hardware cursor support, to prioritize mouse movement
Any feedback or ideas on these todos are very welcome. And, as always, patches are very welcome and will be incorporated as quickly as possible.
DisplayLink kernel framebuffer performance
Posted on 02. Nov, 2009 by Bernie Thompson in Programming
There are three codelines of Linux DisplayLink kernel frambuffer drivers currently in use:
- udlfb (Roberto De Ioris), which is in the Linux kernel staging tree of 2.6.31 and later, and is enabled by default in some recent distros (Ubuntu 9.10). Capable of working with all DisplayLink devices.
- displaylink-mod (Roberto De Ioris), which adds dynamic mode support and a few other minor changes
- displaylinkfb (Jaya Kumar), which tries out some innovative approaches (defio page-fault change detection), uses existing fbdev x servers without modification, but is much slower (80% slower on example test) than udlfb and displaylink-mod, which both use X damage information and RL/RAW compression
- udl (Marcus Glocker) for FreeBSD (not Linux) text/graphics console driver interface, which makes use of damage and ports Huffman-style compression from Florian Echtler’s libtubecable library. This support is currently ahead of what’s on Linux.
You can get more information about these drivers at http://libdlo.freedesktop.org/wiki/HowTo
In general, most of the demos and videos posted here use displaylink-mod. Some performance improvement patches are available at http://git.plugable.com/, but they are a relatively small improvement.
So any conclusions of of the performance work so far?
- Graphics benchmarks on Linux are in rough shape. Most practical approach so far has been simplistically using a combination of glxgears and a few select tests from x11perf. Good video playback tests needed.
- Jaya’s displaylinkfb driver tries some interesting concepts, but the lack of damage information from X means it runs much slower (up to 80% slower) than the alternatives for now.
- The original RL compression in udlfb by Henrik Bjerregaard Pedersen is surprisingly effective, even though it only uses one of RLE or RAW for each 255 pixel segment. It’s relatively CPU-efficient with simple inner loops, and decently USB-efficient in practice.
- The alternating RL/RAW algorithm in the master branch at http://git.plugable.com/ does slightly better, but it varies per test and is not dramatic.
- The shadow/back buffer that udlfb and displaylink-mod keep do provide a significant gain on maybe one out of every 4 tests or so, but X’s damage information is quite accurate — so saving that allocation and the ongoing reads/writes to that extra memory by forgoing the back buffer is definitely viable and should at least be a module option. The ‘noback’ branch at http://git.plugable.com/ has this removed (despite what one comment checkin says), for anyone who wants to try it.
Perhaps the best and quickest path forward to getting support more widely distributed is
- Bringing udlfb up to snuff, since there are just a few functional changes in the displaylink-mod branch, and udlfb already is going through the staging->mainline confidence building process
- For the matching X server, it would be great to have kernel driver that works with both the standard http://cgit.freedesktop.org/xorg/driver/xf86-video-fbdev/ at some level of performance, or with the custom displaylink xserver at some (better) level of performance. Then move things (like damage support, which is key to performance) from the displaylink server to the fbdev server in a standardized way over time.
- Then there’s a bunch of other more involved work to come on configuration and coexistence with multiple graphics controllers. Rough plans are visible in the fog here, but they involve other projects and people.
Recent Comments