Last year, he demoed turning your router into a full-fledged computer.

This year, it’s the Google Nexus One phone as a full computer — with attached external keyboard, mouse, display, and more.

The Nexus One demo is using the udlfb Linux kernel module to talk with the DisplayLink device, and it will work with any DisplayLink device, including Plugable’s.

Note the version of udlfb in the 2.6.34 staging tree unfortunately didn’t work for Sven. So he’s now using the latest udlfb from http://git.plugable.com/, which will likely get merged in for kernel 2.6.36.

Also, he used a dual headed cable to get enough power. A powered hub or a docking station/terminal like the UD-160-A won’t need that — it supplies its own power from AC – all the hardware needed is in the one package.

It’s exciting to have all this open source work come together in interesting demos like this.

There was a question recently why udlfb doesn’t use the same compression technique as the Windows drivers. Among other reasons, one is that the RL compression used by udlfb scales down to devices like the ones Sven has been working on — it’s as light as possible on CPU load, while getting decent compression.

Stepping back, it’s clear Apple (and now Microsoft) are making a mistake by limiting the hardware ecosystem around their devices. Android and the other Linux variants have an opportunity here — and considering the Apple juggernaut, they definitely need every advantage.

Sven’s demos show how powerful these scenarios can be. The hardware is ready. Devices like Plugable’s are designed with these scenarios in mind. Now we need to get the software refined and included in standard distributions, so normal consumers can take advantage of all the possibilities and benefits here.

We’re very excited about the Google funded project to refine USB multiseat on Linux, with the winning proposal from Lucas Nascimento Ferreira at the Federal University of Parana in Brazil.

In addition to providing mentoring, Plugable is also providing donated hardware for this project. Two Plugable Universal Docking stations with recent enhancements for use as a terminal, were just picked up by Lucas in Brazil today. By next month, we expect to have updated versions of the UGA-125-HUB for terminal use to send down.

For those interested in learning more and potentially following Lucas’ work:

• Google’s SoC 2010 Timeline
• Lucas’ winning USB Multiseat Proposal

And we’ll post periodic status updates here.

Linux hasn’t had that support — it would try to set the highest mode the monitor is capable of, often resulting in a black screen. Especially common for the DL-125 chip, with its mode limits of 1440×900/1280×1024.

That’s a shame because the DL-125 chip is a smart choice in many cases – by limiting itself to those lower modes, it stays more consistently within the limits of the USB 2.0 bus, resulting in more consistent performance.

So coinciding with the launch of Plugable’s DisplayLink DL-125 based products (UGA-125 and UGA-125-HUB), changes have been implemented to bring Linux roughly up to the level of Windows and Mac in this area for DisplayLink devices. This also helps devices like the UD-160-A when running on monitors greater than its limit of 1920×1080.

The kernel framebuffer driver udlfb has been enhanced to read the resolution limit from the firmware descriptors of the device, and adhere to it.

On the X server side, we needed a driver which would limit itself to the resulting reduced mode list. Unfortunately, the existing displaylink X server reads EDID directly, and assumes the adapter can do whatever the monitor can do.

We’ve been wanting to get rid of the need for a displaylink-specific X server, and the standard xf86-video-fbdev driver runs with the best existing mode, rather than trying to set a higher one in EDID. So this was a good trigger for converting over.

So xf86-video-fbdev has been enhanced with X Damage protocol support, ported from Roberto’s displaylink driver. This is still a little in-flux from an interface perspective, but from a functional perspective it’s done and fully performant.

So it’s now possible to run with a modified generic fbdev driver, which talks to udlfb, with full performance and without needing defio (although there’s also some good news in the defio space, which will be posted about later).

You can grab the latest udlfb kernel module with a “git clone http://git.plugable.com/webdav/udlfb”. Compile with “make && sudo make install && sudo depmod -a”

And you can grab the latest modified xf86-video-fbdev with a “git clone http://git.plugable.com/webdav/xf86-video-fbdev”. Compile with “./autogen.sh && make && sudo make install”

You’ll need a very recent xorg-macros version (1.4), which in package “sudo apt-get install xutils-dev”

To use the new X server, you must turn on the new “ReportDamage” option to fbdev. Modify your existing xorg conf like this:

Section "Device"
  Identifier "dl"
  Driver "fbdev"
  Option "ReportDamage" "true"
  Option "fbdev" "/dev/fb0"
EndSection

And you should be all set to go. This new X server should work with the existing udlfb in the staging tree of kernel 2.6.31+ for now, as it’s re-using the same original ioctl. But may require modeset changes that are only in 2.6.34+.

The main focus is USB multiseat, and the details are on the xorg wiki for SoC 2010

With all the pieces that are just coming together now, there is a potential here to do a project with huge impact, without a massive amount of engineering. There is already a very solid proposal coming from a student in Brazil who has previously been involved with the MDM multiseat project.

Google’s deadline for applications is now just a few days away: April 9th.

If there’s interest from other parties, we’ll get everyone talking – or there are other related proposals that we might be able to get in at the last minute. Foremost among those is conversion of the DisplayLink USB driver udlfb and matching X server from a fbdev driver to the KMS model.

Here’s the basics of the USB multiseat opportunity:

USB Multiseat Refinement

Linux Multiseat setups have potential to significantly reduce the cost of computing, but can be hard to configure. Some progress has been made on USB multiseat, where all components of the “terminal’ (display, keyboard, mouse, and more) are on USB, so configuration can be fully plug and play – you can just assume that all devices on the same USB hub constitute a terminal.

Some early prototypes of this are working (see http://plugable.com/2009/11/16/setting-up-usb-multiseat-with-displaylink-on-linux-gdm-up-to-2-20/). The underlying kernel drivers and X servers are largely in place.

But recent changes to the X Server, ConsoleKit, and other components open the possibility for a cleaner implementation.

This SoC project would constitute the refinement/creation of configuration scripts to enable a standard Linux or *nix computer to automatically launch additional seats when a USB terminal is plugged in

* udev rules to detect hubs/devices which should be collectively treated as terminals
* udev attributes to label the set of devices with a common seat id
* udev triggers for on-demand generation of the appropriate Xorg config files, to allow seats to coexist with the primary display/devices.
* ConsoleKit scripts to launch independent GDM/X sessions for each USB terminal seat
* InputClass rules to cause the primary X session to ignore multiseat-assigned devices, and the appropriate seat to use them
* udev rules and X init scripts to grant access to audio, storage, and other devices to the person logged into the matching seat

The one-sentence goal of this project: To make USB multiseat fully plug and play for the end-user, and ready for any distro to safely and cleanly drop in at any time.

Know any aspiring software engineering students that might be interested? Check SoC info from Google and the SoC guidelines from xorg.

There will still be some bugs and issues to resolve — but this release has most of the features, specifically related to input handling and udev-based dynamic configuration, of a more solid USB multiseat solution.

In short:

• udev rules, which already can detect a grouping of USB devices that constitute a terminal, can now tell X to ignore or use particular devices more easily through udev attributes and more dynamic/independent xorg.conf.d scripts.
• udev is now the default configuration mechanism (the torch has been passed from the less flexible hal)

So the challenges to explore are now more in the area of ConsoleKit integration and whether we can use these new capabilities support not just pure-USB multiseat setups, but also a mix of seats with PCIe attached graphics, and with fully USB seats.

You can read more about the new 1.8 configuration features which multiseat will use at Peter Hutterer’s blog.

And read more about the RC itself at Phoronix

This catches the official kernel staging tree with everything on git.plugable.com up to Feb, 2010.

Background

udlfb provides Linux framebuffer (fbdev) support plus a private “damage” notification ioctl used by the Roberto DeIoris’ “displaylink” X server. Udlfb automatically detects and supports all current DisplayLink USB graphics chips – 120,160,115,125,165,195. It supports 16bpp, with modes up to the maximum supported by the DisplayLink chips. Plugging any DisplayLink device into a Linux PC with this support should result in a “green screen” which means udlfb has loaded, set the monitor default mode, and drawn to the device successfully.

What udlfb does *not* do on its own is make configuring X easy – X sits on udlfb, but getting X running still involves editing configuration files like xorg.conf in different ways depending on what you’re trying to do, what distro version you have, and what primary GPU you have.

Features in Linux kernel 2.6.34 so far

• Improved performance (about 20% average improvement over a variety of “benchmarks” like x11perf, gtkperf, glxgears). Video playback, flash games, etc. are all improved
• Asynchronous urb dispatch and no mutexes held for extended periods during render
• Slightly lower CPU consumption, fewer non-localized memory accesses, slightly higher average compression
• Better handling of switching from X to fbcon VTs and back
• Driver unloads more cleanly from either bottom-up (USB removal) or top-down (shutdown). Problem reports welcome.
• More standard EDID parsing and mode handling, using fbdev’s built in mode libraries (but creates a new dependency on them being enabled in kernel)
• Performance metrics reported through sysfs (read /sys/class/graphics/fbX/*metrics* – but will move to debugfs)
• EDID reported through sysfs (read /sys/class/graphics/graphics/fbX/edid)
• Can compile as module with or without defio or sys_ dependencies
• Lots of cleanup – tested on 64-bit, closer to endian clean, checkpatch.pl clean
• Support for standard fbdev clients (via defio) – unfortunately buggy yet (see todos)

Depending on timing, some additional todo items might make it, we’ll see.

Versioning

At one point, we wanted udlfb to have its own project cycles and release versioning to help clarify things here while udlfb evolved in the early days. Got pushback on doing this from the kernel maintainers. So versioning of udlfb is purely by kernel version – “this bug is in udlfb in kernel 2.6.34rc1″ or whatever. There’ll also be a newer changes that haven’t been submitted to the kernel (e.g. at http://git.plugable.com/). And it’s probably best to just version that per commit/checkin. “This bug shows up in commit 234b4e22…”. This is nice in a way – just check the stuff in somewhere let the world know what you’re doing – the only “release process” you need to coordinate with is getting the patches into the kernel itself. Or feel free also to send patches here, we’ll try to help them through the process.

Todo

• Add code to read pixel resolution limits from the device’s USB descriptors, and adhere to them. Avoids blank screen when monitors caps exceeds that of the adapter
• Add pseudo-32bpp mode to help the case of using Xinerama to extend desktop across PCIe/USB adapters. Not planning true 24/32-bpp mode yet because of complexity and perf tradeoffs
• Disable defio by default, unfortunately, because of several bugs without obvious solutions
  • rendering problems with pages (lines) that no longer get updated – writes to those are no longer triggering deferred processing
  • When running 2 USB adapters, dirty pages for one instance seem to affect the other, and vice versa (udlfb has no common state, so appears to be in defio or mmu handling)
  • cases where we hit a kernel oops in the deferred io handler when it tries to touch the pages it’s been asked to handle.
• Move perf metrics from sysfs to debugfs and add ABI doc for what remains, per GregKH
• Fix two ignored return values that Greg’s build environment caught
• Get a safe method from DisplayLink to detect future chips that might not be compatible with current ones, and bail out of probe

There’s lot of other stuff that could be done – and any patches are very welcome. One big open question is whether fbdev has legs, or whether this effort should be converted over to KMS/DRM/DRI.

Style wars (e.g. “tabs vs. spaces”) are a never-ending source of tension on projects, so I actually appreciate automation like checkpatch.pl to just lay down the law, and be done with it.

But .. having to sift through endless warnings when you run checkpatch.pl late in the development process is frustrating. And there’s no way most occasional kernel patchers will remember all the rules (especially if you strongly disagree with some of them — which I do). It’s much better if your editor can just warn you when you step off the golden path of enlightenment. I use emacs, so I looked around for solutions, but they were only incomplete, or not real-time (e.g. checkpatch.pl has an –emacs option to run in the compile window of emacs).

So here’s something a little less incomplete that I’ve added to my .emacs file to catch the majority of checkpatch.pl errors in real-time (by highlighting the offending code in red). Works only on recent emacs versions. Hopefully it’ll be useful to others pounding away on their patches.

;; *** BEGIN highlight checkpatch.pl warnings and errors ***
(add-hook 'c-mode-common-hook
    (lambda ()
      ;; this sets defaults to match many checkpatch.pl guidelines
      (c-set-style "linux")))
;; but doesn't warn us about violations these regexp catch common ones
(custom-set-faces
    '(my-warning-face ((((class color)) (:background "red"))) t))
(add-hook 'font-lock-mode-hook
    (function
        (lambda ()
            (setq font-lock-keywords
                (append font-lock-keywords
		    '(("^.\\{81\\}" (0 'my-warning-face t)))
                    '(("\\/\\/.*" (0 'my-warning-face t)))
                    '((";[_A-Za-z0-9]+" (0 'my-warning-face t)))
                    '((",[_A-Za-z0-9]+" (0 'my-warning-face t)))
                    '(("return[[:blank:]]*(.+);" (0 'my-warning-face t)))
                    '(("([_A-Za-z0-9]+[\\*]+)" (0 'my-warning-face t)))
                    '(("[[:blank:]]+\\)"(0 'my-warning-face t)))
		    '(("^[[:blank:]]+{[[:blank:]]*$" (0 'my-warning-face t)))
		    '(("[_A-Za-z0-9]+\\*[[:blank:]]" (0 'my-warning-face t)))
		)))))
;; exercise to reader - move regexps into c hook
;; *** END highlight checkpatch.pl warnings and errors ***

Any enhancements or corrections are welcome.

Also note this driver is just a foundation on which other stuff (like X) sits, so it doesn’t mean Linux is easy for real end users yet. But it’s getting closer.

Here’s the announce that went to the list today:

The next patch series for udlfb is “nearly ready” for submission, hopefully will be coming in the next few days.

* The defio rendering problems (e.g. with xf86-video-fbdev X server) are still unsolved
* Still 16bpp only. Pseudo-truecolor 32->16 is on the wishlist.
* No DL 1×5 SKU detection yet (you’ll get a blank screen if your monitor’s res exceeds the adapter)

A “no” in the above table means the client application will fail if that’s the only API option supported by the kernel framebuffer driver.

udlfb supports all four, with some limitations (e.g. defio is required to support mmap without damage reports).

The custom displaylink X server is the only client today which supports the last option, which is the most performant option on indirect displays (like USB displays) – and hopefully will become a standard interface as it’s useful in many cases.

Unfortunately for the DisplayLink udlfb framebuffer, each API has different constraints it places on the driver. Those constraints aren’t easily met all at once for a driver rendering to USB. AFAICT, it looks like the driver actually needs to figure out what kind of client it’s dealing with, and assume that client will stick to that sub-set of the API. That seems to be a safe assumption for the major fbdev apps today, but betting on something like this is always ugly.

Any misconceptions?