Linux is a big and constantly shifting platform. With our USB graphics products (and generally for DisplayLink based products, since we try to make them work for all devices), it’s easy to output a few pixels, but configuring a USB display as an X terminal, or certainly for an extended desktop, is still a process on Linux that requires manual xorg.conf editing and is for very advanced users only. But we try to contribute what we can at Plugable, and that has meant focusing on making the kernel driver that actually talks to the hardware and everything else builds on, as solid as possible.

The contributed patches start with this post to the linux driver project list.

————–
To: devel-request@linuxdriverproject.org
[PATCH 0/11] staging: udlfb: patches from udlfb development branch (Bernie Thompson)

This patch series contains all current fixes and features from
the udlfb development branch.

udlfb is a framebuffer driver for DisplayLink USB 2.0 era chips.

Diffstat of this 11 part patch series:

udlfb.c | 989 +++++++++++++++++++++++++++++++++++++++++———————–
udlfb.h | 41 +-
2 files changed, 664 insertions(+), 366 deletions(-)

Major changes:

* Added summary documentation for users of udlfb
* Added logic to query DisplayLink chip for max area mode,
so low-end chips on high-end monitors no longer get black screen
* Added support for DPMS. X servers now control monitor
power with existing standard interface
* Added back in support for char interface (e.g. cat file > /dev/fb0)
* Systems without EDID or with failing EDID can now supply fixed
EDID to sysfs interface, also avoiding black screen
* Fixed big-endian (PowerPC) rendering
* Fixed teardown race conditions that could result in shutdown hang
* Added fb_defio and fb console module options (default off)
* Fixed udlfb’s fb_defio client code so no longer incorrectly shares
state across udlfb device instances – fixes hangs and errant rendering
* Removed IFDEFs for building against older kernels – those will
be retained in the udlfb development branches at git.plugable.com

Todo:

There have been no additional reported bugs in the last few months,
although there are several wishlist features. Udlfb may be ready
to move out of staging at this point.

Patches are against Linus’ latest 2.6 tree.

This complete quilt patch series can be downloaded from http://plugable.com/udlfb-patches-2.6.35-v2.tgz

Plugable UD-160-A driving 1920x1080 display and more, all from a humble Nexus One phone

This setup uses the udlfb DisplayLink Linux driver work from here (git.plugable.com) and a bunch of other components Sven has developed or pulled together, to turn the phone into a full computer with display, keyboard, audio and more. His video of this has gone viral (over 85,000 views). You can see the video and full instructions at Sven’s site. Very cool.

The photo above is of the Plugable UD-160-A Universal Docking Station, in use at Sven’s desk (the dock is laying on its side to better fit in the photo – Sven has removed the weighted bottom stand).

The Plugable dock is perfect for this kind of application with the Nexus One phone — it includes all the USB devices in one place, all with open source drivers. The dock has its own 2.5A AC power (see the USB dual-power Y cable plugged into the front USB A and back USB B ports), along with driving the 1920×1080 monitor (the big DVI cable in the back), and providing ethernet (above the DVI) and audio (in the front – he doesn’t have them plugged in in the pic).

So this all is great for geeks, but when will this work out of the box? Several comments on Sven’s work are messages like “Google! Integrate this into Android now!”

With Google offices near here, and with related USB terminal work on the same Plugable dock being funded as a Google Summer of Code project, it would be great to demo this and get things moving… Whatcha say, Google!?

Last year, he demoed turning your router into a full-fledged computer.

This year, it’s the Google Nexus One phone as a full computer — with attached external keyboard, mouse, display, and more.

The Nexus One demo is using the udlfb Linux kernel module to talk with the DisplayLink device, and it will work with any DisplayLink device, including Plugable’s.

Note the version of udlfb in the 2.6.34 staging tree unfortunately didn’t work for Sven. So he’s now using the latest udlfb from http://git.plugable.com/, which will likely get merged in for kernel 2.6.36.

Also, he used a dual headed cable to get enough power. A powered hub or a docking station/terminal like the UD-160-A won’t need that — it supplies its own power from AC – all the hardware needed is in the one package.

It’s exciting to have all this open source work come together in interesting demos like this.

There was a question recently why udlfb doesn’t use the same compression technique as the Windows drivers. Among other reasons, one is that the RL compression used by udlfb scales down to devices like the ones Sven has been working on — it’s as light as possible on CPU load, while getting decent compression.

Stepping back, it’s clear Apple (and now Microsoft) are making a mistake by limiting the hardware ecosystem around their devices. Android and the other Linux variants have an opportunity here — and considering the Apple juggernaut, they definitely need every advantage.

Sven’s demos show how powerful these scenarios can be. The hardware is ready. Devices like Plugable’s are designed with these scenarios in mind. Now we need to get the software refined and included in standard distributions, so normal consumers can take advantage of all the possibilities and benefits here.

The goal of their project was to get an embedded platform working on Centos 5.3. You can read more in their release note for the work.

To make the contributions easier to diff, they’ve been checked into a git branch from a (best guess) of the code Endurance started from.

The changes are available here for the kernel driver and here for the X server.

To actually build it, you may need their full package.

It’s unclear what of these changes might get picked up, as things have moved on since. One of the main features of this code is providing a default EDID blob in udlfb. The plan for udlfb at this point is to allow the edid sysfs interface of udlfb to be written to, to allow (and give responsibility) to a user-mode entity to supply a default edid, if one isn’t available from hardware.

Thanks again to Endurance for being diligent about contributing changes back.

We’re very excited about the Google funded project to refine USB multiseat on Linux, with the winning proposal from Lucas Nascimento Ferreira at the Federal University of Parana in Brazil.

In addition to providing mentoring, Plugable is also providing donated hardware for this project. Two Plugable Universal Docking stations with recent enhancements for use as a terminal, were just picked up by Lucas in Brazil today. By next month, we expect to have updated versions of the UGA-125-HUB for terminal use to send down.

For those interested in learning more and potentially following Lucas’ work:

• Google’s SoC 2010 Timeline
• Lucas’ winning USB Multiseat Proposal

And we’ll post periodic status updates here.

Linux hasn’t had that support — it would try to set the highest mode the monitor is capable of, often resulting in a black screen. Especially common for the DL-125 chip, with its mode limits of 1440×900/1280×1024.

That’s a shame because the DL-125 chip is a smart choice in many cases – by limiting itself to those lower modes, it stays more consistently within the limits of the USB 2.0 bus, resulting in more consistent performance.

So coinciding with the launch of Plugable’s DisplayLink DL-125 based products (UGA-125 and UGA-125-HUB), changes have been implemented to bring Linux roughly up to the level of Windows and Mac in this area for DisplayLink devices. This also helps devices like the UD-160-A when running on monitors greater than its limit of 1920×1080.

The kernel framebuffer driver udlfb has been enhanced to read the resolution limit from the firmware descriptors of the device, and adhere to it.

On the X server side, we needed a driver which would limit itself to the resulting reduced mode list. Unfortunately, the existing displaylink X server reads EDID directly, and assumes the adapter can do whatever the monitor can do.

We’ve been wanting to get rid of the need for a displaylink-specific X server, and the standard xf86-video-fbdev driver runs with the best existing mode, rather than trying to set a higher one in EDID. So this was a good trigger for converting over.

So xf86-video-fbdev has been enhanced with X Damage protocol support, ported from Roberto’s displaylink driver. This is still a little in-flux from an interface perspective, but from a functional perspective it’s done and fully performant.

So it’s now possible to run with a modified generic fbdev driver, which talks to udlfb, with full performance and without needing defio (although there’s also some good news in the defio space, which will be posted about later).

You can grab the latest udlfb kernel module with a “git clone http://git.plugable.com/webdav/udlfb”. Compile with “make && sudo make install && sudo depmod -a”

And you can grab the latest modified xf86-video-fbdev with a “git clone http://git.plugable.com/webdav/xf86-video-fbdev”. Compile with “./autogen.sh && make && sudo make install”

You’ll need a very recent xorg-macros version (1.4), which in package “sudo apt-get install xutils-dev”

To use the new X server, you must turn on the new “ReportDamage” option to fbdev. Modify your existing xorg conf like this:

Section "Device"
  Identifier "dl"
  Driver "fbdev"
  Option "ReportDamage" "true"
  Option "fbdev" "/dev/fb0"
EndSection

And you should be all set to go. This new X server should work with the existing udlfb in the staging tree of kernel 2.6.31+ for now, as it’s re-using the same original ioctl. But may require modeset changes that are only in 2.6.34+.

The main focus is USB multiseat, and the details are on the xorg wiki for SoC 2010

With all the pieces that are just coming together now, there is a potential here to do a project with huge impact, without a massive amount of engineering. There is already a very solid proposal coming from a student in Brazil who has previously been involved with the MDM multiseat project.

Google’s deadline for applications is now just a few days away: April 9th.

If there’s interest from other parties, we’ll get everyone talking – or there are other related proposals that we might be able to get in at the last minute. Foremost among those is conversion of the DisplayLink USB driver udlfb and matching X server from a fbdev driver to the KMS model.

Here’s the basics of the USB multiseat opportunity:

USB Multiseat Refinement

Linux Multiseat setups have potential to significantly reduce the cost of computing, but can be hard to configure. Some progress has been made on USB multiseat, where all components of the “terminal’ (display, keyboard, mouse, and more) are on USB, so configuration can be fully plug and play – you can just assume that all devices on the same USB hub constitute a terminal.

Some early prototypes of this are working (see http://plugable.com/2009/11/16/setting-up-usb-multiseat-with-displaylink-on-linux-gdm-up-to-2-20/). The underlying kernel drivers and X servers are largely in place.

But recent changes to the X Server, ConsoleKit, and other components open the possibility for a cleaner implementation.

This SoC project would constitute the refinement/creation of configuration scripts to enable a standard Linux or *nix computer to automatically launch additional seats when a USB terminal is plugged in

* udev rules to detect hubs/devices which should be collectively treated as terminals
* udev attributes to label the set of devices with a common seat id
* udev triggers for on-demand generation of the appropriate Xorg config files, to allow seats to coexist with the primary display/devices.
* ConsoleKit scripts to launch independent GDM/X sessions for each USB terminal seat
* InputClass rules to cause the primary X session to ignore multiseat-assigned devices, and the appropriate seat to use them
* udev rules and X init scripts to grant access to audio, storage, and other devices to the person logged into the matching seat

The one-sentence goal of this project: To make USB multiseat fully plug and play for the end-user, and ready for any distro to safely and cleanly drop in at any time.

Know any aspiring software engineering students that might be interested? Check SoC info from Google and the SoC guidelines from xorg.

There will still be some bugs and issues to resolve — but this release has most of the features, specifically related to input handling and udev-based dynamic configuration, of a more solid USB multiseat solution.

In short:

• udev rules, which already can detect a grouping of USB devices that constitute a terminal, can now tell X to ignore or use particular devices more easily through udev attributes and more dynamic/independent xorg.conf.d scripts.
• udev is now the default configuration mechanism (the torch has been passed from the less flexible hal)

So the challenges to explore are now more in the area of ConsoleKit integration and whether we can use these new capabilities support not just pure-USB multiseat setups, but also a mix of seats with PCIe attached graphics, and with fully USB seats.

This catches the official kernel staging tree with everything on git.plugable.com up to Feb, 2010.

Background

udlfb provides Linux framebuffer (fbdev) support plus a private “damage” notification ioctl used by the Roberto DeIoris’ “displaylink” X server. Udlfb automatically detects and supports all current DisplayLink USB graphics chips – 120,160,115,125,165,195. It supports 16bpp, with modes up to the maximum supported by the DisplayLink chips. Plugging any DisplayLink device into a Linux PC with this support should result in a “green screen” which means udlfb has loaded, set the monitor default mode, and drawn to the device successfully.

What udlfb does *not* do on its own is make configuring X easy – X sits on udlfb, but getting X running still involves editing configuration files like xorg.conf in different ways depending on what you’re trying to do, what distro version you have, and what primary GPU you have.

Features in Linux kernel 2.6.34 so far

• Improved performance (about 20% average improvement over a variety of “benchmarks” like x11perf, gtkperf, glxgears). Video playback, flash games, etc. are all improved
• Asynchronous urb dispatch and no mutexes held for extended periods during render
• Slightly lower CPU consumption, fewer non-localized memory accesses, slightly higher average compression
• Better handling of switching from X to fbcon VTs and back
• Driver unloads more cleanly from either bottom-up (USB removal) or top-down (shutdown). Problem reports welcome.
• More standard EDID parsing and mode handling, using fbdev’s built in mode libraries (but creates a new dependency on them being enabled in kernel)
• Performance metrics reported through sysfs (read /sys/class/graphics/fbX/*metrics* – but will move to debugfs)
• EDID reported through sysfs (read /sys/class/graphics/graphics/fbX/edid)
• Can compile as module with or without defio or sys_ dependencies
• Lots of cleanup – tested on 64-bit, closer to endian clean, checkpatch.pl clean
• Support for standard fbdev clients (via defio) – unfortunately buggy yet (see todos)

Depending on timing, some additional todo items might make it, we’ll see.

Versioning

At one point, we wanted udlfb to have its own project cycles and release versioning to help clarify things here while udlfb evolved in the early days. Got pushback on doing this from the kernel maintainers. So versioning of udlfb is purely by kernel version – “this bug is in udlfb in kernel 2.6.34rc1″ or whatever. There’ll also be a newer changes that haven’t been submitted to the kernel (e.g. at http://git.plugable.com/). And it’s probably best to just version that per commit/checkin. “This bug shows up in commit 234b4e22…”. This is nice in a way – just check the stuff in somewhere let the world know what you’re doing – the only “release process” you need to coordinate with is getting the patches into the kernel itself. Or feel free also to send patches here, we’ll try to help them through the process.

Todo

• Add code to read pixel resolution limits from the device’s USB descriptors, and adhere to them. Avoids blank screen when monitors caps exceeds that of the adapter
• Add pseudo-32bpp mode to help the case of using Xinerama to extend desktop across PCIe/USB adapters. Not planning true 24/32-bpp mode yet because of complexity and perf tradeoffs
• Disable defio by default, unfortunately, because of several bugs without obvious solutions
  • rendering problems with pages (lines) that no longer get updated – writes to those are no longer triggering deferred processing
  • When running 2 USB adapters, dirty pages for one instance seem to affect the other, and vice versa (udlfb has no common state, so appears to be in defio or mmu handling)
  • cases where we hit a kernel oops in the deferred io handler when it tries to touch the pages it’s been asked to handle.
• Move perf metrics from sysfs to debugfs and add ABI doc for what remains, per GregKH
• Fix two ignored return values that Greg’s build environment caught
• Get a safe method from DisplayLink to detect future chips that might not be compatible with current ones, and bail out of probe

There’s lot of other stuff that could be done – and any patches are very welcome. One big open question is whether fbdev has legs, or whether this effort should be converted over to KMS/DRM/DRI.

Style wars (e.g. “tabs vs. spaces”) are a never-ending source of tension on projects, so I actually appreciate automation like checkpatch.pl to just lay down the law, and be done with it.

But .. having to sift through endless warnings when you run checkpatch.pl late in the development process is frustrating. And there’s no way most occasional kernel patchers will remember all the rules (especially if you strongly disagree with some of them — which I do). It’s much better if your editor can just warn you when you step off the golden path of enlightenment. I use emacs, so I looked around for solutions, but they were only incomplete, or not real-time (e.g. checkpatch.pl has an –emacs option to run in the compile window of emacs).

So here’s something a little less incomplete that I’ve added to my .emacs file to catch the majority of checkpatch.pl errors in real-time (by highlighting the offending code in red). Works only on recent emacs versions. Hopefully it’ll be useful to others pounding away on their patches.

;; *** BEGIN highlight checkpatch.pl warnings and errors ***
(add-hook 'c-mode-common-hook
    (lambda ()
      ;; this sets defaults to match many checkpatch.pl guidelines
      (c-set-style "linux")))
;; but doesn't warn us about violations these regexp catch common ones
(custom-set-faces
    '(my-warning-face ((((class color)) (:background "red"))) t))
(add-hook 'font-lock-mode-hook
    (function
        (lambda ()
            (setq font-lock-keywords
                (append font-lock-keywords
		    '(("^.\\{81\\}" (0 'my-warning-face t)))
                    '(("\\/\\/.*" (0 'my-warning-face t)))
                    '((";[_A-Za-z0-9]+" (0 'my-warning-face t)))
                    '((",[_A-Za-z0-9]+" (0 'my-warning-face t)))
                    '(("return[[:blank:]]*(.+);" (0 'my-warning-face t)))
                    '(("([_A-Za-z0-9]+[\\*]+)" (0 'my-warning-face t)))
                    '(("[[:blank:]]+\\)"(0 'my-warning-face t)))
		    '(("^[[:blank:]]+{[[:blank:]]*$" (0 'my-warning-face t)))
		    '(("[_A-Za-z0-9]+\\*[[:blank:]]" (0 'my-warning-face t)))
		)))))
;; exercise to reader - move regexps into c hook
;; *** END highlight checkpatch.pl warnings and errors ***

Any enhancements or corrections are welcome.