ENAC's Interactive Computing Laboratory多点触控

Linux input

Researchers had suggested it for decades: humans do not have a left hand with 101 fingers and a right hand with 1 to 3 fingers. Now that the computing industry is realising this, we need operating systems with a rich enough input model. Contributing to Linux-based operating systems is a good reality check for the lab's research on input management in interactive software. It can also help the aeronautical industry to better understand some issues associated to modern input systems.

The lab's involvement in Linux started with multitouch devices. Ultimately, we hope to be able to contribute to the evolution of input management so as to support a wider range of input configurations, and even multimodal input.

Public resources

ourmultitouch primer

ourlist of multitouch devices and their support status in Linux

ourgeneral Linux multitouch howtos

ourUbuntu-specific multitouch howtos

ourAndroid-specific multitouch howtos

ourtuiototouch bridge

ourmultitouch diagnostic application

outgit repositories

Related projects

ShareIT,Istar Related publications

Stéphane Chatty, Alexandre Lemort, Stéphane Valès. Multiple input support in a model-based interaction framework . Proceedings of the second annual IEEE international workshop on horizontal interactive human-computer systems (Tabletop 2007), IEEE computer society, 2007

Contact: chatty at enac.fr

Updated Dec 29, 2010.

January 2011: multi touch typing

How it works: Our experimental multitouch gesture recognition algorithm feeds simulated keyboard events to the Linux kernel. Linux does the rest.

Contact: chatty at enac.fr, pengfei.zheng at eleve.enac.fr

Updated Jan 19, 2011.

January 2011: The tuiototouch bridge

You need:

Ubuntu 10.10

a computer with a camera

a TUIO-compatible computer vision software (eg. CCV)

ourtuiototouch bridge

How it works: The tuiototouch bridge converts multitouch events emitted by the vision software on TUIO and injects it in the kernel, simulating a standard multitouch panel. Ubuntu Unity or your favorite multitouch-compatible software does the rest.

Contact: chatty at enac.fr, charvet at lii-enac.fr

Updated Jan 19, 2011.

December 2009: Support for multitouch in Xorg

This video made by Benjamin Tissoires, Stéphane Chatty and Gilles Tabart from the Interactive Computing Lab atENAC demonstrates how Stephane's kernel drivers and Benjamin's patched X.org evdev driver allow to use existing applications with multitouch displays.

You need:

Linux kernel 2.6.31 or more recent;

a computer with a Linux-compatible multitouch device. Here is a list ofavailable multitouch devices and their status;

the corresponding driver. Some are already in the Linux kernel sources, others are in the lab'sgit repositories or here:Stantum MTP,DiamondTouch;

you will also need to import bits from a recent hid-core.c and hid-ids.h into the drivers/hid directory of your kernel sources. Here is some more information onhow to proceed with the kernel;

X.org version X11R7.5, available for instance in Fedora 12;

Benjamin's X.orgevdev driver andmultitouch management client (obsolete since september 2010). You can find more up to date informationhere on how to proceed.

How it works: Actually, most of the work is done by MPX. When loading the customized evdev driver, the multitouch device is recognized as a touchscreen.

By changing the property "Evdev MultiTouch", the user can control the number of contact points that are recognized. Thus the device only transfers the contact points to the newly created subdevices.

This property can easily be modified with "multitouchctl x", where 'x' has the following meaning:

if x = 0, touchscreen emulation,

if x > 0 it is the number of contact points recognized.

The daemon "multitouchd" monitors/controls the property and creates the master devices. It hides or shows the cursors depending on their valuator "TrackingID" and also sends the press/release events at the beginning/end of movements.

We split the creation of subdevices and cursors in order to be able to implement a gesture recognition in user-space (TODO).

To understand what is actually happening inside your server, the command "xinput" is very useful!

Context: This work was done as part of project ShareIT, a research project in which we collaborate with our good friends the multitouch hardware makers atStantum, the multitouch software and interaction experts atIntuiLab and the aircraft cockpit designers atThales Avionics to explore the use of multitouch user interfaces in cockpits. But no, there is no plan to use Linux in the cockpit, this is just for the lab's research :-)

Multitouch? Here is more information about multitouch interaction and available devices, multitouch on Linux, etc.

Contact: chatty at enac.fr, tissoire at lii-enac.fr

Updated Dec 29, 2010.

June 2009: Linux drivers for multitouch screens

This demo made by Mohamed-Ikbel Boulabiar, Stéphane Chatty and Sébastien Hamdani from the Interactive Computing Lab atENAC shows how one can use the multi-touch capabilities that Henrik Rydberg added to the Linux input system.

You need:

Linux kernel 2.6.31 or more recent;

a computer with a Linux-compatible multitouch device. Here is a list ofavailable multitouch devices and their support status;

the corresponding driver. Most are already in the Linux kernel sources, others are available from the list of devices above. Here is information about howhow to install the drivers;

ourdemo code.

How to use the code:

The demo code reads input directly from the device file (/dev/input/eventX). The X server is not involved in input handling at this stage. The demos perform very simple gesture recognition then send DBus messages to Compiz to produce effects. You need to activate Compiz with the DBus plugin to get the demo working, and for window rotate/scale you will also need the freewins plugin.

The roadmap:

These demos were produced in Spring 2009. The multitouch events are in the kernel since version 2.6.30, as well as the Broadcom 5974 driver. Our multitouch code in the N-Trig driver is in the kernel since version 2.6.31. Our Stantum HID, Quanta, MosArt and 3M drivers are in the kernel since 2.6.34 and in Ubuntu since 10.04. Our DiamondTouch drivers will be submitted as soon as CircleTwelve and/or Mitsubish Electric give us the green light for redistributing their firmware under GPL. The lab is currently contributing to the multitouch management in X.org, and involved in the thinking on how to reorganise input management in both Xorg and the kernel to better deal with multitouch, including gesture recognition.

Context:

This work was done as part of project ShareIT, a research project in which we collaborate with our good friends the multitouch hardware makers atStantum, the multitouch software and interaction experts atIntuiLab and the aircraft cockpit designers atThales Avionics to explore the use of multitouch user interfaces in cockpits. But no, there is no plan to use Linux in the cockpit, this is just for the lab's research :-)

Multitouch?

Here is more information about multitouch interaction and available devices, multitouch on Linux, etc.

Contact: chatty at enac.fr

Updated Feb 6, 2010.