These days new interaction methods with the computer are getting more
important. One way is to use a multi touch surface, this means the user can use
more than one finger to control the computer. Two optical techniques are Frustrated Total Internal Reflection~(FTIR) and Diffuse Illumination~(DI). FTIR is using the physical effect of total internal reflection to generate blobs on a camera image if a
finger is touching the surface. DI uses a diffuse surface to generate camera
pictures that have only the hands clearly on them. Both of these techniques use
infrared light sources so the user would not see them when interacting with the
surface. The main idea of the thesis is, to combine FTIR and DI to enhance the
manual interaction between multi touch surfaces and the user as seen in
the image. FTIR can be used to determine whether a finger is touching the surface, but this technique can not be used to determine if specific touches belong together~(fingers of a hand). DI can be used to find a silhouette of the hand and therefore touches can be recognized as a hand. But with DI, it is hard to determine if a finger is really touching the surface. As we want to exploit the advantages of both techniques, we combine them in the following manner. We take the silhouette of the hand~(taken from DI image) and overlay it with the blobs~(taken from FTIR
image). The technique of using reference images(Ref) to suppress the ambient
infrared light is used, which is described here [2]. This combination should enhance the interaction with the computer, therefore a investigation is done whether the single techniques work better or the combination is a better. With the combination functions can be described for each finger of a hand, also it can be determined which hand is used(left or right). This information can be used to describe new gestures. It can be approximated how many users are working on this surface by counting hands(left and right).

A piece of software is developed to track the manual interaction with the multi
touch interface. Therefore the software, which is called from now on tracker,
controls a digital camera that takes FTIR and DI images alternating. To
suppress the ambient light that is coming from the environment a reference
image is taken from time to time and the reference image is subtracted by the
FTIR/DI images. These images are filtered and combined to get images that can
be analyzed. The position, rotation and distance of finger and hands on the
surface are estimated from that images. The gathered information is sent to a
client, which is interpreting this information to control user programs.

Download (pdf [3])