Abstract
The rising popularity of digital table surfaces has spawned considerable interest in new interaction techniques. Most interactions fall into one of two modalities: 1) direct touch and multi-touch (by hand and by tangibles) directly on the surface, and 2) hand gestures above the surface. The limitation is that these two modalities ignore the rich interaction space between them. To move beyond this limitation, we first contribute a unification of these discrete interaction modalities called the continuous interaction space. The idea is that many interaction techniques can be developed that go beyond these two modalities, where they can leverage the space between them. That is, we believe that the underlying system should treat the space on and above the surface as a continuum, where a person can use touch, gestures, and tangibles anywhere in the space and naturally move between them. Our second contribution illustrates this, where we introduce a variety of interaction categories that exploit the space between these modalities. For example, with our Extended Continuous Gestures category, a person can start an interaction with a direct touch and drag, then naturally lift off the surface and continue their drag with a hand gesture over the surface. For each interaction category, we implement an example (or use prior work) that illustrates how that technique can be applied. In summary, our primary contribution is to broaden the design space of interaction techniques for digital surfaces, where we populate the continuous interaction space both with concepts and examples that emerge from considering this space as a continuum.
Chapter PDF
Similar content being viewed by others
Keywords
References
Bartindale, T., Harrison, C.: Stacks on the surface: resolving physical order using fiducial markers with structured transparency. In: Proc. of ITS 2009. ACM, New York (2009)
Baudisch, P., Becker, T., Rudeck, F.: Lumino: tangible blocks for tabletop computers based on glass fiber bundles. In: Proc. of CHI 2010, pp. 1165–1174. ACM, New York (2010)
Benko, H., Ishak, E.W., Feiner, S.: Cross-dimensional gestural interaction techniques for hybrid immersive environments. In: Proc. of IEEE VR 2005, pp. 209–216 (2005)
Benko, H., Saponas, T.S., Morris, D., Tan, D.: Enhancing input on and above the interactive surface with muscle sensing. In: Proc. of ITS 2009, pp. 93–100. ACM, New York (2009)
Bier, E.A., Stone, M.C., Pier, K., Buxton, W., DeRose, T.D.: Toolglass and magic lenses: the see-through interface. In: Proc. of SIGGRAPH 1993, pp. 73–80. ACM, New York (1993)
Buxton, W.: A three-state model of graphical input. In: Proc. of INTERACT 1990, pp. 449–456. North-Holland Publishing Co, Amsterdam (1990)
Cao, X., Wilson, A., Balakrishnan, R., Hinckley, K., Hudson, S.: ShapeTouch: Leveraging Contact Shape on Interactive Surfaces. In: Proc. of TABLETOP 2008. IEEE, Los Alamitos (2008)
Cockburn, A., Bryant, A.: Leogo: An Equal Opportunity User Interface for Programming. Journal of Visual Languages & Computing 8(5-6), 601–619 (1997)
Echtler, F., Huber, M., Klinker, G.: Shadow tracking on multi-touch tables. In: Proc. of AVI 2008, pp. 388–391. ACM, New York (2008)
Epps, J., Lichman, S., Wu, M.: A study of hand shape use in tabletop gesture interaction. In: CHI 2006 Extended Abstracts, pp. 748–753. ACM, New York (2006)
Grossman, T., Wigdor, D.: Going Deeper: a Taxonomy of 3D on the Tabletop. In: Proc. of TABLETOP 2007, pp. 137–144. IEEE, Los Alamitos (2007)
Grossman, T., Balakrishnan, R.: The design and evaluation of selection techniques for 3D volumetric displays. In: Proc. of UIST 2006, pp. 3–12. ACM, New York (2006)
Hancock, M., Carpendale, S., Cockburn, A.: Shallow-depth 3d interaction: design and evaluation of one-, two- and three-touch techniques. In: Proc. of CHI 2007, pp. 1147–1156. ACM, New York (2007)
Hancock, M., ten Cate, T., Carpendale, S., Isenberg, T.: Supporting sandtray therapy on an interactive tabletop. In: Proc. of CHI 2010, pp. 2133–2142. ACM, New York (2010)
Hancock, M., ten Cate, T., Carpendale, S.: Sticky tools: full 6DOF force-based interaction for multi-touch tables. In: Proc. of ITS 2009, pp. 133–140. ACM, New York (2009)
Hilliges, O., Izadi, S., Wilson, A.D., Hodges, S., Garcia-Mendoza, A., Butz, A.: Interactions in the air: adding further depth to interactive tabletops. In: Proc. of UIST 2009, pp. 139–148. ACM, New York (2009)
Izadi, S., Hodges, S., Taylor, S., et al.: Going beyond the display: a surface technology with an electronically switchable diffuser. In: Proc. of UIST 2008, pp. 269–278. ACM, New York (2008)
Kattinakere, R.S., Grossman, T., Subramanian, S.: Modeling steering within above-the-surface interaction layers. In: Proc. of CHI 2007, pp. 317–326. ACM, New York (2007)
Lee, J., Lee, J., Kim, H., Kim, J.: Gesture-Based Interactions on Multiple Large Displays with a Tabletop Interface. In: Universal Access in Human-Computer Interaction. Ambient Interaction, pp. 936–942 (2007)
Lucero, A., Aliakseyeu, D., Martens, J.: Augmenting Mood Boards: Flexible and Intuitive Interaction in the Context of the Design Studio. In: Proc. of TABLETOP 2007, pp. 147–154. IEEE, Los Alamitos (2007)
Malik, S., Ranjan, A., Balakrishnan, R.: Interacting with large displays from a distance with vision-tracked multi-finger gestural input. In: Proc. of UIST 2005, pp. 43–52. ACM, New York (2005)
Marquardt, N., Kiemer, J., Greenberg, S.: What Caused That Touch? Expressive Interaction with a Surface through Fiduciary-Tagged Gloves. In: Proc. of ITS 2010, pp. 139–142. ACM, New York (2010)
Parker, J.K., Mandryk, R.L., Inkpen, K.M.: TractorBeam: seamless integration of local and remote pointing for tabletop displays. In: Proc. of Graphics Interface - GI 2005, pp. 33–40. Canadian Information Processing Society (2005)
Parker, J.K., Mandryk, R.L., Inkpen, K.M.: Integrating Point and Touch for Interaction with Digital Tabletop Displays. IEEE Comput. Graph. Appl. 26(5), 28–35 (2006)
Pinelle, D., Nacenta, M., Gutwin, C., Stach, T.: The effects of co-present embodiments on awareness and collaboration in tabletop groupware. In: Proc. of Graphics Interface - GI 2008, pp. 1–8. Canadian Information Processing Society (2008)
Shoemaker, G., Tsukitani, T., Kitamura, Y., Booth, K.S.: Body-centric interaction techniques for very large wall displays. In: Proc. of NordiCHI 2010, pp. 463–472. ACM, New York (2010)
Spindler, M., Stellmach, S., Dachselt, R.: PaperLens: advanced magic lens interaction above the tabletop. In: Proc. of ITS 2009, pp. 69–76. ACM, New York (2009)
Starner, T., Leibe, B., Minnen, D., Westeyn, T., Hurst, A., Weeks, J.: The perceptive workbench: Computer-vision-based gesture tracking, object tracking, and 3D reconstruction for augmented desks. Machine Vision and Applications 14(1) (2003)
Sturman, D.J., Zeltzer, D.: A Survey of Glove-based Input. IEEE Comput. Graph. Appl. 14(1), 30–39 (1994)
Subramanian, S., Aliakseyeu, D., Lucero, A.: Multi-layer interaction for digital tables. In: Proc. UIST 2006, pp. 269–272. ACM, New York (2006)
Ullmer, B., Ishii, H.: The metaDESK: models and prototypes for tangible user interfaces. In: Proc. of UIST 1997, pp. 223–232. ACM, New York (1997)
Underkoffler, J., Ishii, H.: Urp: a luminous-tangible workbench for urban planning and design. In: Proc. of CHI 1999, pp. 386–393. ACM, New York (1999)
Vogel, D., Balakrishnan, R.: Distant freehand pointing and clicking on very large, high resolution displays. In: Proc. of UIST 2005, pp. 33–42. ACM, New York (2005)
Voida, S., Tobiasz, M., Stromer, J., Isenberg, P., Carpendale, S.: Getting practical with interactive tabletop displays: designing for dense data, “fat fingers,” diverse interactions, and face-to-face collaboration. In: Proc. of ITS 2009, pp. 109–116. ACM, New York (2009)
Weiss, M., Schwarz, F., Jakubowski, S., Borchers, J.: Madgets: actuating widgets on interactive tabletops. In: Proc. of UIST 2010, pp. 293–302. ACM, New York (2010)
Wilson, A.: Depth-Sensing Video Cameras for 3D Tangible Tabletop Interaction. In: Proc. of TABLETOP 2007, pp. 201–204. IEEE, Los Alamitos (2007)
Wilson, A.D., Benko, H.: Combining multiple depth cameras and projectors for interactions on, above and between surfaces. In: Proc. of UIST 2010, pp. 273–282. ACM, New York (2010)
Wilson, A.D., Izadi, S., Hilliges, O., Garcia-Mendoza, A., Kirk, D.: Bringing physics to the surface. In: Proc. of UIST 2008, pp. 67–76. ACM, New York (2008)
Wilson, A.D.: TouchLight: an imaging touch screen and display for gesture-based interaction. In: Proc. of ICMI, pp. 69–76. ACM Press, New York (2004)
Wu, M., Balakrishnan, R.: Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays. In: Proc. of UIST 2003, pp. 193–202. ACM, New York (2003)
Yilmaz, A., Javed, O., Shah, M.: Object tracking: A survey. ACM Computing Surveys (CSUR) 38 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 IFIP International Federation for Information Processing
About this paper
Cite this paper
Marquardt, N., Jota, R., Greenberg, S., Jorge, J.A. (2011). The Continuous Interaction Space: Interaction Techniques Unifying Touch and Gesture on and above a Digital Surface. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds) Human-Computer Interaction – INTERACT 2011. INTERACT 2011. Lecture Notes in Computer Science, vol 6948. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23765-2_32
Download citation
DOI: https://doi.org/10.1007/978-3-642-23765-2_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23764-5
Online ISBN: 978-3-642-23765-2
eBook Packages: Computer ScienceComputer Science (R0)