Abstract
Large screens or projections in public and private settings have become part of our daily lives, as they enable the collaboration and presentation of information in many diverse ways. When discussing the shown information with other persons, we often point to a displayed object with our index finger or a laser pointer in order to talk about it. Although mobile phone-based interactions with remote screens have been investigated intensively in the last decade, none of them considered such direct pointing interactions for application in everyday tasks. In this paper, we present the concept and design space of PointerPhone which enables users to directly point at objects on a remote screen with their mobile phone and interact with them in a natural and seamless way. We detail the design space and distinguish three categories of interactions including low-level interactions using the mobile phone as a precise and fast pointing device, as well as an input and output device. We detail the category of widget-level interactions. Further, we demonstrate versatile high-level interaction techniques and show their application in a collaborative presentation scenario. Based on the results of a qualitative study, we provide design implications for application designs.
Chapter PDF
Similar content being viewed by others
References
Alt, F., Kubitza, T., Bial, D., Zaidan, F., Ortel, M., Zurmaar, B., Lewen, T., Shirazi, A.S., Schmidt, A.: Digifieds: insights into deploying digital public notice areas in the wild. In: MUM 2011, pp. 165–174. ACM, New York (2011)
Ballagas, R., Borchers, J., Rohs, M., Sheridan, J.: The smart phone: a ubiquitous input device. IEEE Pervasive Computing 5(1), 70–77 (2006)
Baur, D., Boring, S., Feiner, S.: Virtual projection: Exploring optical projection as a metaphor for multi-device interaction. In: CHI 2012. ACM, Austin (2012)
Boring, S., Altendorfer, M., Broll, G., Hilliges, O., Butz, A.: Shoot & Copy: Phonecam-based information transfer from public displays onto mobile phones. In: Mobility 2007, pp. 24–31. ACM, New York (2007)
Boring, S., Baur, D., Butz, A., Gustafson, S., Baudisch, P.: Touch projector: mobile interaction through video. In: CHI 2010, pp. 2287–2296. ACM, New York (2010)
Boring, S., Jurmu, M., Butz, A.: Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays. In: OZCHI 2009, pp. 161–168. ACM, New York (2009)
Cesar, P., Bulterman, D.C., Jansen, A.J.: Usages of the secondary screen in an interactive television environment: Control, enrich, share, and transfer television content. In: Tscheligi, M., Obrist, M., Lugmayr, A. (eds.) EuroITV 2008. LNCS, vol. 5066, pp. 168–177. Springer, Heidelberg (2008)
Chehimi, F., Rukzio, E.: Throw your photos: an intuitive approach for sharing between mobile phones and interactive tables. In: Ubicomp 2010 Adjunct, pp. 443–444. ACM, New York (2010)
Cruickshank, L., Tsekleves, E., Whitham, R., Hill, A., Kondo, K.: Making inter- active tv easier to use: Interface design for a second screen approach. The Design Journal 10(3), 41–53 (2007)
Dachselt, R., Buchholz, R.: Natural throw and tilt interaction between mobile phones and distant displays. In: CHI EA 2009, pp. 3253–3258. ACM, New York (2009)
Google Inc. Android developers (2013), http://developer.android.com/index.html (last access January 15, 2013)
Google Inc. LG TV remote 2011 - Android Apps on Google Play (2013), https://play.google.com/store/apps/details?id=com.clipcomm.WiFiRemocon&hl=en (last access January 15, 2013)
Jeon, S., Hwang, J., Kim, G.J., Billinghurst, M.: Interaction with large ubiquitous displays using camera-equipped mobile phones. Personal Ubiquitous Comput. 14(2), 83–94 (2010)
Microsoft. Welcome to Microsoft Pixelsense (2013), http://www.microsoft.com/en-us/pixelsense/default.aspx (last access January 15, 2013)
Morris, M.R.: Web on the wall: insights from a multimodal interaction elicitation study. In: ITS 2012, pp. 95–104. ACM, New York (2012)
Myers, B.A., Bhatnagar, R., Nichols, J., Peck, C.H., Kong, D., Miller, R., Long, A.C.: Interacting at a distance: measuring the performance of laser pointers and other devices. In: CHI 2002, pp. 33–40. ACM, New York (2002)
Myers, B.A., Stiel, H., Gargiulo, R.: Collaboration using multiple PDAs connected to a pc. In: CSCW 1998, pp. 285–294. ACM, New York (1998)
Nintendo. Wii u official site at Nintendo (2012), http://www.nintendo.com/wiiu (last access December 18, 2012)
Olsen Jr., D.R., Nielsen, T.: Laser pointer interaction. In: CHI 2001, pp. 17–22. ACM, New York (2001)
Pan, G., Ren, H., Hua, W., Zheng, Q., Li, S.: Easypointer: what you pointing at is what you get. In: CHI EA 2011, pp. 499–502. ACM, New York (2011)
Pawar, U.S., Pal, J., Gupta, R., Toyama, K.: Multiple mice for retention tasks in disadvantaged schools. In: CHI 2007, pp. 1581–1590. ACM, New York (2007)
Rashid, U., Kauko, J., Häkkilä, J., Quigley, A.: Proximal and distal selection of widgets: designing distributed ui for mobile interaction with large display. In: MobileHCI 2011, pp. 495–498. ACM, New York (2011)
Rekimoto, J., Saitoh, M.: Augmented surfaces: a spatially continuous work space for hybrid computing environments. In: CHI 1999, pp. 378–385. ACM, New York (1999)
Rukzio, E., Leichtenstern, K., Callaghan, V., Holleis, P., Schmidt, A., Chin, J.: An experimental comparison of physical mobile interaction techniques: touching, pointing and scanning. In: Dourish, P., Friday, A. (eds.) UbiComp 2006. LNCS, vol. 4206, pp. 87–104. Springer, Heidelberg (2006)
Schmidt, D., Molyneaux, D., Cao, X.: Picontrol: using a handheld projector for direct control of physical devices through visible light. In: UIST 2012, pp. 379–388. ACM, New York (2012)
Schmidt, D., Seifert, J., Rukzio, E., Gellersen, H.: A cross-device interaction style for mobiles and surfaces. In: DIS 2012, pp. 318–327. ACM, New York (2012)
Shizuki, B., Hisamatsu, T., Takahashi, S., Tanaka, J.: Laser pointer interaction techniques using peripheral areas of screens. In: Proceedings of the Working Conference on Advanced Visual Interfaces, AVI 2006, pp. 95–98. ACM, New York (2006)
Sugimoto, M., Hosoi, K., Hashizume, H.: Caretta: a system for supporting face- to-face collaboration by integrating personal and shared spaces. In: CHI 2004, pp. 41–48. ACM, New York (2004)
Tangram. Tangram - Smart Dot Pro (2013), http://www.tangramdesignlab.com/smartdot/en/ (last access January 15, 2013)
Tse, E., Histon, J., Scott, S.D., Greenberg, S.: Avoiding interference: how people use spatial separation and partitioning in SDG workspaces. In: CSCW 2004, pp. 252–261. ACM, New York (2004)
Vogt, F., Wong, J., Fels, S.S., Cavens, D.: Tracking multiple laser pointers for large screen interaction. In: Extended Abstracts of ACM UIST 2003, pp. 95–96 (2003)
Vogt, F., Wong, J., Po, B.A., Argue, R., Fels, S.S., Booth, K.S.: Exploring collaboration with group pointer interaction. In: CGI 2004, pp. 636–639 (June 2004)
Wilson, A.D., Shafer, S.: XWand: UI for intelligent spaces. In: CHI 2003, pp. 545–552 (2003)
Zizka, J., Olwal, A., Raskar, R.: Specklesense: fast, precise, low-costandcompact motion sensing using laser speckle. In: UIST 2011, pp. 489–498. ACM, New York (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Seifert, J., Bayer, A., Rukzio, E. (2013). PointerPhone: Using Mobile Phones for Direct Pointing Interactions with Remote Displays. In: Kotzé, P., Marsden, G., Lindgaard, G., Wesson, J., Winckler, M. (eds) Human-Computer Interaction – INTERACT 2013. INTERACT 2013. Lecture Notes in Computer Science, vol 8119. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40477-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-40477-1_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40476-4
Online ISBN: 978-3-642-40477-1
eBook Packages: Computer ScienceComputer Science (R0)