Castellucci, S. J., & MacKenzie, I. S. (2013). Touch me, hear me, feel me: Feedback preference for mobile text entry. Proceedings of the International Conference on Multimedia and Human-Computer Interaction - MHCI 2013, pp. 123.1-123.6. Ottawa, Canada: International ASET, Inc. [PDF]
Touch Me, Hear Me, Feel Me: Feedback Preferences for Mobile Text Entry
Steven J. Castellucci & I. Scott MacKenzie
Dept. of Computer Science and EngineeringYork University, Toronto, Canada
stevenc@cse.yorku.ca; mack@cse.yorku.ca
Abstract - Many mobile devices use touchscreens for text entry. They also provide vibration and audio feedback to simulate interaction with a physical keyboard. To facilitate design decisions of new mobile text entry methods, we conducted a survey and a user study to measure user's feedback preference and the effect of feedback modes on typing performance. We found that nearly half of respondents prefer no aural or haptic feedback. In our study, feedback mode had no statistically significant effect on entry speed or accuracy. However, comments made by users suggest that feedback preference is due to situational factors, rather than performance considerations.Keywords: Mobile text entry, soft keyboards, audio, haptic, feedback
1. Introduction
Mobile devices are ubiquitous in contemporary society. An estimated 9.2 trillion text messages will be sent worldwide in 2013 (Web-1) and more than 680 million users currently access Facebook from a mobile device (Web-2). Thus, investigating and developing methods for mobile text entry is a significant research topic.
Many mobile devices use touchscreens and soft keyboards instead of physical keyboards. This allows for a larger display without increasing the size of the device. Furthermore, soft keyboards change their layout based on user input and disappear when not needed. To compensate for the lack of tactile feedback provided by physical keys, soft keyboards can include aural and haptic feedback. The feedback takes the form of audible clicks from a speaker and device vibration, respectively.
The prevalence of easily programmable mobile touchscreen devices encouraged us to develop our own soft keyboards. However, it is important to investigate how feedback will affect users. This is especially true for a commercial product, where success depends on user acceptance. The use of feedback might annoy users, or cause decreased performance. We had three questions regarding aural and haptic feedback options during text entry:
- What feedback (or combination of feedback) do users prefer and why?
- Does the type of feedback affect users' performance?
- Does the type of feedback affect users' perception of performance?
We first summarize other research related to aural and haptic feedback during text entry. We then detail the survey and user study used to investigate our questions. Finally, we present the results and discuss their significance.
2. Related Work
Existing research has investigated the effect of haptic feedback on text entry performance. Some use vibration to indicate key presses, erroneous input, or to alert the user to input options. Koskinen et al. (2008) evaluated the effect of various forms of haptic feedback when entering numbers via a soft keypad. Although the effect was not statistically significant, participants found vibrations of 16 ms the most pleasant. However, the authors state that preferences are not necessarily generalizable and might vary between devices. This might explain why other studies evaluating haptic feedback yield conflicting results.
Dunlop and Taylor (2009) used a 75 ms vibration to indicate "helpful" word completions during text entry and a 150 ms vibration to signal entry of a non-dictionary word. The feedback significantly improved entry speed by 3 wpm.
McAdam and Brewster (2009) also found that haptic feedback significantly benefitted entry speed. A vibration of 30 ms signaled a correct key press, while 500 ms signaled a key slip. The vibrations were delivered to one of six locations on the participant, with the upper arm and wrist performing the best. They did not find any significant effect of vibration on accuracy.
Brewster et al. (2007) used a "smooth" vibration to indicate correct input and a "rough" one to signal errors. Both were 800 ms in duration. Though this gave participants a perceived increase in performance, the feedback had no significant effect on entry speed or total error rate. However, it significantly improved accuracy in the form of fewer uncorrected errors.
Hoggan et al. (2008) used 30 ms and 500 ms vibrations to signal correct input and errors, respectively. They found a significant effect on both speed and accuracy. Furthermore, Hoggan et al. (2009) used both audio and haptic feedback individually in noisy and moving environments and found they each improved speed and accuracy over the condition with no feedback. The effect of each mode depended on the environment. Haptic feedback improved performance in noisy environments, while audio was better in high vibration environments.
Mobile devices use less sophisticated haptic actuators than those used in the aforementioned research. This is perhaps to minimized size, weight, or cost. Thus, evaluating the effect of haptic feedback using an actual mobile device is valuable.
3. Method 1 (Survey)
We conducted a survey to poll mobile users on their feedback preferences when typing on touchscreen devices. To reach a large sample of users, we posted the following question on various online forums that cater to mobile technology:
Smartphones allow feedback when typing. This feedback could be audio (e.g., a "tick" sound from the speaker), vibration (i.e., the device shakes a little), or a combination of the two. What feedback do you prefer when typing (e.g., texting, emailing, etc.)?
Participants were able to select only one of the following responses: "Audio", "Vibration", "Audio and Vibration", or "None". They were also allowed to post comments elaborating on their choice. The poll results and the more informative comments are summarized in the Results and Discussion section.
Although the context of text entry was not specified, we believe that users resist changing their audio or haptic feedback settings based on their environment or situation – they "set it and forget it". This is often demonstrated by phones that ring during movies or lectures.
4. Method 2 (User Study)
In addition to the survey, we conducted a user study to determine the effect of a combination of feedback modes on mobile text entry performance.
4.1. Participants
Twelve participants (2 females, 10 males) with an age range of 20 to 31 years (mean = 26; SD = 4.1) entered text on a mobile phone. The number of participants is consistent with related studies (Brewster et al. 2007, Hoggan et al. 2008, Hoggan et al. 2009). All participants were fluent in English and frequently typed on a touchscreen device.
4.2. Apparatus
The phone used for the study was a Samsung Galaxy S Vibrant (GT I9000M),
running Android 2.3.3 (Fig. 1). The touchscreen measured 4.0 inches
diagonally and had a resolution of 480 × 800 pixels. The audio feedback was the
default key "click" sound, as defined by AudioManager in the Android API. The
phone's volume was set to provide feedback that was clearly audible, but not
intense. We created and analyzed an audio recording of the phone's haptic
feedback and measured the vibration to be about 80 ms in duration.
For text entry, the default QWERTY keyboard was used with auto-spacing and auto-correction options disabled. The Text Entry Metrics for Android (TEMA) (Castellucci and MacKenzie 2011) application was used to administer transcription phrases from (MacKenzie and Soukoreff 2003), record participant input, and calculate text entry metrics.
4.3. Procedure
Participants entered 30 phrases in each condition. However, the first 5 phrases served as a warm-up and were not included in the analysis. To eliminate variability in the task, all participants held the phone in a portrait orientation and entered text using their thumbs (Fig. 1). Furthermore, they were instructed to enter text as quickly as possible, to correct errors if noticed immediately, but to ignore errors initially missed (i.e., to prevent deletion of many correct characters to correct an early mistake).
Fig. 1. Participants' hand position during the user study.
Study sessions typically lasted 30 minutes and took place in a quiet office, with participants seated at a desk. Participants also completed a questionnaire to elicit their text entry preferences and to gather demographic information.
4.4. Design
The study employed a within-subjects factor, feedback mode, with four levels: Audio, Vibration, Both (audio and vibration), and None. The order of testing was counterbalanced using a balanced Latin Square. Each participant entered 30 phrases (5 warm-up, 25 experimental) in each condition, which is consistent with previous text entry research (Hoggan et al. 2008, McAdam and Brewster 2009). Our analysis was based on the resulting 1200 (12 × 25 × 4) trials.
The dependent variables were entry speed and accuracy, as calculated by TEMA. Entry speed was reported in words-per-minute (wpm) using a word length of five characters (including spaces). In addition, accuracy was measured according to the total error rate (TER), corrected error rate (CER), and uncorrected error rate (UER) metrics (Soukoreff and MacKenzie 2003).
5. Results and Discussion
5.1. Survey Results
The results of the survey appear in Fig. 2 and include responses from participants in the user study. A total of 92 people cast a vote indicating their preferred feedback mode when typing on a mobile touchscreen device. While just over one third of respondents opt for only haptic feedback, almost half prefer no aural or haptic feedback at all. The margin of error is 9.6% with a confidence level of 95%.
Fig. 2. Survey participants' feedback preference when typing on a mobile touchscreen device (n = 92).
5.2. Entry Speed and Accuracy
Entry speeds for the Audio and None conditions were identical at 29.9 wpm, with the Both condition being slightly higher at 30.3 wpm and Vibration being slightly lower at 28.7 wpm (Fig. 3). Dunlop and Taylor (2009) used a 12-key phone keypad for input and recorded a speed of 23 wpm when using vibration. However, McAdam and Brewster (2009) and Hoggan et al. (2009) both used touchscreen keyboards and reported speeds of approximately 30 wpm, consistent with our results. Unfortunately, the other studies measured entry speed in "time to enter phrases" or "number of lines entered", thus preventing direct comparisons.
Fig. 3. Entry speed values gathered from our user study. Error bars represent ±1 SD.
The difference in entry speed between the four conditions was not statistically significant (F3,24 = 1.25, p > .05). This is consistent with the findings of Brewster et al. (2007), but differs from the findings of Dunlop and Taylor (2009) and McAdam and Brewster (2009). In addition, the ANOVA indicates that counterbalancing worked, as the order of the conditions was not significant (F3,8 = 1.53, p > .05).
Accuracy results appear in Fig. 4. The None condition was the most accurate, as it yielded the lowest CER (7.0%) and TER (9.7%), respectively. Participants committed (and corrected) more errors in the conditions that provided feedback. Evidently haptic feedback motivated participants to correct their errors. The Vibration condition had the highest CER (8.1%) and the lowest UER (2.1%). Surprisingly, the combination of haptic and aural feedback resulted in the highest UER (3.3%) and TER (10.7%). Participants committed the most errors in the Both condition and did not correct them. Unfortunately, the effect of feedback was not statistically significant for TER (F3,24 = 0.69, ns), CER (F3,24 = 0.94, ns), or UER (F3,24 = 1.15, p > .05). As with entry speed, the group effect on accuracy was not significant (F3,8 = 3.83, p > .05).
Unfortunately, the use of different accuracy metrics in related studies prevents accurate comparison with our results. One study (Brewster et al. 2007) measured accuracy as "total errors" and "number of errors uncorrected", suggesting analogs to TER and UER metrics, respectively. However, the accuracy measurements appear on the same chart as entry speed, with an "average score" on the y-axis rather than the expected error rate.
Fig. 4. Accuracy values gathered from our user study. Error bars represent ±1 SD of TER.
Other studies reported the number of phrases entered correctly. Unfortunately, this metric does not convey how many errors appeared in incorrect phrases, nor the number of errors corrected during input. McAdam and Brewster (2009) reported 75% to 80% of phrases were entered correctly, with vibration having no statistically significant effect on accuracy. In comparison Hoggan et al. (2008, 2009) reported accuracy rates from 55% to 90% and found that feedback had a significant effect on accuracy; vibration improved accuracy in noisy environments, but audio was better in high vibration environments.
5.3. Users' Perception of Performance
After the study sessions, participants were asked to select the feedback mode they felt resulted in the fastest typing and which resulted in the most accurate typing. This was to investigate whether or not feedback mode had any effect on perceived performance.
The majority of participants' selections were evenly split between the Audio and None conditions for both speed and accuracy. However, the results show that most participants typed fastest in the None condition, but typed most accurately in the Vibration condition. Half the participants correctly identified the fastest feedback mode, while only a quarter of participants correctly identified the most accurate feedback mode. Thus, there was a significant disconnect between actual and perceived performance.
5.4. Users' Preferences
What contributes to users' preference for one feedback mode over another? Comments received by our survey respondents provide insight and are summarized below. (Respondents' usernames appear in parentheses.) As university instructors, we understand that some mobile phone users mute their devices in an attempt to hide text entry activities during lecture. However, survey comments suggest that, in social settings, an ethic of reciprocity might also influence the preference for no audio feedback. Some users are bothered by other people's noisy devices. Thus, they choose to disable audio feedback on their own device to not disturb people around them.
"Audio feedback annoys me a little when using it, and it annoys me A LOT when the person next to me is using it!" (Big Ang)"i [sic] prefer silence, no audio, no vibration, because audio will influence other people, while vibration will make me uncomfortable." (jean2012)
Seven respondents stated specifically that the sound clips used for audio feedback in mobile devices are "annoying". Others commented that the audio feedback seems unnatural.
"I can't stand that fakey clicking sound." (synaesthetic)"The audio feedback is often annoying; this isn't the nineteenth century anymore [in reference to typewriters?], and often the noises devices choose are silly." (primetechv2)
While most respondents dislike aural feedback, many appreciate having haptic feedback to indicate that input to the mobile device was received.
"I activate the haptic feedback, because it give [sic] me a sense that the phone is really typing." (Felimenta97)
Finally, one respondent turns off audio and vibration feedback in an effort to conserve battery power.
"I prefer no audible or haptic feedback what so ever. To me, they are pointless and help eat battery life that I can be better used for programs I use." (moonzbabysh)
6. Conclusion
In our user study, feedback mode had no significant effect on typing speed or accuracy. Some related studies conclude that feedback significantly effects speed, but not accuracy, while other studies that use similar feedback conclude the opposite. These results highlight the disagreement on the effect of feedback on performance and raise the question, "Why do users prefer one feedback over another?"
To that end, our paper also provides insight into mobile users' feedback preferences. Almost half of users we surveyed prefer no aural or haptic feedback during text entry. Thus, the cost to create, evaluate, and deploy new feedback techniques might outweigh the benefit if few users adopt it. Survey comments indicated that other issues, such as power consumption and social etiquette, also influence preferences. To cater to user preferences, soft keyboard developers should provide at least simple haptic feedback as an option when typing, but not with the expectation that it will necessarily affect text entry performance.
References
Brewster, S., Chohan, F. and Brown, L. (2007). Tactile feedback for mobile interactions "Proc. CHI 2007," San Jose, United States, April 28-May 3, pp. 159-162.
Castellucci, S. J. and MacKenzie, I. S. (2011). Gathering text entry metrics on Android devices "Ext. Abs. CHI 2011," Vancouver, Canada, pp. 1507-1512.
Dunlop, M. D. and Taylor, F. (2009). Tactile feedback for predictive text entry "Proc. CHI 2009," Boston, United States, April 4-9, pp. 2257-2260.
Hoggan, E., Brewster, S. and Johnston, J. (2008). Investigating the effectiveness of tactile feedback for mobile touchscreens "Proc. CHI 2008," Florence, Italy, April 5-10, pp. 1573-1582.
Hoggan, E., Crossan, A., Brewster, S. and Kaaresoja, T. (2009). Audio or tactile feedback: Which modality when? "Proc. CHI 2009," Boston, United States, April 4-9, pp. 2253-2256.
Koskinen, E., Kaaresoja, T. and Laitinen, P. (2008). Feel-good touch: Finding the most pleasant tactile feedback for a mobile touch screen button "Proc. ICMI 2008," Crete, Greece, October 20-22, pp. 297-304.
MacKenzie, I. S. and Soukoreff, R. W. (2003). Phrase sets for evaluating text entry techniques "Ext. Abs. CHI 2003," Ft. Lauderdale, United States, pp. 754-755.
McAdam, C. and Brewster, S. (2009). Distal tactile feedback for text entry on tabletop computers "Proc. BCS-HCI 2009," Cambridge, UK, September 1-5, pp. 504-511.
Soukoreff, R. W. and MacKenzie, I. S. (2003). Metrics for text entry research: An evaluation of MSD and KSPC, and a new unified error metric "Proc. CHI 2003," Ft. Lauderdale, United States, pp. 113-120.
-----
Web sites:
Web-1: http://www.portioresearch.com/blog/2012/12/happy-birthday-sms!.aspx, consulted 20 Feb. 2013.
Web-2: http://newsroom.fb.com/content/default.aspx?NewsAreaId=22, consulted 20 Feb. 2013.