L. Kim et al.

: A Universal Remote Control with Haptic Interface for Customer Electronic Devices

913

A Universal Remote Control with Haptic Interface for Customer Electronic Devices
Laehyun Kim, Member, IEEE, Wanjoo Park, Hyunchul Cho and Sehyung Park

Abstract We introduce a new universal remote control that gives easy-to-control interface for home devices such as TV, video/audio player, room lighting and temperature control. In order to use conventional remote controls, people need to understand complex instruction manuals and remember functions assigned to buttons. In addition, the button-based control does not provide an intuitive interface so the user presses a button several times to browse information and has difficulty in searching the right button among many buttons. Our universal remote control addresses these limitations by using a touch screen, a force-feedback dial knob, and two buttons instead of many buttons. We suggest an example scenario to interact with a conventional TV set, room lighting, and air conditioner using our universal remote control. The result of a user study to evaluate the usability of the device shows that the universal remote control is very efficient and intuitive interface to control customer electronics devices1. Index Terms Universal Remote Control, Haptic Interface, Home device.

The touch screen displays visual information about the home appliance selected by touching the menu on the screen. The force-feedback dial knob is used to browse and select items by rotating the knob. It generates various tactile/haptic effects such as spring, friction, detent, hard stop, and combination of these effects depending on the situation. Two buttons can be used for selecting power on/off and returning the main (home) menu respectively.

Fig. 1. Concept of the universal remote control. It consists of a LCD touch screen, two buttons, and a haptic dial.

I. INTRODUCTION Usually home appliances such as TV, radio, video/audio players, home theater, air conditioner, and room lighting come with remote controls that are used to select items and change status. As a result, people keep many remote controls to interact with their home devices. In addition, each control has different look and feel and functions assigned to buttons. This configuration gives the user difficulty in browsing information and finding a correct button. Sometimes the user needs to press a button repeatedly to set the volume and TV channel. In order to address theses limitations, we developed a universal remote control that provides easy-to-use control and simple interface to control home appliances (see Fig. 1). The remote control we have developed consists of a touch screen, a force-feedback dial knob, and two buttons instead of many buttons.
This work has been supported by Tangible Web Technology Project in Korea Institute of Science and Technology Laehyun Kim is with Korea Institute of Science and Technology as a senior researcher, Seoul, Korea (e-mail: [email protected]) Wanjoo Park is with Korea Institute of Science and Technology as a researcher, Seoul, Korea (e-mail: [email protected]) Hyunchul Cho is with Korea Institute of Science and Technology as a researcher, Seoul, Korea (e-mail: [email protected]) Sehyung Park is with Korea Institute of Science and Technology as a principal researcher, Seoul, Korea (e-mail: [email protected]) Manuscript received February 17, 2010 Current version published 06 29 2010; Electronic version published 07 06 2010.
1

II. PREVIOUS WORK Recently haptic interfaces have been developed that allow users to interact with digital information via the sense of touch. Haptic technology has various customer electronics such as mobile phones, touch screen, automobiles, and games. Over the past several years, there have been a number of studies of haptic feedback controls. Karon E. MacLean et al. introduced a variety of haptic devices and design parameters [1-3]. They considered characteristics of the touch sense and designed various tactile signals on their experimental devices. El Saddik A. suggested an identity authentication method using a haptic device [4]. He extracted behavioral features when users controlled the device and used the features to identify them. Several studies have been conducted of the 1 DOF (Degree of Freedom) dial knob. Scott S. Snibbe et al. suggested several haptic techniques for manipulating digital media based on intuitive physical metaphors [5]. Mircea Badescu et al. developed a single force feedback knob to imitate the senses of various conventional control knobs in motor vehicles [6]. Laehyun Kim et al. used a haptic dial system for multimodal prototyping in the early phase of product development [7]. They applied the haptic technology to prototype the dial module of a washing

0098 3063/10/$20.00 2010 IEEE

914

IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010

machine. There are several works to apply the haptic interface to games. Yuichiro Sekiguchi applied the haptic effect to a game, proposing a device that gives a user the illusion of a virtual object inside the device when shaking it using accelerators and actuators [8]. Wanjoo Park et al. proposed a brickout game using a haptic dial interface and some haptic effects for the game [9]. Recently a remote control manufacture announced a universal remote control with tactile feedback and touch screen. When the user touches the button, the remote generate simple vibration. III. IMPLEMENTATION OF THE CONTROL In this section, we explain in detail the hardware configuration of the haptic dial knob. A block diagram of the system is shown in Fig. 2.
Fig. 3. System architecture of the haptic dial interface.

IV. GRAPHICAL USER INTERFACE (GUI) GUI for the universal remote control consists of main (home) menu, TV channels, room temperature, and room lighting. The main menu has three icons that represent three sub menus to control home appliances. The user can select one of them by touching an icon. He/she can go back to the main menu by pressing the right button (we called it home button) anytime and select other home devices (Fig. 4(a)).

(a) Motor, Control unit PCB and Touch screen module

(a) Main menu

(b) TV channel

(b) Haptic remote control Fig. 2. Implementation of the universal haptic remote control.

MCU (Micro Controller Unit) computes the torque amount at a given angular position and sends the command to a DC motor via a DAC (Digital to Analog Convertor). The DC motor executes the command and generates various haptic patterns. We use a gear box of 5:1 ratio to change the angle of rotation axis. The dial knob is installed on the motor gear box, allowing the user to rotate the knob and to feel various haptic effects which are programmed along the angular position. An encoder measures the angular position with 1000 pulse per turn and an encoder counter has a 24bits quadrature counter and 25MHz count frequency. A Touch LCD Module is used to display visual information and to allow touch input.

(c) Room temperature

(d) Room lighting

Fig. 4. Graphical user interface for the universal remote control. Main menu (a), TV channel (b), room temperature (c) and room lighting (d)

TV channel menu shows the channel list and information about current selected channel. The channel numbers are displayed on the rounded band to go well with rotational action of the dial knob. Some numbers has the heart mark that indicates popular channels or favorite channels programmed

L. Kim et al.: A Universal Remote Control with Haptic Interface for Customer Electronic Devices

915

by the user. When the user selects the channel with the heart mark, he/she feels different tactile feedback. It helps the user to find intuitively the channel that he/she may want to see (Fig. 4(b)). In room temperature menu, the user can select cooling or heating function by touching the screen and control the room temperature by spinning the dial knob. Yellow point indicates the current temperature the user select. The user spins the dial easily in the proper temperature range. But he/she feels strong tactile feedback out of the range (Fig. 4(c)). Room lighting menu shows a light bulb to visualize room lighting. As the user rotates the dial knob, the brightness of the bulb is changed to indicate the room lighting level. The user feels the increasing/decreasing friction when he/she spins the dial clockwise/counterclockwise. It gives very institutive interface visually and haptically (Fig. 4(d)). The red point indicates the current brightness level on the bulb.

The torque profile for the detent effect is shown in Fig. 5. By modulating A and b, the amount and frequency of torque can be changed. We set A to a bigger value for the popular or favorite channels having a heart mark in Fig. 4(b) so that the user can identify the channels intuitively. For instance, channel 7 is a favorite channel and channel 5, 6, and 8 are non popular channels (Fig. 5). B. Room lighting control Friction haptic effect is used to adjust the room brightness level by rotating the dial knob. The friction effects generate resistant torque opposite to the direction of movement as a movement-based effect. This is implemented based on the friction con model [10]. The friction torque can be calculated by eq. (2).

Lf
V. HAPTIC EFFECTS In this section, we describe how to implement various haptic effects for the universal remote control. We designed various haptic effects which are defined by adjusting torque profiles along the angular position and time. Haptic effects include detent, friction, hard stop, and a combination of these effects. These haptic effects help the user to browse and find information easily and intuitively. A. TV channel control In order to give a feeling to select a channel to the user, we use the detent effect. The detent effect simulates notches with different height along the angular position using sine functions. Feeling a notch confirms each channel selection. Eq. (1) shows the detent effect which is defined by a sine function.

exp(Pnow (n) / S f 1 )

Pcurr _ f (n) Pprev (n  1)  ( Pnow (n)  Ppre (n  1)) S f 2 Pdiff (n) ( Pnow (n)  Pcurr _ f (n)) L f Pprev (n) Pcurr _ f (n) if Pdiff (n) ! Tf _ max then Tf (n) Tf _ max Pdiff (n)  Tf _ min then Tf (n) Tf _ min else if
(2)

else Tf (n) Pdiff (n)

where Pcurr_f is the current position, Pnow is the angular position of the dial knob, Ppre is the previous position, Sf1 and Sf2 are the scaling factors, Pdiff is the difference of position, Lf is the friction level, Tf is the friction torque. Lf (friction level) increases exponentially as the brightness level increases since the exponential increasing make much better feeling than the linear motion (Fig. 6). We designed the friction profile depending on the dial knobs movement. When the dial knob is rotated slowly, the rotational friction is low. On the other hand, when the user spins the dial fast, he/she feels more string friction. Fig. 7 shows the profile of the friction according to the knob movement.
30 Haptic Effect - Dimming Interaction

f d (T )

AT sin(b T )

(1)

where, A is the amplitude, b is the number of notch per turn, is the rotation angle of the dial knob.
2 1.5 1 0.5 0 -0.5 -1 -1.5 -2 Haptic Effect - Channel Interaction

25

20 Friction Level

Torque

15

10

5.5

6.5 7 Channel

7.5

8.5

50

100

150

200 Angle

250

300

350

400

Fig. 5 Haptic profile by channel changing

Fig. 6 Friction level by angle position

916
Haptic Effect - Dimming Interaction 3.5

IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010

3 level level level level level 0.1 1 5 15 25

where, T is temperature, Sb is scaling factor, AT is the amplitude, b is the number of notch per turn, is the rotation angle of the dial knob. VI. APPLY TO GAME CONTROLLER We applied the universal remote controller to a game controller that gives the user haptic feedback during the game. For this, we developed a new brickout game in which conventional bricks are replaced by banana bricks, a game ball is replaced by a monkey, and a paddle is replaced by people holding up boards. In addition, a new item is added, a cloth wrapper.
60 70

2.5

Torque

1.5

0.5

0 0

10

20

30 40 Movement of Dial

50

Fig. 7 Torque profile comparison of levels

The friction effect help the user feel the brightness intuitively along with the visual effect. C. Room temperature control In order to give a haptic feedback for room temperature control, we used a combination of detent, friction, and hard stop effect. This effect is designed to guide the user to feel the proper temperature range with tactile feedback. The proper temperature for cooling ranges from 23 to 26 centigrade and one for heating ranges from 19 to 22 centigrade. In the proper temperature range, the user can feel only detent effect and adjust the temperature easily. But if the user tries to set the temperature out of the proper range, he/she feels the additional friction force proportionally to distance from the range. In the temperature range lower 16 and above 30 centigrade, the user feels very strong resistance force (hard stop) in Eq. (3).
8 6 4 Torque / Friction Level 2 0 -2 -4 -6 -8 14 Torque profile Friction level of Cooling Friction level of Heating 16 18 20 22 24 Temperature 26 28 30 32 Haptic Effect - Cooling/Heating Interaction

Bricks

Ball

Paddle

Fig. 9 Development of a new brickout game

Fig. 10 Playing a brickout game using the universal remote control

The banana bricks work the same as general bricks and the cloth wrapper contains haptic items. The game ball bounces off the top or side of the wall depending on the collision conditions between the ball and the paddle. When the ball hits a cloth wrapper, the haptic item falls down. If the user catches the item by moving the paddle, a predefined haptic effect is felt that lasts a few seconds. This haptic feedback can make the game easier or more difficult. VII. USER STUDY We performed a user study to evaluate usability of the universal remote control. A. Usability factors For the user study, we followed a guideline suggested by Jinwoo Kim [11] who defined usability factors with a hierarchical structure. We choose appropriate factors to measure the usability of the universal remote control (Fig. 11).

Fig. 8 Haptic profile and friction level by temperature

Fig. 8 shows the torque profile with regards to the temperature.

fT (T ) (T  16) 2 / Sb , (T  16) AT sin(b T ), (16 d T d 30) (T  30) 2 / Sb , (30 d T )

(3)

L. Kim et al.: A Universal Remote Control with Haptic Interface for Customer Electronic Devices
Basic domain Efficiency Responsiveness Minimal action Accuracy Error prevention Error recovery Meaningfulness Consistency Understandability Predictability Familiarity

917

4. TV Channel control: Now, TV shows some advertises. You change the TV channel to number 17 for searching another program.

Concomitant domain

Fig. 11 Evaluation factors of usability suggested by Jinwoo Kim.

Evaluation factors in the first level consist of basic and concomitant domains. In the second level, the basic domain has two factors; efficiency and accuracy factor. Efficiency factor is about how effectively users can achieve given tasks in terms of responsiveness and minimal action. Responsiveness is a factor to measure the response time and minimal action is about how simply users can deal with given tasks. Accuracy factor is about users mistakes while using the control and has error prevention and error recovery. Error prevention is about how well the remote control prevent users mistake and error recovery is about how easily users can correct the mistakes. Similarly, the concomitant domain has meaningfulness and consistency factors. Meaningfulness is a factor to measure how well the remote control provides information and functions which users want to use. Meaningfulness has understandability that is about how well users can understand system status via given information. Consistency is to measure how similar a function of the remote control is to other conventional ones. Consistency has predictability and familiarity as detailed factors. Predictability is about how well users can predict instructions of the remote control through experience of using other similar ones. Familiarity is about how easily users can know instructions of a system through experience of real world. B. Method Subjects are 20-30s thirty people who are familiar with the conventional remote controls. Before starting the survey, we gave information about the universal remote control and allow subjects to use it freely for 5 minutes. Subjects did not know the task scenario and are asked to do tasks one by one using the universal remote control. These tasks are below:
1. Temperature control: You came back home and it is too cold, 5 C outside. You are sitting down a sofa and control indoor temperature to 28C. 2. TV Channel control: Now, you want to watch TV. The default channel is number 2 and it is boring to you so you change the TV channel to your favorite channel, number 11. 3. Temperature control: you feel it is too warm. You control indoor temperature to proper temperature in winter season, 24C.

After completing the tasks, subjects were asked to answer a questionnaire. The questionnaire for the user study is divided into three sections such as general control (touch screen), TV channel control and temperature control. The light control section is excluded because it is not popular way to control by a remote control. There are five questions excepting responsiveness and recovery factor in general section. For channel and temperature section, there are eleven and ten questions each section with seven factors. Questions on the questionnaire were 7-point (-3 to 3) Likert scales. C. Result Fig. 12 shows results for each factor. For Efficiency dimension, both of Responsiveness and Minimal Action got the high scores (2.50 and 2.08 respectively). It means that the user can use the universal remote control effectively. The subjects said that the haptic dial knob is very intuitive interface to control TV channel and temperature. The touch screen also provides easy-to-use interface to select menus. For Accuracy factor, both of Error prevention and Error recovery got the medium scores. Error prevention factor got medium score (1.39). The score for Error prevention of General (main screen) was not high (0.86) because if it is not correct position on touch screen, the system operates another program. In the case of Error Recovery factor, temperature control got high score (2.27). As the results of interview, subjects recognized it is useful that friction is increased as deviate degree from proper temperature.

Fig. 12 The results of user-study

For Meaningfulness factor, Understandability got high score (2.09) and especially for the general section got the highest score (2.68) because touch screen display shows various information according to context such as current temperature and channel information. In case of Consistency factor, Predictability got high score (2.42). It means the universal remote control has very

918

IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010

Mircea Badescu, Charles Wampler and Constantinos Mavroidis, "Rotary Haptic Knob for Vehicular Instrument Controls", Haptics'02, Haptic Interfaces For Virtual Envir & Teleoperator Systs, pp. 342-343, 2002 [7] Laehyun Kim, Manchul Han, Sang Kyun Shin, Se Hyung Park, "A Haptic Dial System for Multimodal Prototyping", 18th International Conference on Artificial Reality and Telexistence (ICAT 2008), 2008 [8] Yuichiro Sekiguchi, Koichi Hirota and Michitaka Hirose, The Design and Implementation of Ubiquitous Haptic Device, Eurohaptics Conference, 2005 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2005. World Haptics 2005. First Joint , pp. 527 528, 2005 [9] Wanjoo Park, Laehyun Kim, Hyunchul Cho and Sehung Park, Design of Haptic Interface for Brickout Game, IEEE International Workshop on Haptic Audio Visual Environments and Games (HAVE 2009), pp. 6468, 2009 [10] N. Melder and W. S. Harwin, Extending the Friction Cone Algorithm for Arbitrary Polygon Based Haptic Objects, Proceedings of the 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS04), pp 234 241, 2004 [11] Jinwoo Kim, et al, Introduce to Human Computer Interaction, Ahn Graphics Press: 2005, pp. 195-235 [6]

simple and stable interface. However Familiarity has very low score (-0.32) because our universal remote control has a dial knob with haptic feedback and a touch screen unlike conventional remote controls. VIII. CONCLUSION Today people need to keep several remote controls to control home appliances in their living room. Universal remote control allows people to control multiple devices using a single remote control. However conventional universal remote controls still have problems. Many buttons make the user spend time to browse and find the right one. In many case, people need to press a button several times, for instance volume control and TV channel selection. In addition, the user needs to read and remember the manual before he/she use it. In this paper, we introduce a new universal remote control to address these limitations. It has a simple interface which consists of a touch screen, two buttons, and a dial knob with haptic feedback. Touch screen can display various information and be used an input device to select home device the user want to control. Two buttons is used for the power on/off and returning to the home menu. Haptic dial knob provides a very intuitive input and output interface. The user can change the status by rotating the dial knob and at the same time, he/she receives tactile feedbacks depending on the situation. Our universal remote control allows the user to control TV channel, room temperature, and room light brightness. In addition, we applied the device to a game controller for a brickout game. It makes the game more fun and more immersive. The result of a user study to evaluate the usability shows that our universal remote control is very efficient and intuitive device to control home appliances. In the future we will design new haptic effects to control other devices and make the universal remote control smaller and lighter for better usability. REFERENCES
[1] Karon E. MacLean, "Designing with Haptic Feedback" Symposium on Haptic Feedback in the Proc. of IEEE Robotics and Automation (ICRA 2000), pp.22-28, 2000 Vincent Hayward and Karon E. Maclean, Do It Yourself Haptics: Part 1, IEEE Robotics & Automation Magazine December 2007, pp. 88-104, 2007 Karon E. MacLean, Foundations of Transparency in Tactile Information Design, IEEE Transaction on Haptics, Vol. 1, No. 2, pp. 84-95, 2008 El Saddik A., et al., A Novel Biometric System for Identification and Verification of Haptic Users, IEEE Transactions On Instrumentation and Measurement, Vol. 56, No. 3, pp.895-906 Scott S. Snibbe, et al., Haptic Techniques for Media Control, Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology (UIST 2001) , pp.199-208, 2001

BIOGRAPHIES Laehyun Kim (M09) is a senior research scientist in Intelligent Systems Technology Division at Korea Institute of Science and Technology. His research interests include haptics, computer graphics, and virtual reality. Kim Laehyun received his BS from Hanyang University in Material Engineering, an MS from Yonsei University in Computer Science, and a PhD in Computer Science from the University of Southern California. Wanjoo Park received his B.S. in electronics engineering from the Korea Aerospace University, Korea, in 2006, and his M.S. in electronics engineering from Sogang University, Korea, in 2008. Since 2008, he has been a researcher with the Intelligence & Interaction Research Center of the Korea Institute of Science and Technology. His research interests include machine learning, pattern recognition, haptics, and HCI. Hyunchul Cho is a research scientist in the Intelligence and Interaction Research Center at Korea Institute of Science and Technology. He received his B.S. in Computer Science and Engineering from Korea University, Seoul, Korea in 2003, and an M.S. in Computer Science and Engineering from Korea University, Seoul, Korea in 2005.

[2]

[3]

[4]

[5]

Sehyung Park is a principal researcher in Intelligent Systems Technology Division at Korea Institute of Science Technology. His research interests include the study of geometric modeling, the human-computer interface, reverse engineering, and NC programming. He received his BS from Seoul National University in Mechanical Design and Production Engineering, an MS from Cornell Unversity in Mechanical Engineering, and a PhD in Mechnical Engineering from Korea Advanced Institute of Science and Technology.