LevioPole
We present LevioPole, a rod-like device that provides mid-air haptic feedback for full-body interaction in virtual reality, augmented reality, or other daily activities. The device is constructed from two rotor units, which are designed using propellers, motors, speed controllers, batteries, and sensors, allowing portability and ease of use. Having each group of rotor units on both ends of the pole, these rotors generate both rotational and linear forces that can be driven according to the target application. In this paper, we introduce example applications in both VR and physical environment; embodied gaming with haptic feedback and walking navigation in a specific direction.
INTRODUCTION
We, humans, interact with the real world via our own body and tools.The development of tools change our behavior and the relationships between the world and us. Especially, in human-computer interaction, because a characteristic of the devices allows inducing human actions, which kind of tool we use is important following contents and media. As shown in figure 1(a), human activity would be able to be augmented with the progress of technologies in future.
Recently developed technologies for devices, such as Microsoft Kinect or Nintendo Switch, allow full-body interactions in gaming. These technologies can be applied to various kind of contents, such as virtual reality (VR) or augmented reality (AR). The intensity of force feedback or haptic sensation is as important as audio-visual feedback in full-body interaction. Each factor is limited by the size of the devices, mechanism and use situations. To provide strong mid- air haptic interaction, it is effective to use a grounded device which is mounted on the base, wall, or ceiling. However, the grounded device limit body motion by the base of them.
Attempting on improvement, researches on integrating haptic sensation or tangible UI on mid-air devices are being conducted. Thus, naturally there are many aspects to explore in this field. In case of haptic, a pseudo tactile sensation, which is generated by a wearable device is sometimes used [Katsumoto 2017; Murakami et al. 2017].
As different approaches to realize mid-air physical interaction, there are some trials using technologies related to an unmanned aerial vehicle (UAV). HoverBall [Nitta et al. 2014] is an interactive ball containing a UAV inside. It can hover, and users can physi- cally interact with it during sports or gameplay. By introducing a metaphor of a ball, UAV can provide users with an experience different from levitating itself. HapticDrone [Abdullah et al. 2017] provides mid-air haptic feedback in 1D direction using UAV. It can generate controlled force rendering such as stiffness or weight sensation of a virtual object.
Our proposal, LevioPole, is a novel device that is able to provide mid-air haptic feedback during full-body interaction. Overview of LevioPole is a human-scale, rod-like device, which can be used in VR/AR contents to augment human motion as shown in figure 1(a). There are some advantages of LevioPole, being a rod-like device and having multirotors on the tip of each side; 1) users can easily understand how to handle the device by a metaphor of stick as a tangible interface, 2) interaction between user and device is safe because of a physical boundary, which separates the part which the user touches with mounted rotors, 3) various haptic patterns can be generated by driving force from multirotors.
LevioPole can be applied to various situations of VR, AR, or other daily activities. As shown in figure 1(b), the device can be used in VR games. By handling the device as a part of a tool, a player can interact with virtual objects and feel haptic sensation by collisions. As another situation, the device can be used in the physical world such as guiding a user to walk in a specific direction as shown in figure 1(c).
SYSTEM DESCRIPTION
The hardware of the device is constructed from a rod and two quadrotor units which are mounted equally distant on each end of the rod. As shown in figure 2, electronic components of the rotor unit are similar to general components of a UAV; brushless motor, electronic speed controller (ESC), propeller, battery, IMU sensor, distance sensor, touch sensor, and microcontroller. As designed above, it allows portability and ease-of-use as a gaming controller or tool.
These rotors generate both rotational and linear forces through independently controlled motors. As shown in figure 3, the roll and pitch rotational forces, and the throttle linear force are provided by resultant forces from each thrust of propellers. Additionally, some characteristic haptic pattern can be provided by controlling the timing and the intensity of motors. For example, the dynamics of water or the bumps of solid objects can be generated according to VR contents (Figure 1(b)).
The device can connect to PC via serial communication for send- ing the data of device conditions and receiving commands to gen- erate haptic patterns. It can also be used in VR environment using head-mounted display (HMD) and motion tracking system.
Figure 2: electronic components of a rotor unit
USER EXPERIENCE
At SIGGRAPH 2018 Emerging Technology exhibition we will demon- strate three type of VR experience; 1) extreme sports gaming on the river using paddles, 2) high jumping actions with flying/falling, 3) fitness application such as weightlifting. By integrating LevioPole with a HMD and a motion tracker, attendees can experience full- body mid-air interactions and feel various force feedback or haptic sensation triggered by VR contents.
Figure 3: generated forces driven by thrust of rotors
REFERENCES
Muhammad Abdullah, Minji Kim, Waseem Hassan, Yoshihiro Kuroda, and Seokhee Jeon. 2017. HapticDrone: An Encountered-Type Kinesthetic Haptic Interface with Controllable Force Feedback: Initial Example for 1D Haptic Feedback. In Adjunct Publication of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST ’17). ACM, New York, NY, USA.https://doi.org/10.1145/3131785. 3131821Yuichiro Katsumoto. 2017. Bottomless Joystick 2. In ACM SIGGRAPH 2017 Emerging Technologies (SIGGRAPH ’17). ACM, New York, NY, USA.https://doi.org/10.1145/ 3084822.3084826Takaki Murakami, Tanner Person, Charith Lasantha Fernando, and Kouta Minamizawa. 2017. Altered Touch: Miniature Haptic Display with Force, Thermal and Tactile Feed- back for Augmented Haptics. In ACM SIGGRAPH 2017 Emerging Technologies (SIG- GRAPH ’17). ACM, New York, NY, USA.https://doi.org/10.1145/3084822.3084836Kei Nitta, Keita Higuchi, and Jun Rekimoto. 2014. HoverBall: Augmented Sports with a Flying Ball. In Proceedings of the 5th Augmented Human International Conference (AH ’14). ACM, New York, NY, USA.https://doi.org/10.1145/2582051.2582064
Publications
“LevioPole: Mid-air Haptic Interactions using Multirotor”Tomoya Sasaki,Richard Sahala Hartanto,Kao-Hua Liu,Keitarou Tsuchiya,Atsushi Hiyama,Masahik InamiＤＯＩ10.1145/3214907.3214913
Press&Award
Tomoya Sasaki, Richard Sahala Hartanto, Kao-Hua Liu, Keitarou Tsuchiya, Atsushi Hiyama, Masahiko Inami, LevioPole, General Category/Entertainment Division, Grand Award/The Minister of METI Award, 2018 Asia Digital Art Award FUKUOKA, 2018
Founder
This project was supported by JST ERATO Grant Number JPM- JER1701, Japan.