This work introduces novel interaction and applications using smartphones with kinetic capabilities. We develop an accessory module with robot wheels for a smartphone. With this module, the smartphone can move in a linear direction or rotate with sufficient power. The module also includes rotary encoders, allowing us to use the wheels as an input modality. We demonstrate a series of novel mobile interaction for mobile devices with kinetic capabilities through three applications.
Takefumi Hiraki, Koya Narumi, Koji Yatani, and Yoshihiro Kawahara. 2016. Phones on Wheels: Exploring Interaction for Smartphones with Kinetic Capabilities. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST ’16 Adjunct), 121-122.
Musical instrument playing is a skill many people desire to acquire, and learners now have a wide variety of learning materials. However, their volume is enormous, and novice learners may easily get lost in which songs they should practice first. We develop Strummer: an interactive multimedial system for guitar practice. Strummer provide data-driven and personalized practice for learners in order to identify important and easy-to-learn chords and songs. This practice design is intended to encourage smooth skill transfers to songs that learners even have not seen. Our user study confirms the benefits and possible improvements of the Strummer system. In particular, participants expressed their positive impressions on lessons provided by the system.
Shunya Ariga, Masataka Goto, and Koji Yatani. Strummer: An Interactive Guitar Chord Practice System. In Proc. ICME 2017, pp. 1057-1062. (paper)
Video consumption is being shifted from sit-and-watch to selective skimming. Existing video player interfaces, however, only provide indirect manipulation to support this emerging behavior. Video summarization alleviates this issue to some extent, shortening a video based on the desired length of a summary as an input variable. But an optimal length of a summarized video is often not available in advance. Moreover, the user cannot edit the summary once it is produced, limiting its practical applications. We argue that video summarization should be an interactive, mixed-initiative process in which users have control over the summarization procedure while algorithms help users achieve their goal via video understanding. In this paper, we introduce ElasticPlay, a mixed-initiative approach that combines an advanced video summarization technique with direct interface manipulation to help users control the video summarization process. Users can specify a time budget for the remaining content while watching a video; our system then immediately updates the playback plan using our proposed cut-and-forward algorithm, determining which parts to skip or to fast-forward. This interactive process allows users to fine-tune the summarization result with immediate feedback. We show that our system outperforms existing video summarization techniques on the TVSum50 dataset. We also report two lab studies (22 participants) and a Mechanical Turk deployment study (60 participants), and show that the participants responded favorably to ElasticPlay.
We present a full-paper of work in collaboration with researchers in Microsoft Research at CHI 2015.
CHI 2015にてMicrosoft Researchの研究者との共同研究の成果をフルペーパーとして発表します．
Darren Edge, Sumit Gulwani, Natasa Milic-Frayling, Mohammad Raza, Reza Adhitya Saputra, Chao Wang, and Koji Yatani. Mixed-Initiative Approaches to Global Editing in Slideware. In Proceedings of CHI 2015.
Koji is serving as a Program Committee for UIST 2015
Koji Yatani will be serving as a Program Committee for UIST 2015.
We present a full-paper of work in collaboration with professors and students in KAIST at CSCW 2015.
Minsam Ko, Subin Yang, Joonwon Lee, Christian Heizmann, Jinyoung Jeong, Uichin Lee, Daehee Shin, Koji Yatani, Junehwa Song, and Kyong-Mee Chung. 2015. NUGU: A Group-based Intervention App for Improving Self-Regulation of Limiting Smartphone Use. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15). ACM, New York, NY, USA, 1235-1245.
Koji is serving as a Program Committee for Mobiquitous 2015
Our ReviewCollage (in collaboration with Haojian Jin and Tetsuya Sakai) and TalkZones (in collaboration with Bahador Saket, Sijie Yang, Hong Tan, and Darren Edge) projects won Honorable Mention Awards at MobileHCI 2014. Our papers will be available on this website soon.