Browsing by Author "Cheng, Hsien-Tzu"

Now showing 1 - 2 of 2

Deep 360 Pilot: Learning a Deep Agent for Piloting through 360 degrees Sports Videos
(2017-01-01)
Hu, Hou-Ning
;
Lin, Yen-Chen
;
Liu, Ming-Yu
;
Cheng, Hsien-Tzu
;
Chang, Yung-Ju
Watching a 360 degrees sports video requires a viewer to continuously select a viewing angle, either through a sequence of mouse clicks or head movements. To relieve the viewer from this "360 piloting" task, we propose "deep 360 pilot" - a deep learning-based agent for piloting through 360 degrees sports videos automatically. At each frame, the agent observes a panoramic image and has the knowledge of previously selected viewing angles. The task of the agent is to shift the current viewing angle (i.e. action) to the next preferred one (i.e., goal). We propose to directly learn an online policy of the agent from data. Specifically, we leverage a state-of-the-art object detector to propose a few candidate objects of interest (yellow boxes in Fig. 1). Then, a recurrent neural network is used to select the main object (green dash boxes in Fig. 1). Given the main object and previously selected viewing angles, our method regresses a shift in viewing angle to move to the next one. We use the policy gradient technique to jointly train our pipeline, by minimizing: (1) a regression loss measuring the distance between the selected and ground truth viewing angles, (2) a smoothness loss encouraging smooth transition in viewing angle, and (3) maximizing an expected reward of focusing on a foreground object. To evaluate our method, we built a new 360-Sports video dataset consisting of five sports domains. We trained domain-specific agents and achieved the best performance on viewing angle selection accuracy and users' preference compared to [53] and other baselines.
Tell Me Where to Look: Investigating Ways for Assisting Focus in 360 degrees Video
(2017-01-01)
Lin, Yen-Chen
;
Chang, Yung-Ju
;
Hu, Hou-Ning
;
Cheng, Hsien-Tzu
;
Huang, Chi-Wen
360 degrees videos give viewers a spherical view and immersive experience of surroundings. However, one challenge of watching 360 degrees videos is continuously focusing and refocusing intended targets. To address this challenge, we developed two Focus Assistance techniques: Auto Pilot (directly bringing viewers to the target), and Visual Guidance (indicating the direction of the target). We conducted an experiment to measure viewers' video-watching experience and discomfort using these techniques and obtained their qualitative feedback. We showed that: 1) Focus Assistance improved ease of focus. 2) Focus Assistance techniques have specificity to video content. 3) Participants' preference of and experience with Focus Assistance depended not only on individual difference but also on their goal of watching the video. 4) Factors such as view-moving-distance, salience of the intended target and guidance, and language comprehension affected participants' video-watching experience. Based on these findings, we provide design implications for better 360 degrees video focus assistance.