Teleimmersion People

Principle Investigator

Collaborators

Current Members

  • Shannon Chen (PhD)
  • Zhenhuan Gao (PhD)
  • Tarek Elgamal (PhD)
  • Mingwei Hu (MS)

Recent PhD Students

  • Raoul Rivas (PhD)
  • Ahsan Arefin (PhD)
  • Pengye Xia
  • Shu Shi (PhD)
  • Zixia Huang (PhD)
  • Wanmin Wu (PhD)
  • Rahul Malik (PhD)
  • Hossein Mobahi (PhD)
  • Zhenyu Yang (PhD & Post-doc)
  • Morihiko Tamai (Postdoc, Visiting Scholar)
  • Mahsa Kamali (PhD)
  • Dongyun Jin (PhD)
  • Jin Liang (PhD)
  • Bin Yu (PhD)

Recent Master and Undergraduate Students

  • Andrew Kryczka (Undergrad)
  • Matt Jurik (Undergrad)
  • Pooja Agarwal (MS)
  • Zhiyuan (Jason) Teo (Undergraduate)
  • Nathan Matthews (Undergraduate)
  • Chanhong Min (Undergraduate)
  • Tom Felker (Undergraduate)
  • Jigar Doshi (Master)
  • Muyuan Wang (Master)
  • Roger Cheng (Master)
  • Miles Johnson (Master)
  • Ravishankar Sathyam (Master)

User-centric Resource Adaptation in 3DTI

Resource consumption of a multi-model DIME (distributed interactive multimedia environment) is high. Due to the real-time guarantee and the involvement of multiple heterogeneous streams, applications in DIME are highly demanding on both computing and network resources. Thus, an intelligent resource adaptation scheme must be provided in order to utilize the limited resource in an efficient manner. Depending on the type of activity user(s) are conducting in the 3DTI environment, users’ requirement on the quality of service (QoS) can be very different. Thus, we utilize mobile phones of users’ as body sensors to help us detect the type of activity happening in a 3DTI environment. Base on the detected activity type, resolution of I/O devices and scheduling of computing resource can be dynamically fine-tuned and hence accomplish resource saving without degrading the quality of experience (QoE). This topic is currently investigated by Shannon Chen.

Adaptive 3D Streaming on Web

Bandwidth consumption of 3D data delivery is higher than 2D. With the advancement in Web technology, e.g. HTML 5, webrtc, the trend of video streaming has turned to web from desktop applications. Due to the real-time requirement and the involvement of multiple heterogeneous streams, 3D streaming is facing more challenges. Thus, we are in need of a new system that can stream 3D data in an efficient manner. This topic is currently investigated by Zhenhuan Gao.

Multimodality in 3D Tele-immersion

3D Tele-immersive systems often include more types of stream other than traditional video and audio streams. For example, in healthcare applications (e.g., physical therapy), there will be various streams of data coming from body sensors for a remote doctor to access the condition of a patient. And in interactive excergame applications, streams carrying the information of local user interaction or realtime virtual game events can be involved. We would like to investigate the impact of this multi-modality on the design of 3D Tele-immersive system, from the perspectives of QoS management and stream synchronization. This topic is currently investigated by Pengye Xia.

Large-Scale 3DTI Multi-stream Content Distribution

3D Tele-immersive (3DTI) systems create real-time multi-stream and multi-view 3D collaborative contents from multiple sites to allow interactive shared activities in virtual environments. In addition to interactive participants in 3DTI environments, we envision a large number of passive viewers that (a) watch the interactive activities in 3DTI shared environments, and (b) select views of the activities at run time. To achieve this vision, we develop 4D TeleCast, a novel 3D content distribution framework providing the functionality of multi-view selection. It addresses the following challenges: (1) supporting a large number of concurrent views as well as viewers, (2) preserving the unique nature of 3DTI multi-stream and multi-view dependencies, and (3) allowing dynamic viewer behavior such as view changes and large-scale simultaneous viewer arrivals or departures. This topic is current investigated by Ahsan Arefin.

Activity Driven Session Management in 3DTI

Current interactive tele-presence systems are designed and optimized for one particular type of cyber-physical activity such as conversation, video chat, or gaming. However, with the emerging new 3D tele-immersive (TI) systems, such as our own TI system, called TEEVE (TEle-immersion for EVErybody), we observe that the same TI system platform is being used for very different activities. In this paper, we classify the TI activities with respect to their physical characteristics, qualitatively analyze the cyber side of TI activities, and argue that one needs to consider very different performance profiles of the same TI system platform in order to achieve high quality of experience (QoE) for different cyber-physical TI activities. In this work, we also develop an optimization approach in the global control plane for optimizing the content distribution considering the activity-driven QoS requirements. This topic is current investigated by Ahsan Arefin and Raoul Rivas.

Operating System Support for 3DTI Systems

3DTI Systems have become ubiquitous and examples are Halo, Tele-prescence and Teeve. These systems are comprised of a large number of streaming devices distributed across multiple geographical locations.3D TIs allow real-time collaborative activities like Exergaming, and Physical Training. To allow rich audio-visual real-time interactions in the virtual space, 3D TIs are usually composed of input devices, output devices and disseminating infrastructures. 3D TIs are characterized by diversity and contingency of activities, high-degree of interactivity, heterogeneity of I/O devices and large scale of concurrent, collocated and codependent streaming devices. Current operating systems do not provide adequate support for 3D TIs. Our solution follows a holistic approach based on Streaming as a Service (SAS) paradigm, where correlated multi-streams are first class objects in a distributed Operating System. We use the concept of Streaming as a Service to provide Resource Management, Ubiquitous Access, Location Independence, Platform Independence and Universal Access to streaming devices in 3D TIs. This topic is current investigated by Raoul Rivas.