CyPhy – Multi-Modal Teleimmersion for Tele-Physiotherapy

Simulationsimulation

People | Research | Publications | Video | Gallery | Links | Traces | Contact

News!

  • Paper accepted in ACM Multimedia 2015: Paper of Shannon Chen,”Exploiting Contextual Information to Enable Efficient Content Delivery for 3D Tele-Immersion Applications”, has been accepted by the 23rd ACM International Conference on Multimedia (ACM MM 2015) Doctoral Symposium.
  • Demo accepted in ACM Multimedia 2015: Demo of Zhenhuan Gao, Shannon Chen and Klara Nahrstedt, “OmniViewer: Enabling Multi-modal 3D DASH” has been accepted for demonstration in the 23rd ACM International Conference on Multimedia (ACM MM 2015).
  • Our demo video of FreeViewer has been awarded the “Video of the Month” by Coordinated Science Lab! [Video]
  • Paper accepted in ACM TOMM 2015: Paper of Shannon Chen, Zhenhuan Gao, Klara Nahrstedt, and Indranil Gupta, “3DTI Amphitheater: Towards 3DTI Broadcasting” has been accepted by ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 2015.
  • Prof. Nahrstedt awarded the SIGMM award: Professor Nahrstedt has been awarded the prestigious ACM SIGMM award for outstanding technical contributions to multimedia computing, communications and applications. She has been selected for her “Pioneering Contributions in Quality of Service for Multimedia Systems and Networking and for Visionary Leadership of the Multimedia Community”. This award is also truly an acknowledgement of our MONET group’s work over the years and contributions that we have made to multimedia systems and networks.
  • Paper accepted in ACM Multimedia (MM) 2014: Paper of Sabrina Schulte, Shannon Chen and Klara Nahrstedt, “Stevens’ Power Law in 3D Tele-immersion: Towards Subjective Modeling of Multimodal Cyber Interaction” has been accepted for publication in the 22nd ACM International Conference on Multimedia (ACM MM 2014). Congratulations!
  • Demo accepted in ACM Multimedia (MM) 2014: Demo of Zhenhuan Gao, Shannon Chen and Klara Nahrstedt, “FreeViewer: A 3D Tele-Immersive Intelligent Director” has been accepted for demonstration in the 22nd ACM International Conference on Multimedia (ACM MM 2014).
  • Shannon Chen Awarded the Saburo Muroga Endowed Fellowship: Shannon Chen has been awarded the Saburo Muroga Endowed Fellowship for the academic year from August 16, 2014 to May 15, 2015 due to his outstanding achievement as a PhD student in the CS department of UIUC. Congratulations!
  • Paper accepted in ACM MMSys 2014: Our paper on “3DTI Amphitheater: A Manageable 3DTI Environment with Hierarchical Stream Prioritization” by Shannon Chen, Klara Nahrstedt, and Indranil Gupta has been accepted for publication in MMSys 2014. Congratulations!
  • ACMMM Huawei/3DLife Grand Challenge Winner: Paper of Shannon Chen, Pengye Xia, and Prof. Klara Nahrstedt: “Activity-Aware Adaptive Compression: A Morphing-based Frame Synthesis Application in 3DTI,” has won the Huawei/3DLife – 3D human reconstruction and action recognition Grand Challenge of ACM Multimedia 2013. Congratulations!
  • Paper accepted in IEEE ISM 2013: Our paper on “Impact of Morphing-based Frame Synthesis on Bandwidth Optimization for 3DTI Video” by Shannon Chen and Klara Nahrstedt has been accepted for publication in IEEE ISM 2013.
  • ACM Multimedia Grand Challenge Finalist: Our paper on “Activity-Aware Adaptive Compression: A Morphing-based Frame Synthesis Applicaiotn in 3DTI” by Shannon Chen, Pengye Xia and Klara Nahrstedt has been accepted as the finalist of ACM Multimedia Grand Challenge 2013.
  • Paper accepted in ICNP 2013: Our paper introducing a unique synergy between application and network to maintain multimedia QoS during session run-time has been accepted for publication in International Conference on Network Protocols (ICNP), 2013. The paper title is “OpenSession: SDN-based Cross-layer Multi-stream Management Protocol for 3D Teleimmersion”
  • Notable books and articles 2012: Our paper “CZLoD: A psychophysical approach for 3D tele-immersive video” has been selected as a notable paper in computing in 2012 by ACM Computing Reviews. The complete list is available at http://bit.ly/150YwcZ
  • Paper accepted in ICME: Our paper on “ACTIVITY-BASED SYNTHESIZED FRAME GENERATION IN 3DTI VIDEO” by Chien-nan Chen and Klara Nahrstedt has been accepted for publication in ICME 2013.
  • C.W. Gear Outstanding Undergraduate Award: Congratulations to Aadhar Jain for winning the C.W. Gear Outstanding Undergraduate Award! Created by Pengye Xia, Apr 22, 2013
  • Bronze Table Student Award: Aadhar Jain and Andrew Kryzcka, two undergraduate students working on TEEVE project, have received Bronze Table Student Award for 2013. Created by Pengye Xia, Apr 22, 2013
  • Paper accepted in ICME: Our paper on “Object-level bandwidth adaptation for 3D Tele-immersion” by Pengye Xia and Klara Nahrstedt has been accepted for publication in ICME 2013.
  • Prof. Nahrstedt named ACM Fellow of 2012: Professor Klara Nahrstedt has been named to the 2012 class of ACM Fellows for her “contributions to quality-of-service management for distributed multimedia systems.”
  • Paper accepted in MMSys2013: Our paper on “Prioritized Evolutionary Optimization in Open Session Management for 3D Tele-immersion” by Ahsan Arefin, Raoul Rivas and Klara Nahrstedt has been accepted for publication in ACM MMSys 2013. The acceptance rate this year is 15/63.

Media Coverage of TEEVE

At the Monet group in UIUC we work on the emerging tele-immersive 3D multi-camera room environments. These environments allow us to engage in distributed physical activities such as physical therapy, sport activities, and entertainment.

Such environments need 3D multi-camera setups at the sending side and multi-display setups at the receiving side connected via appropriate network infrastructure. They also pose a large number of challenging research questions.

  • What are the next generation of system architectures that allow to build these environments in a flexible manner, with COTS components and for a broader audience?
  • What are the compression algorithms required for 3D video transfer?
  • What are the user models?
  • What are the adaptive control mechanisms that are required for the multi-camera, multi-display TEEVE environment.

We are building a cross-layer control and streaming framework over general purpose delivery infrastructure, called TEEVE (Tele-immersive Environment for Everybody). We tackle problems such as capturing, reconstructing and displaying 4D content, and coordination, synchronization and QoS-enabled delivery of tele-immersive 3D visual streams to remote room(s) over Internet 2.

The first experiments of TEEVE and few other next generation architectures are encouraging, as we start to sustain communication of up to 12 3D streams with several frames per second (e.g., TEEVE can reach around 5 frames per second) over Internet 2. However a lot of work remains and a lot of problems need to be solved.

Funding Agencies

This project is supported by the National Science Foundation and National Center for Supercomputing Applications. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

  • NSF NeTSE 1012194 (Exploiting Multi-Modality for Tele-Immersion)
  • NSF NeTTS 0964081 (Modeling, Design and Evaluation of P2P Real-Time Streaming Networks)
  • NSF SGER 0840323 (Exploration of Distributed Creativity in Multi-Site 3D Tele-Immersive Spaces)
  • NSF CSR 0834480 (VIEW-CALL: View Control Management in Geographically Distributed Tele-immersive Environments)
  • NSF CSR 0720702 (H-Media: The Holistic multistream environment for distributed teleimmersive applications)
  • NSF SGER 0724464 (Interactive Choreography in 3D Tele-Immersive Spaces)
  • NSF SCI SGER 0549242 (Experimentation with TEEVE Tele-immersive Infrastructure)
  • NSF NeTS 0520182 (Application-level Broadband Protocol Framework for Multi-camera 3D Video Delivery)