NEWS!

  • 2018
    Coming soon!

Where there is a will, there is a way.

Academic Experiences

  • Present 2017.08

    Visiting Graduate Student

    UCLA, Department of Statistics

  • 2016.07 2016.01

    Research Intern

    Microsoft Research Asia

  • 2012.09 2012.04

    Research Intern

    North China Research Institute of Electro-Optics

Education & Training

  • Ph.D. 2013-Present

    Ph.D. in Optical Engineering

    Beijing Institute of Technology

  • B.A.2009-2013

    Bachelor of Electronic Science & Technology

    Beijing Institute of Technology


Web Counter unique visitors since October 2018

Research Summary

My research involves multiple fields. My main researches are focused on virtual reality (VR), mixed reality (MR) and augmented reality (AR). I spent much energy on augmented reality based on head-mounted displays (HMDs). Meanwhile, I also use virtual reality as a platform to study some general tasks in the domain of artificial intelligence (AI). I am also interested in computer vision and machine learning. As for computer vision, I am working on the high-level vision, such as visual reasoning. Machine learning is a basic technique for computer vision, which facilitates the study on computer vision. Finally, I also want to integrate AI to the current framework about physical and virtual reality, which will construct a beautiful symmetrical structure of the physical world and the virtual world, leading to some new theories in understanding the nature.

Interests

  • Virtual Reality, Augmented/Mixed Reality
  • Computer Vision & Machine Learning
  • Artificial Intelligence in Mirrored World

Laboratory Personel

Chi Zhang

Master Student

Follow

Yixin Zhu

Postdoc

Follow

Feng Gao

Ph.D. Student

Follow

Hangxin Liu

Ph.D. Student

Follow

Mark Edmonds

Ph.D. Student

Follow

Shuwen Qiu

Master Student

Follow

Xu Xie

Ph.D. Student

Follow

Yuxing Qiu

Ph.D. Student

Follow

Song-Chun Zhu

Professor

Follow

Zeyu Zhang

Master Student

Follow

Great lab Personel!

There are so many excellent researchers. Their researches may be very helpful for you!

Just "Click" the left "Follow" button to know more.

Filter by type:

Sort by year:

Subjective and objective evaluation of visual fatigue caused by continuous and discontinuous use of HMDs


Jie Guo, Dongdong Weng, Zhenliang Zhang, Yue Liu, Henry B.L. Duh, Yongtian Wang

Journal Paper Journal of the Society for Information Display, 2018

Abstract

During continuous use of displays, a short rest can relax users' eyes and relieve visual fatigue. As one of the most important devices of virtual reality, head‐mounted displays (HMDs) can create an immersive 3D virtual world. When users have a short rest during the using of HMDs, they will experience a transition from virtual world to real world. In order to investigate how this change affects users' eye condition, we designed a 2 × 2 experiment to explore the effects of short rest during continuous using of HMDs and compared the results with those of 2D displays. The Visual Fatigue Scale, critical flicker frequency, visual acuity, pupillary diameter, and accommodation response of 80 participants were measured to assess the subject's performance. The experimental results indicated that a short rest during the using of 2D displays could significantly reduce users' visual fatigue. However, the experimental results of using HMDs showed that short rest during continuous using of HMD induced more severe symptoms of subjectively visual discomfort, but reduced the objectively visual fatigue.

Vision‐Tangible Interactive Display Method for Mixed and Virtual Reality: Toward the Human‐Centered Editable Reality


Zhenliang Zhang, Yue Li, Jie Guo, Dongdong Weng, Yue Liu, Yongtian Wang

Journal Paper Journal of the Society for Information Display, 2018

Abstract

Building a human‐centered editable world can be fully realized in a virtual environment. Both mixed reality (MR) and virtual reality (VR) are feasible solutions to support the attribute of edition. Based on the current development of MR and VR, we present the vision‐tangible interactive display method and its implementation in both MR and VR. We address the issue of MR and VR together because they are similar regarding the proposed method. The editable mixed and virtual reality system is useful for studies, which exploit it as a platform. In this paper, we construct a virtual reality environment based on the Oculus Rift, and an MR system based on a binocular optical see‐through head‐mounted display. In the MR system about manipulating the Rubik's cube, and the VR system about deforming the virtual objects, the proposed vision‐tangible interactive display method is utilized to provide users with a more immersive environment. Experimental results indicate that the vision‐tangible interactive display method can improve the user experience and can be a promising way to make the virtual environment better.

Depth-Aware Interactive Display Method for Vision-Tangible Mixed Reality


Zhenliang Zhang, Yue Li, Jie Guo, Dongdong Weng, Yue Liu, Yongtian Wang

Conference Paper (Oral) SPIE/COS Photonics Asia, 2018

Abstract

Vision-tangible mixed reality (VTMR) is a further development of the traditional mixed reality. It provides an experience of directly manipulating virtual objects at the perceptual level of vision. In this paper, we propose a mixed reality system called “VTouch”. VTouch is composed of an optical see-through head-mounted display (OST-HMD) and a depth camera, supporting a direct 6 degree-of-freedom transformation and a detailed manipulation of 6 sides of the Rubik’s cube. All operations can be performed based on the spatial physical detection between virtual and real objects. We have not only implemented a qualitative analysis of the effectiveness of the system by a functional test, but also performed quantitative experiments to test the effects of depth occlusion. In this way, we put forward basic design principles and give suggestions for future development of similar systems. This kind of mixed reality system is significant for promoting the development of the intelligent environment with state-of-the-art interaction techniques.

HiKeyb: High-Efficiency Mixed Reality System for Text Entry


Haiyan Jiang, Dongdong Weng, Zhenliang Zhang, Yihua Bao, Yufei Jia, Mengman Nie

Conference Paper (Poster) IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), 2018

Abstract

Text entry is an imperative issue to be addressed in current entry systems for virtual environments (VEs). The entry method using a physical keyboard is still the most dominant choice for an efficient interaction regarding text entry. In this paper, we propose a typing system with a style of mixed reality, which is called HiKeyb, and it possesses a similar high-efficiency with the single physical keyboard in the real environment. The HiKeyb system consists of a depth camera, a pose tracking module, a head-mounted display (HMD), a QWERTY keyboard and a black table mat. This system can guarantee the entry efficiency and the amenity by not only introducing the force feedback from a movable physical keyboard, but also improving the immersion with the real hand image. In addition, the infrared absorption material helps improve the robustness of the system against different lighting environments. Experiments have proved that users wearing HMDs in Virtual Phrases session can achieve an entry rate of 23.1 words per minute and an error rate of 2.76\\%, and the rate ratio of virtual reality to real world is 78\\% when typing phrases. Besides, we find that the proposed system can provide a relatively close entry efficiency to that using a pure physical keyboard in the real environment.

Inverse Augmented Reality: A Virtual Agent's Perspective


Zhenliang Zhang, Dongdong Weng, Haiyan Jiang, Yue Liu, Yongtian Wang

Conference Paper (Poster) IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), 2018

Abstract

We propose a framework called inverse augmented reality (IAR) which describes the scenario that a virtual agent living in the virtual world can observe both virtual objects and real objects. This is different from the traditional augmented reality. The traditional virtual reality, mixed reality and augmented reality are all generated for humans, i.e., they are human-centered frameworks. On the contrary, the proposed inverse augmented reality is a virtual agent-centered framework, which represents and analyzes the reality from a virtual agent's perspective. In this paper, we elaborate the framework of inverse augmented reality to argue the equivalence of the virtual world and the physical world regarding the whole physical structure.

Task‐Driven Latent Active Correction for Physics‐Inspired Input Method in Near‐Field Mixed Reality Applications

Zhenliang Zhang, Yue Li, Jie Guo, Dongdong Weng, Yue Liu, Yongtian Wang

Journal Paper Journal of the Society for Information Display, 2018

Abstract

Calibration accuracy is one of the most important factors to affect the user experience in mixed reality applications. For a typical mixed reality system built with the optical see‐through head‐mounted display, a key problem is how to guarantee the accuracy of hand–eye coordination by decreasing the instability of the eye and the head‐mounted display in long‐term use. In this paper, we propose a real‐time latent active correction algorithm to decrease hand–eye calibration errors accumulated over time. Experimental results show that we can guarantee an effective calibration result and improve the user experience with the proposed latent active correction algorithm. Based on the proposed system, experiments about virtual buttons are also designed, and the interactive performance regarding different scales of virtual buttons is presented. Finally, a direct physics‐inspired input method is constructed, which shares a similar performance with the gesture‐based input method but provides a lower learning cost due to its naturalness.

Enhancing Data Acquisition for Calibration of Optical See-Through Head-Mounted Displays

Zhenliang Zhang, Dongdong Weng, Jie Guo, Yue Liu, Yongtian Wang, Hua Huang

Journal Paper Optical Engineering, 2018

Abstract

Single point active alignment method is a widely used calibration method for optical-see-through head-mounted displays (OST-HMDs) since its appearance. It always requires high-accuracy alignment for data acquisition, and the collected data affect the calibration accuracy to a large extent. However, there are often many kinds of alignment errors occurring in the calibration process. These errors may contain random errors of manual alignment and system errors of the fixed eye-HMD model. To tackle these problems, we first leverage a random sample consensus approach to recurrently decrease the random error of the collected data sequence and use a region-induced data enhancement method to reduce the system error. We design a typical framework to enhance the data acquisition for calibration, sequentially reducing the random error and the system error. Experimental results show that the proposed method can significantly make the calibration more robust due to the elimination of sampling points with large errors. At the same time, the calibration accuracy can be increased by the proposed dynamic eye-HMD model that takes the eye movement into consideration. The improvement about calibration should be significant to promote the applications based on OST-HMDs.

Employing Different Viewpoints for Remote Guidance in a Collaborative Augmented Environment


Hongling Sun, Yue Liu, Zhenliang Zhang, Xiaoxu Liu, Yongtian Wang

Conference Paper (Oral) The Sixth International Symposium of Chinese CHI, 2018

Abstract

This paper details the design, implementation and an initial evaluation of a collaborative platform named OptoBridge, which is aimed at enhancing remote guidance and skill acquisition for spatially distributed users. OptoBridge integrates augmented reality (AR), gesture interaction with video mediated communication and is preliminarily applied to the experimental teaching of the adjustment task with Michelson interferometer. An exploratory study has been conducted to qualitatively and quantitatively evaluate the extent to which different viewpoints affect the student's sense of presence, task performance, learning outcomes and subjective feelings in the remote collaborative augmented environment. 16 students from local universities have participated in the evaluation. The result shows the influence of two different viewpoints and indicates that OptoBridge can effectively support remote guidance and enhance the collaborators' experience.

Evaluation of Hand-Based Interaction for Near-Field Mixed Reality with Optical See-Through Head-Mounted Displays


Zhenliang Zhang, Benyang Cao, Dongdong Weng, Yue Liu, Yongtian Wang, Hua Huang

Conference Paper (Poster) IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2018

Abstract

Hand-based interaction is one of the most widely-used interaction modes in the applications based on optical see-through head-mounted displays (OST-HMDs). In this paper, such interaction modes as gesture-based interaction (GBI) and physics-based interaction (PBI) are developed to construct a mixed reality system to evaluate the advantages and disadvantages of different interaction modes for near-field mixed reality. The experimental results show that PBI leads to a better performance of users regarding their work efficiency in the proposed tasks. The statistical analysis of T-test has been adopted to prove that the difference of efficiency between different interaction modes is significant.

Physics-Inspired Input Method for Near-Field Mixed Reality Applications Using Latent Active Correction


Zhenliang Zhang, Yue Li, Dongdong Weng, Yue Liu, Yongtian Wang

Conference Paper (Poster) IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2018

Abstract

Calibration accuracy is one of the most important factors to affect the user experience in mixed reality applications. For a typical mixed reality system built with the optical see-through head-mounted display (OST-HMD), a key problem is how to guarantee the accuracy of hand-eye coordination by decreasing the instability of the eye and the HMD in long-term use. In this paper, we propose a real-time latent active correction (LAC) algorithm to decrease hand-eye calibration errors accumulated over time. Experimental results show that we can successfully use the LAC algorithm to physics-inspired virtual input methods.

Inverse Virtual Reality: Intelligence-Driven Mutually Mirrored World


Zhenliang Zhang, Benyang Cao, Jie Guo, Dongdong Weng, Yue Liu, Yongtian Wang

Conference Paper (Poster) IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2018

Abstract

Since artificial intelligence has been integrated into virtual reality, a new branch of virtual reality, which is called inverse virtual reality (IVR), is created. A typical IVR system contains both the intelligence-driven virtual reality and the physical reality, thus constructing an intelligence-driven mutually mirrored world. We propose the concept of IVR, and describe the details about the definition, structure and implementation of a typical IVR system. The paral-lei living environment is proposed as a typical application of IVR, which reveals that IVR has a significant potential to extend the human living environment.

[Poster] An Accurate Calibration Method for Optical See-Through Head-Mounted Displays Based on Actual Eye-Observation Model

Zhenliang Zhang, Dongdong Weng, Jie Guo, Yue Liu, Yongtian Wang

Conference Paper (Poster) IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), 2017

Abstract

Single point active alignment method (SPAAM) has become the basic calibration method for optical-see-through head-mounted displays since its appearance. However, SPAAM is based on a simple static pinhole camera model that assumes a static relationship between the user's eye and the HMD. Such theoretic defects lead to a limitation in calibration accuracy. We model the eye as a dynamic pinhole camera to account for the displacement of the eye during the calibration process. We use region-induced data enhancement (RIDE) to reduce the system error in the acquisition process. The experimental results prove that the proposed dynamic model performs better than the traditional static model, and the RIDE method can help users obtain a more accurate calibration result based on the dynamic model, which improves the accuracy significantly compared to the standard SPAAM.

RIDE: Region-Induced Data Enhancement Method for Dynamic Calibration of Optical See-Through Head-Mounted Displays

Zhenliang Zhang, Dongdong Weng, Yue Liu, Yongtian Wang, Xinjun Zhao

Conference Paper (Poster) IEEE Virtual Reality (VR), 2017

Abstract

The most commonly used single point active alignment method (SPAAM) is based on a static pinhole camera model, in which it is assumed that both the eye and the HMD are fixed. This leads to a limitation for calibration precision. In this work, we propose a dynamic pinhole camera model according to the fact that the human eye would experience an obvious displacement over the whole calibration process. Based on such a camera model, we propose a new calibration data acquisition method called the region-induced data enhancement (RIDE) to revise the calibration data. The experimental results prove that the proposed dynamic model performs better than the traditional static model in actual calibration.

OptoBridge: Assisting Skill Acquisition in the Remote Experimental Collaboration

Hongling Sun, Zhenliang Zhang, Yue Liu, Henry BL Duh

Conference Paper (Oral) The 28th Australian Conference on Computer-Human Interaction, 2016

Abstract

In this paper an experimental teaching platform named OptoBridge is presented which supports the sharing of the collaborative space for spatially distributed users to assist skill acquisition. The development of OptoBridge is based on augmented reality (AR) and integrates free-hand gesture interactions with the video mediated communication. The prototype is preliminarily applied in the optics field to promote skill execution in the case of the Michelson interferometer. OptoBridge enables the remote teacher to monitor the experimental scenario as well as the detailed optical phenomena through the transmitted video captured on the local side. Meanwhile the local learner equipped with the optical see-through head-mounted display (OSTHMD) can be indicated by virtual hands and augmented annotations controlled by the teacher's gestures and follow the guidance to get their skills practiced. The implementation of OptoBridge is also presented, aimed at providing a more engaging and efficient approach for remote skill teaching.

A Modular Calibration Framework for 3D Interaction System Based on Optical See-Through Head-Mounted Displays in Augmented Reality

Zhenliang Zhang, Dongdong Weng, Yue Liu, Yongtian Wang

Conference Paper (Oral) International Conference on Virtual Reality and Visualization (ICVRV), 2016

Abstract

With the rapid development of Virtual and Augmented Reality systems, it becomes more and more important to develop an efficient calibration method for optical see-through head-mounted displays (OST-HMDs). In this paper, a modular calibration framework with two calibration phases is proposed. In the first phase, an eye-involved equivalent camera model is proposed in order to compute the spatial position of the human eye directly, in the second phase, the gesture information is integrated to the system with a depth camera. In addition, a fast correction algorithm is introduced to ensure that the calibration result work for new users without additional complex recalibration procedures. The precision of the proposed modular calibration and optimization method is evaluated, and the result shows that the proposed method can simplify the recalibration procedures for OST-HMDs.

Impact of Consistency Between Visually Perceived Movement and Real Movement on Cybersickness

Li Cai, Dongdong Weng, Zhenliang Zhang, Xingyao Yu

Journal Paper Journal of System Simulation, 2016

Abstract

The cybersickness is still a huge obstacle for virtual reality(VR) system. Current researches on cybersickness are mostly based on static or dynamic simulators, while studies in the real movement state is rare. An evaluating system which employed a head-mounted display(HMD) with a running vehicle was proposed to study the cybersickness in the real movement state. In this system, users sitting in the vehicle could see virtual scenes which was consistent with the real motion through the HMD. Subjective and objective evaluating experiments were proposed to analyze the different levels of the cybersickness caused by visual-vestibular conflict. The results show that the consistency of real movement and visually perceived movement have a great impact on cybersickness. Cybersickness gets worse when the consistency decreases. Serious cybersickness may lead to extreme situation where the discomfort may not be afforded by users.

3D Optical See-Through Head-Mounted Display Based Augmented Reality System and Its Application

Zhenliang Zhang, Dongdong Weng, Yue Liu, Xiang Li

Conference Paper (Oral, Invited) International Conference on Optical and Photonic Engineering (icOPEN), 2015

Abstract

The combination of health and entertainment becomes possible due to the development of wearable augmented reality equipment and corresponding application software. In this paper, we implemented a fast calibration extended from SPAAM for an optical see-through head-mounted display (OSTHMD) which was made in our lab. During the calibration, the tracking and recognition techniques upon natural targets were used, and the spatial corresponding points had been set in dispersed and well-distributed positions. We evaluated the precision of this calibration, in which the view angle ranged from 0 degree to 70 degrees. Relying on the results above, we calculated the position of human eyes relative to the world coordinate system and rendered 3D objects in real time with arbitrary complexity on OSTHMD, which accurately matched the real world. Finally, we gave the degree of satisfaction about our device in the combination of entertainment and prevention of cervical vertebra diseases through user feedbacks.

  • image

    Virtual-Real Fusion

    3D calibration for mixed reality based on optical see-through head-mounted displays.

    The detailed description is coming soon.
  • image

    Text Entry using Virtual Keyboard

    Input words with a virtual keyboard displayed by an OST-HMD.

    The detailed description is coming soon.
  • image

    High-Fidelity Grasp in VR

    Use a VR glove to grasp virtual objects in a natural way.

    The detailed description is coming soon.

Contact & Meet Me

I would be happy to talk to you if you need my assistance in your research or you need technical support for your company. Though I have limited time, I will try my best to help you.

At My Office

You can find me at my office located at Bolter Hall, UCLA.

I am at my office every day from 9:00 am to 6:00 pm, but you may consider an e-mail to fix an appointment.