Occlusion Leak Compensation for Optical See-Through Displays using a Single-layer Transmissive Spatial Light Modulator
Yuta Itoh, Takumi Hamasaki, Maki Sugimoto
—Making Optical See-Through Displays Occlusion-Enabled 
(This work is to be presented at ISMAR 2017)
Abstract
Enabling occlusion—a visual depth cue for the 3D perception—in optical see-through displays is a key feature to make see-through augmented reality (AR) realistic.
We propose an occlusion compensation method for optical see-through head-mounted displays (OST-HMDs) equipped with a singlelayer transmissive spatial light modulator (SLM), in particular, a liquid crystal display (LCD). Occlusion is an important depth cue for 3D perception, yet realizing it on OST-HMDs is particularly difficult due to the displays’ semitransparent nature.
A key component for the occlusion support is the SLM—a device that can selectively interfere with light rays passing through it. For example, an LCD is a transmissive SLM that can block or pass incoming light rays by turning pixels black or transparent. A straightforward solution places an LCD in front of an OST-HMD and drives the LCD to block light rays that could pass through rendered virtual objects at the viewpoint. This simple approach is, however, defective due to the depth mismatch between the LCD panel and the virtual objects, leading to blurred occlusion. This led existing OST-HMDs to employ dedicated hardware such as focus optics and multi-stacked SLMs.
Contrary to these viable, yet complex and/or computationally expensive solutions, we return to the single-layer LCD approach for the hardware simplicity while maintaining fine occlusion—we compensate for a degraded occlusion area by overlaying a compensation image. We compute the image based on the HMD parameters and the background scene captured by a scene camera. The evaluation demonstrates that the proposed method reduced the occlusion leak error by 61.4% and the occlusion error by 85.7%.
Here’s some direct shots of the system when no occlusion or when our method was used:
Paper and Video
A pre-print of the publication: PDF.
Demonstration of our method:
Publications
Yuta Itoh, Takumi Hamasaki, Maki Sugimoto, “Occlusion Leak Compensation for Optical See-Through Displays using a Single-layer Transmissive Spatial Light Modulator,” IEEE Transactions on Visualization and Computer Graphics (will be presented at ISMAR 2017), vol. xx, num. x pp. xxxx–xxxx, DOI: 10.1109/TVCG.2017.2734427