Reciprocal Shading for Mixed Reality

The aim of the RESHADE project is to simulate the mutual influence between real and virtual objects in mixed reality applications. Virtual objects in such applications appear disturbingly artificial because rendering completely ignores the real environment. But from the term mixed reality one would expect that virtual and real object harmoniously blend into one visual perception and cannot be distinguished easily.

The ambitious goal of this project is to provide users with a perfect illusion, so that they cannot perceive a difference between virtual and real objects. That means virtual objects are responsive to changes in the real environment in real time. They have to be rendered in a believable realistic way so that they smoothly blend with reality. It is also important to simulate the visual effects of virtual objects on real ones.

Besides for realistic rendering this technique can also be used for innovative ways of interaction and dynamic behaviour of virtual objects, which are key aspects of mixed reality.

Research focuses on real time rendering algorithms for mixed reality applications that consider the lighting and geometry of the real environment. Special cameras will continuously capture reality. The real geometry will also be used to simulate effects of virtual objects on real ones in real time. Shading of real objects will be corrected by digital composition.

Research results will be repeatedly evaluated in the course of the project within three different application scenarios. Thereby human perception is the sole key factor for assessment of the quality of the solution.