IllumiRoom may be part of Microsoft’s next gen strategy

In January at CES 2013, Eric Rudder, Microsoft’s chief technology strategy officer, unveiled the IllumiRoom—a new Kinect driven device for use with next generation gaming consoles. This proof-of-concept project being developed by Microsoft Research “augments the area surrounding a television screen with projected visualizations to enhance the traditional living room entertainment experience.”

Pairing a Kinect for Windows with a projector allows for enhanced gameplay experiences by extending the screen beyond the TV. Unlike previous technology, such as Focus+Context screens, the IllumiRoom can be used in any type of living room setup.

While sitting on a coffee table, the self-calibrating Kinect will map out the user’s living room by detecting the placement of the TV, stationary objects, and colors in the environment. This allows the projector to “change the appearance of the room, induce apparent motion, extend the field of view, and enable entirely new game experiences.” Microsoft Research released the video below to demonstrate what the IllumiRoom can do.

A recent research paper released by Microsoft offers an in depth look at the IllumiRoom. If it has access to a game’s rendering process, several Focus+Context illusions can be created, such as extending the entire game or selected objects beyond the TV. Without access to the renderer, peripheral flow gives the user the sensation of movement.

The device can also displace textures in the room. The radial wobble can be used to simulate a ripple illusion throughout the room. Ideally, Microsoft Research would expand this to explosions and other more grand effects with a more sophisticated projector.

The IllumiRoom is also capable of color augmentation. For example, if a game or movie uses black and white video, the projector will display the appropriate colors to make the area around the TV appear black and white. Perhaps, a more saturated, cartoonish look is being displayed. The device will project the color of objects in the room back onto themselves to further saturate the area surrounding the TV.

All of these effects work because the human eye has two distinct ways of taking in it’s surroundings. The fovea is central to the retina and responsible for sharp, focused viewing. Outside of the focused area is peripheral vision. While very attuned to movement, peripheral vision has trouble fully discerning what’s being seen. In IllumiRoom’s case, users are focusing on the TV, so the effects can take place in the peripheral area, making them much more believable.

Microsoft Research’s Brett Jones told Polygon IllumiRoom is only a “research project;” it’s reasonable to assume the technology could come to the next generation Xbox at some point. In fact, the previously mentioned research paper offers that possibility: “The concept could be developed into a next generation game console with game content designed from the ground up, or it could be an ‘addon’ for existing consoles . . .” One of the team’s current obstacles is the Kinect’s limited area of view. However, the Kinect 2 is rumored to have an increased field of view, so the IllumiRoom could have a bright future in households.

Source: Microsoft Research, via The Verge