Facing You is an interactive installation that envisions what an experiential internet could look and feel like. In this experience two people in two different places physically meet and interact in virtual space. Both users are represented with full-body skeletal and volumetric data on a website. They can create effects in their shared environment with different interactions, like touching hands, touching feet or sharing the same virtual space. The experience runs entirely in the browser, and an online audience can tune in from other locations.

Facing You is built on an experimental open source software called Kinectron, which allows artists and activists to explore the power of bringing the body online. Read more about Kinectron.

Facing You is currently under a new phase of Shanghai-New York development under the NYU Visual Arts Initiative grant for 2017-18.

I presented Facing You as my NYU ITP graduate thesis project in May 2017.

Role	Creator, Developer
Tools	Kinect, Kinectron, Live Web, Three.js, Javascript, Projection
Showings	CultureHub Season Opening, September 2017
	Experiments in Storytelling Show, May 2017
	ITP Thesis Week, May 2017
	ReFest, March 2017
Process	Thesis Journal

How It Works

In Facing You, there are two rooms in two different places. Each room has a screen, a PC running Kinectron, and a Kinect V2. The Kinects are placed perpendicular to the screens so that the users’ images are facing each other in the virtual space. There is one person in each room. Each user’s image is captured by the Kinect, broadcast and picked on a website, where the images are combined. Both users view the website on the screen in front of them in real time, and the user experience is built to encourage them to interact with one another.

Because this project is running on a website, an online audience can tune in from other locations.

Experience 1: Volumetric Video

In this experience two people represented by live, color volumetric video meet in a virtual space facing each other. Their depth combines and turns white when they overlap, so it appears as a point cloud coming out of the screen.

Experience 2: Skeletal Webs

In this experience the users are represented by line drawings of their skeletal data. They have webs between their joints, encouraging them to push and pull at one another.

Experience 3: Triangle Bones

In this experience, the users can move the floor by touching hands and change the color of their bodies by overlapping.