OPEN SOURCE TOOLKIT
Kinectron
Kinectron is an open-source toolkit that brings realtime, motion capture data into the browser. Kinectron uses the same networking technology as common video calling software, but rather than broadcasting video from a webcam, Kinectron broadcasts unique three-dimensional pose and image data from the Microsoft Kinect, a lightweight and inexpensive motion capture camera. Kinectron is a tool for artists, activists, and educators, who use it to make interactive sketches, to teach about computation and the body, and in both in-person and telematic performances. Kinectron was developed under the Google xStory Experiments in Storytelling Research Grant at NYU ITP with Shawn Van Every.
GRADUATE ELECTIVE COURSE
NYU: The Body, Everywhere and Here
The Body, Everywhere and Here is a graduate elective course that encourages students to explore embodied interaction on the web. The course will include discussion of influential works in the development of online embodied interaction, including the works of Kit Galloway and Sherrie Rabinowitz, Susan Kozel, and Laurie Anderson. Students experiment with embodied interactions in the browser and across networks using technologies like TensorFlow.js (PoseNet and BodyPix), Microsoft Kinect, p5.js and Three.js.
TECHNICAL WRITING
Writing on “Javascript in Plain English” blog and Medium
Writing based on work around the embodied internet, featuring articles around the use of the phyiscal body on web pages. Topics include Microsoft Kinect to browser, peer to peer networking, webRTC, and using pose detection in interactive installations.
OPEN SOURCE TEMPLATE
Olos Template
Ὅλος (pronounced “Olos”) is a telematic web performance between Chile and the US using a standard LogiTech webcam and all open source web-based technologies. The Olos Template is an open-source release of the underlying structure built to connect dancers across distance. It uses peer-to-peer networking and TensorFlow.js BlazePose model to help others get started in building telematic web performance.
OPEN SOURCE TOOLKIT
Simple Peer Server and Client Wrapper
Simple-peer is an excellent library by Feross for creating webRTC peer connections, however, it does not include a signaling server, which is necessary for establishing the peer connections used by simple-peer. Simple-peer-server and client wrapper provide a signaling server to establish a connection between two or more peers. It uses Socket.IO to transport the signaling messages until the peer connections are created.