AI / ML Experiments

AI / ML Experiments is a series of projects exploring intelligence and learning as creative tools in the browser. The projects are built using browser-based creative coding libraries three.js and p5.js. They have been developed in part as my work on the Google-backed ml5.js research group and as a research resident at NYU ITP.

Role Creator, Developer
Tools Three.js/p5.js/Tensorflow.js/ml5.js/Javascript, Python/Scikit, Processing/Java, Node

Teachable Machine With Skeleton

Teachable Machine With Skeleton was inspired by Google’s Teachable Machine project—in which a user can quickly train a program in the browser to recognize three images with their webcam—and the Machines Learning Movement workshop by Kat Sullivan.

In this project the player uses full body gestures to train the machine, and creative code responses change based on the full body gestures. The current code runs in the browser using p5.js and works from data streamed to the browser by Kinectron, an open source software I built and maintain. The upcoming version of the project will use the recently released PoseNet model, which was released with TensorFlow.js.

KNN Animations

Algorithms control so much of modern-day life—from the news that we see, to the friends with whom we connect—yet it’s hard to understand just how pliant they are.

KNN Animations visualizes the radical change in results based on changing just two variables in the K nearest neighbor algorithm. The program loops “k” from 0-50 with each frame. The user can change the label that is visualized between three options with a slider. Here’s a live version of the sketch running in p5.js.

Binary We

Binary We is a visualization of my iMessage history with my partner in the final four weeks of my graduate degree. The information is organized in typical calendar form, with each day’s messages sorted into three-dimensional binary trees drawn in three.js. The dates toggle on and off, and hovering over any of the trees will make them rotate.



The project is online at lmj.io/projects/binarywe/.

World Leader Evolutionary Ecosystem

The World Leader Evolutionary Ecosystem is based on evolutionary code written by Dan Shiffman to create an evolutionary battle between current world leaders: Trump, Merkel, Trudeau, Putin, and of course, Kim Jung-un.

Here is a live version of the sketch in p5.js. Click on the image to spawn new leaders. Each time they hit a liberty flame, they gain power, making them survive longer and more likely to reproduce.

The next version of this project will weight the leaders’ starting power based on current social and political metrics such as stock market, currency values and approval ratings.