Real-Time Facial Analysis To Control 3D Objects

Using facial position and gestures to illicit visual 3D responses in real-time

Harnessing the power of the iPhone camera sensor suite, I programmed a facial analysis network in Touchdesigner to track vertical and horizontal head movement which would control the spin movement of the cube, alongside mouth opening and closing data to trigger expansion of the cube. When a user poked out a tongue a fun RGB effect is applied.