Lip-Sync Singing Skeleton for Halloween with A.I.
I’ve been getting more and more into programming with A.I. to let a computer find solutions to problems I give it with reinforcement training or a neural network and a bunch of data. It’s not as simple as it sounds. I recently found a couple music projects that use A.I. to separate vocals, drums, bass, piano and other sounds from each other into their own audio tracks. One is called Spleeter and the other is Facebook Research’s Demucs. They both use Python and TensorFlow (a popular machine learning programming library).
I wrote an application with Node.js that will take advantage of Spleeter and use Youtube to download songs from a playlist and automatically separate them and save the music tracks. Then play the original and silently watches the RMS volume of the vocals and drums to determine when to open his mouth or turn his neck.
When Halloween comes I hope to tie these to lights out front with relays and add a couple more singers to the stage.
I’m working on changing the high voltage servo for the mouth to a stepper motor with a bike brake cable for remote jaw actuation.
Take a look below to see behind the scenes of how the skeleton is controlled.
Also my dispensers measure one shot every time. Making it do half shots and quarter shots is much tricker. Here’s a link to those racks where I used two for 12 bottles. 
The main brains (RasPi) boots up directly to my custom application that holds the drink database, system state (making drink, current ingredient, motors homed, etc), and real-time communication to the motors. I developed the applications using Javascript with Node.JS, C programming language, Socket.IO and JSON.
I have created a simple distance sensing device with an Arduino Mega and an ultrasonic distance sensor to measure up to 5 feet. Also it is communicating to the host computer via USB at 115200 baud which allows for fast data updates and precision.
