Lip-Sync Singing Skeleton for Halloween with A.I.
I’ve been getting more and more into programming with A.I. to let a computer find solutions to problems I give it with reinforcement training or a neural network and a bunch of data. It’s not as simple as it sounds. I recently found a couple music projects that use A.I. to separate vocals, drums, bass, piano and other sounds from each other into their own audio tracks. One is called Spleeter and the other is Facebook Research’s Demucs. They both use Python and TensorFlow (a popular machine learning programming library).
I wrote an application with Node.js that will take advantage of Spleeter and use Youtube to download songs from a playlist and automatically separate them and save the music tracks. Then play the original and silently watches the RMS volume of the vocals and drums to determine when to open his mouth or turn his neck.
When Halloween comes I hope to tie these to lights out front with relays and add a couple more singers to the stage.
I’m working on changing the high voltage servo for the mouth to a stepper motor with a bike brake cable for remote jaw actuation.
Take a look below to see behind the scenes of how the skeleton is controlled.
I have created a simple distance sensing device with an Arduino Mega and an ultrasonic distance sensor to measure up to 5 feet. Also it is communicating to the host computer via USB at 115200 baud which allows for fast data updates and precision.
The LED Soundcloud is a cool piece to have in any room. It has an slim credit card sized remote and has multiple modes. Use it as a color lamp, trippy acid light mode or sound reactive lightning mode. The lightning modes consist of three different types of strikes and randomizes between them.