Lip-Sync Singing Skeleton for Halloween with A.I.

I’ve been getting more and more into programming with A.I. to let a computer find solutions to problems I give it with reinforcement training or a neural network and a bunch of data. It’s not as simple as it sounds. I recently found a couple music projects that use A.I. to separate vocals, drums, bass, piano and other sounds from each other into their own audio tracks. One is called Spleeter and the other is Facebook Research’s Demucs. They both use Python and TensorFlow (a popular machine learning programming library).

I wrote an application with Node.js that will take advantage of Spleeter and use Youtube to download songs from a playlist and automatically separate them and save the music tracks. Then play the original and silently watches the RMS volume of the vocals and drums to determine when to open his mouth or turn his neck.

When Halloween comes I hope to tie these to lights out front with relays and add a couple more singers to the stage.
I’m working on changing the high voltage servo for the mouth to a stepper motor with a bike brake cable for remote jaw actuation.
Take a look below to see behind the scenes of how the skeleton is controlled.

It's only fair to share...Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Share on LinkedIn
Linkedin