Lip-Sync Singing Skeleton for Halloween with A.I.

I’ve been getting more and more into programming with A.I. to let a computer find solutions to problems I give it with reinforcement training or a neural network and a bunch of data. It’s not as simple as it sounds. I recently found a couple music projects that use A.I. to separate vocals, drums, bass, piano and other sounds from each other into their own audio tracks. One is called Spleeter and the other is Facebook Research’s Demucs. They both use Python and TensorFlow (a popular machine learning programming library).

I wrote an application with Node.js that will take advantage of Spleeter and use Youtube to download songs from a playlist and automatically separate them and save the music tracks. Then play the original and silently watches the RMS volume of the vocals and drums to determine when to open his mouth or turn his neck.

When Halloween comes I hope to tie these to lights out front with relays and add a couple more singers to the stage.
I’m working on changing the high voltage servo for the mouth to a stepper motor with a bike brake cable for remote jaw actuation.
Take a look below to see behind the scenes of how the skeleton is controlled.

It's only fair to share...Share on facebook
Facebook
Share on google
Google
Share on twitter
Twitter
Share on linkedin
Linkedin

USB Distance Sensor and Software

I have created a simple distance sensing device with an Arduino Mega and an ultrasonic distance sensor to measure up to 5 feet. Also it is communicating to the host computer via USB at 115200 baud which allows for fast data updates and precision.

Below is a mockup of the software to customize and extend the uses of the device. Do anything from scroll through the web with hand gestures or create an invisible tripwire in your room or doorway. You can even use it to count customers entering your store or how many times your front door opened throughout the day.

Use if for real world data analytics or to improve energy efficiency around the house .

I plan to sell my distance device for a low price to get it into people’s hands and to cover my time in the software. I believe the software to this product will create the most value to average customers of all ages. My price point in mind is $24 on a production scale. But if you have a useful idea for this that has distance measuring needs I would create a custom one with some basic software for $50. It can be expanded on and it’s open source so all the code is available online.

Over the weekend my friends and I had a chuckle by creating a tripline to play randomized fart sound effects. Anything someone would grab a beer in the kitchen they would get a new fart noise. Goofy but a good example of the many uses it can serve. My favorite is scrolling through web pages when reading articles online. You can control volume or skip the song on iTunes. My devices is like a one dimensional Leap Motion. A Leap motion can see your hands in 3D and can do endless number of gestures. to control your computer, interface with software or play guys with your hands being the controllers.

Checkout the Store to see if it pops up in the near future. 😉

Goodbye for now.
-Joe