A new way to “see” live music- A Moment of Science

A Moment of Science

What does music look like?

For musicians, it can be notation. For others, it can be the memory that song reminds you of. Computers see music the same way it visualizes everything-electronic signals. This digital language of 1s and 0s used to create music is called MIDI, or Musical Instrument Digital Interface.

“It’s the standard by which instruments and musical software speak. They’re essentially shuttling numbers amongst each other to tell each control what to do,” says Jeff Whitehead, once a software engineer, now a guitar maker.

Almost all popular music production today uses MIDI to create beats & even entire songs but this software technology typically hasn’t worked very well with electric guitars. Whitehead is hoping to change that with years of experimentation leading to a handcrafted guitar he built with an embedded MIDI transmitter that wirelessly communicates to his computer.

“So when I play a chord (my guitar) translates it to a set of numbers that goes to the computer and triggers the sounds I want. In this case it’s orchestra strings.”

jeff whitehead, local luthier

With MIDI and computer software, there are limitless possibilities for musicians as anything that understands MIDI can be controlled by it. 

“Given that you’re working in a world of MIDI now…it was very straightforward to use the guitar to start triggering visuals.” Whitehead demonstrates by playing a melody that triggers videos based on respective notes. Think Disney’s Fantasia but in real time.

This first prototype is working well, but its a collaborative work in progress with Whitehead and a guitarist, Rotem Sivan, based out of New York who is hoping to use this technology once he’s able to perform live again.

“Combining my guitars, my technology and an incredible performer like Rotem… it looks like it’s going to be possible,” Whitehead says, “I’m very excited to be working with him on this.” 

In addition, Whitehead hopes that this technology and software can be used in the future to create a new way for the deaf and hard of hearing to enjoy live music.

Whitehead used video from Videezy.com for his visualization shown in this segment.

Storm Team 2 Meteorologist David Dickson

Copyright 2021 Nexstar Inc. All rights reserved. This material may not be published, broadcast, rewritten, or redistributed.


Click for latest news and information


Join our daily newsletter!