EXPLANATION OF SCREENSHOT 1 (Main Spectratune Panel):

Importantly, the way I have the program set for the picture, notes go up like reading: as you go right (within an octave range), and as you go down (each new row is the next octave). (You may think I have up and down inverted from what is natural, but somehow the analogy for Westerners of the way we read seemed most natural to me. Anyway, I actually have "up goes up" as an option, so you can run it that way if you like, and that's what I did for the videos on this site.)

The example above was taken with my electronic keyboard feeding my computer sound-card as device 1. I also happen to be listening to that keyboard with headphones, and humming along and matching pitch into my webcam microphone, which is set as device 2.

The keyboard is playing the E right above middle C. What shows in the spectral analysis for the keyboard, in blue, are the note (fundamental) and the first 12 overtones. (The electronic keyboard was set to a saxaphone -- a timbre rich in overtones. On say the piano setting of the keyboard, the higher overtones are less conspicuous. For the Sax from the keyboard that I did use, there are in fact some overtones beyond the first 12 visible in that last half-octave when I raise the "Plot Gain" a little beyond what I have it at for the screen shot.)

Because the keyboard generates the sax with a little vibrato, if you watch the fundamental and its overtones in motion, they actually move up and down in complete synchrony by about a 1/5 of a half-step. At the moment I snapped the screen shot, the vibrato was putting the true frequency just a tad above the note.

If you were using the blue spectral display above to check tuning or intonation, you would look for the lowest overtone to get the exact note, rather than an overtone.

When you have a single sound (with overtones as their usually are, of course), you can also use the single-pitch detection mode, which works like a normal chromatic tuner. (And by the same algorithm -- called autocorrelation based.) This is shown above in red for the keyboard. (Beware that the autocorrelation algorithm is actually imperfect, and in particular it occasionally picks out the right note in the wrong octave -- whether in this software product, or another chromatic tuner product.)

As I indicated, I was humming along trying to match the note in the webcam microphone when I took the shot. I am only running the single-pitch detection on the webcam microphone channel -- no spectrogram. The single pitch detection is showing as the yellow arrow. It is picking up only my humming and not the keyboard because I am listening to the keyboard through headphones.

You might wonder why I was humming along on the webcam microphone and not a microphone plugged into the soundcard. The reason is I wanted a separate analysis for the humming, and didn't want an analysis of the humming mixed in with the keyboard. With my soundcard, at least, there is no way to separate the sound-card microphone sound from the sound of anything else running through the sound card. (In my screen shot of the device panel below, this fact is manifested in that all sound-card signals for computer analysis come through the "Line In/Mic In" "Windows sound input device". Your sound card might be more flexible. Anyway, if you wanted an analysis from a separate mic of higher fidelity than a webcam mic, I believe you could put in a second sound-card. (You wouldn't need higher fidelity just to sing-along, but if you were looking, for whatever reason, at high overtones while singing along, then you would.)

The spectrum display (blue above) works no matter how many sounds are present. When things are more complicated (say a singer with an instrument, or Peter, Paul, and Mary with several instruments), it gets a little difficult to figure out what's what. One hint is: (a) that a single note always shows the straight down pattern of the fundamental, then the first overtone one octave below, and the 3rd overtone one octave below that, plus occasionally the 7th overtone and 15th overtone straight down below those. Another hint, when you see it changing in time, is (b) that overtones from the same sounds move together. You can often pick this up. Finally, (c) when you have more time to look at real detail, the precise pattern of overtones from a single sound that may exist must be as in the screen-shot (except with additional overtones after the 12th), and using this information, you can often deduce additional sounds. (Frequently, while looking at string quartet recordings or the audio from MIDI files, after going to the lowest fundamental, I can deduce two other fundamentals down within the first two octaves of that first fundamental.)

The ear actually works much like the spectrogram. There is a long curled thing in the cochlea of the ear called the basilar membrane that vibrates at different sections corresponding to the fundamental and all overtones present. Several thousand nerves transmit information to the brain about what sections are vibrating. The brain then puts together single sounds basically by method (b) above. With my software, you won't be able to do as good a job at putting things together as your brain does with the ear, partly because my software doesn't respond as quickly as the basilar membrane and auditory nerves. But you get some idea what it does by looking at the software.

(NOTE: I have a slight oversimplification in the last paragraph. At the lowest frequencies, there is evidence that the brain may actually use, additionally to the information about WHERE the basilar membrane is vibrating, or solely, an actual COUNT of the vibrations. The count would be transmitted via neural firing frequency.)

For deaf folks with auditory nerves still intact, a cochlear implant actually works by taking something like the signals from a scattering of points across the spectrogram, and sending them to the appropriate nerves along the basilar membrane.

Some may think the spectrograms (whether from my program, or another spectrogram program) are defective because the overtones don't show on the graph in one sharp point, but rather are a little spread out. But that actually parallels what's going on in the ear on the basilar membrane -- which is again the data that the brain gets. The basilar membrane won't vibrate in just one precise point (even if fed a perfect sine wave), but will vibrate in a small zone, with the amount of vibration peaking at one point. The ear doesn't work by detecting absolutely precise sine waves -- it works by picking up where the basilar membrane is vibrating and figures out the pattern.

As a technical note, spectrograms work by breaking signals down to sine waves. One might wonder what is the significance of the sine waves -- why sines, isn't the choice by all those engineers of that mathematical sine shape arbitrary? No, it's not arbitrary. It works out that that's what the ear, at the basilar membrane, picks up. It vibrates at sections acording to what sines are present.

Oh, I need to explain the first plot down from the top. That is the autocorrelation used in the "single-pitch detection" algorithm. (The plot is split into halves for each device. At the exact moment I snapped the shot, the autocorrelation for device 1 was not on screen.) The autocorrelation is not that interesting, but I put it there because it can help you figure out when the single-pitch detection will work well, and make adjustments, or move your mic, to make it work well. (It will work well when the plots hits very near the top line in clear places -- rather than being kind of ambiguous. In this case, for device 2 it is hitting in 7 clear places -- and the single-pitch-detection algorithm is working well on that device. (For device 1 there was a similar situation, and, thus, the red detected-pitch arrow is showing, and agreeing with the fundamental in the spectrogram.)

Also, I may need to explain about the amount of each frequency that the spectrogram shows. It is in decibels (dB), which is on a "logarithmic scale". Every time you go up 10 dB, you multiply the sound has 10 times as much power. (If you go up 20 dB, the sound has 100x as much power, up 30dB, 1000x as much power). Now, how many dB of power range (i.e. ratio) the full range of my plot shows depends on the dynamic-range setting you make. In the picture its 74dB, so the range in the screen shot above runs over a bit more that 10,000,000 to 1 power ratio. From "eyeballing" it, it looks like the first overtone is about 30 dB down from the fundamental, and so the first overtone has about 1 one thousandth as much power. Some of the higher overtones are stronger than this. (The pattern of overtone strengths you get depends on the instrument and how it is played. The way these vary is perceived as "timbre".)


By the way, if you've downloaded the program, to move any of those 5 "sliders" (of which dynamic range is one), you just left-click on where you want to be on the slider.

Screenshot 3 (Device-Choice Panel)

The picture above is the device-selection panel. Exactly what text appears depends on what your input device maker and/or Microsoft decided to call each device.

In my case, "Line In/Mic In" is my sound-card, and by setting the volume/mixer for the sound card, I can get the sound from a CD, DVD audio, microphone, .wav file, audio portion of any audio/video file, midi output, or an electronic keyboard plugged into the input of my sound card -- anything I can hear on my computer.

"USB audio device" is the microphone of my web-cam. If you select this as device 2, and the sound card as device 1, you can check intonation as you sing along with the sound passing through the sound card in the web-cam. (Usually, while doing this, I have the sound card sound going into headphones rather than speakers, so that the web-cam microphone picks up just my singing.) [NOTE: As with my soundcard + web-cam configuration, one way to analyze something from a microphone simultaneously while something else is passing through the sound-card (and being analyzed separately from the mic sound or not not analyzed), you can use the microphone on the webcam and the webcam's own built in "sound card hardware" as the source for the microphone sound information. For this two-source analysis, you may also be able to not use a webcam at all, but set-up your soundcard to handle two separate sources at the same time, one of which is a standard microphone. I have found recently that I am able to do this on my Realtek soundcard, separating sound-card back-panel and front-panel sources into separate "sound input devices" by, within mixer--record--tool icon, clicking "enable multi-streaming". This lets me have use a better microphone than the web-cam microphone, as well.]

I also stuck in an "undersample". What this is is that, depending on what you're doing, and what tone-range you are looking at, you may not need to analyze based on every sound measurement that your devices give you. You can do the analysis on the sound with a few neighboring measurments averaged together, and speed the processing up (and look at more pictures per second of the sound).




MORE SCREENSHOTS AND APPLICATIONS OF THE SPECTRATUNE can be found on this page.


Psychoacoustic Note on Consonance and Dissonance: Psychoacoustic studies show that dissonance happens when music contains components (fundamental or overtones) that are close to each other (less than the "critical bandwidth"), but not virtually the same. (Thus, the basilar membrane is vibrating at certain points close together but not the same--and the auditory nervous system is getting such as "raw" data.) The critical bandwidth is about 3 half-steps when we are dealing with tones above about A880 = the A two octaves above A220, and a bit larger (about 100 hz -- a frequency-varying number of half-steps) below A880.


For Continuous-Variation Ear Training / Intonation Exercises: My new ToneGen software or else the Cognaxon Two-Channels Free Signal Generator

One ear-training and intonation exercise is to listen to continuously varying frequencies against a fixed note, and learn the exact perceptual sensation as we get near and on exact intervals. You can use your voice as the continuous sound source, or a non-fretted string instrument, of course, but you may find it simpler to use a signal generator or synthesizer that you can vary frequency continuously on.

One such program is: My ToneGen Software.

Another such program is: Cognaxon Two-Channels frequency generator. I will point out that to get fine enough variation in frequency, don't use the slider -- use the up and down arrows (or your mouse wheel). You will want to have the Spectratune on at the same time, to tell you the pitch without having to do a calculation, and you of course need a keyboard or something for the reference note.

Tips for Singing Along with a Recording with Tonal Guidance from the Spectratune

Start with simpler recordings first. I've had very good luck pitch-matching with Pete Seeger, who of course has simpler songs with simple accompaniment, designed for easy audience singing along. This setup has worked well: The Spectratune should be displaying in spiral (despite some early sing-along videos on the site from before I added the superior-for-sing-along spiral mode). Recording playing through sound-card on spectrogram (not-single-pitch) on Spectratune going to sing-alonger through headphones (open-ear best), with sing-along going through web-cam mic analyzed as single-pitch on Spectratune. You can usually find the key on a major-key piece, if you want Spectratune key overlay, by going to the last chord, which should be a tonic triad, (i.e. V-I cadence), which will be heaviest in the pattern: Tonic, 4 halfsteps up from that, 3 half-steps up from that: i.e. when set for the correct key, the Spectratune's heavy Tonic Mediant Dominant radiating rays will overlay the I chords tonal concentrations.

See also the section below about finding the vocalist in recordings.

To get the sing-along or sing-harmony skills in a more structured pace, with the ability to go slower than the pace of an audio recording, you can use TONAL VISION on any MIDI file with the interconnecting Spectratune pitch-monitor feature.

Tips to Find the Vocalist in Recordings or To Find Another Individual Instrument/Transcribe

When there is one sound (such as in a cappella singing), things are pretty easy. You can usually use the single-pitch detection, but even the spectrogram is fine -- you just go to the lowest-frequency blob of signal. (The lowest-frequency blob must be the fundamental, and not an overtone.)

When things are a little more complex, such as a singer with a few instruments, you can only use the spectral representation, and it is a little more complicated because you have all fundamentals and overtones of all instruments in the display. But it is often possible with a little practice, and for many recordings it is quite easy.

Your main way to distinguish which fundamentals and overtones come from the same source the same way as the human auditory system does it -- by looking for blobs that vary together. (They will both move to higher and lower tones, and higher and lower volumes, together.) In order to be able to do this, you have to be able to detect changes over time. This means you have to tweak the Spectratune to show a good number of spectral analyses a second. Maybe at least ten. So (a)use the lowest device sampling rate (say 11.025 khz if you have it), and then (b) try undersampling (say 2x). (c)Turn off the pitch detection, and possibly (d) don't use a second "sing along" device. Cut the sound amount/analysis down (I've been successful with around .05 seconds). (e)Get rid of the octaves you don't really need.

Besides the general varying together of single sound sources, note the general pattern of any one sound source will be: fundamental, then first overtone right underneath it, then 2nd overtone a fifth up from that, then 3rd overtone back in the horizontal line with the fundamental and first overtone, one line down from the first overtone, then other overtones becoming denser on each line. (See first screen-shot on the page -- exactly that pattern through the first 3 1/2 octaves -- what I have written in the last sentence applies when the Spectratune is set up as in the screen-shot, "up is down". When you check "up is up", as I did in my videos, go a row up instead of a row down.) You can pick out this pattern varying together as being a single sound source. Adjusting dynamic range and gain to de-emphasize irrelevancies will help you find the pattern, as well as bearing in mind the note range of the instrument in question. If you know, or can figure out, the key of the tune, superimposing that may also help.

Note that, if there is no loss of a sine due to a fidelity problem with the recording or microphone, you can always find the lowest note being played, as the lowest sine can't be an overtone. Once you find that lowest note, and see where its overtones should be, you may find something extra that is not an overtone of that first note you found. That must be from a 2nd note played by something. (It may not be the fundamental of that 2nd note -- the fundamental might just be mixed in with an overtone of the first note. But often, because of the pattern, you can deduce that it is the fundamental of the 2nd note.)

Also, note that if your are working from a stereo recording, you can often set the computer mixer card (i.e. from "volume control") to focus on the side with either more of the sound you are interested in, or less of an interfering sound. (That is, when you select the sound card as your "input device", Spectratune does a mono analysis on whatever comes out of the sound card to it.)

As people who have experience transcribing from recordings will tell you, its not easy, there is no simple rule, if there are two many instruments it becomes downright impossible -- if you make soup there is no guarantee you can separate the soup back into its constituent ingredients, etc. The computer helps, though, and perhaps using programs like Spectratune to transcribe will make it possible to see how to write computer programs that help people more and more with transcribing from recordings.

Tips to Tweak Execution Speed (i.e. rate of new frame display)

What you're doing with the spectratune (tuning an instrument, checking your voice intonation, looking at intonation of notes on a recording, studying timbre, etc.) will affect how you have things set.

In general, to speed things up (and show a quicker, "more responsive" picture-rate), you can either reduce the number of tasks being performed, or do an adjustment which in some sense reduces the detail of the analysis.

Reducing the number of tasks being done means switching off the analyses you're not really using. In my own screen shot at the top of the page, I have both the spectrogram and the single-pitch detection switched on. This was to make a nice demonstration picture of the program working. But, for faster frame rate, one of those tasks could be turned off. (If the sound contained more than one pitch, the single-pitch detection won't even work, so it of course should be the one switched off.)

Here are the adjustments that trade-off speed vs. detail. Note that in many cases, depending on what you're doing, a lot of speed can be gained with only an imperceptible loss in detail.

Other Tips

For most purposes, your webcam or headphone microphone, or another cheap microphone is adequate. However, whenever a modest microphone and/or modest speakers are involved, you often have quite imperfect frequency response, which will not move any fundamentals or overtones, but will make them show with more or less amplitude than they really have. The response may also roll off substantially at the lower or higher frequencies (even making frequencies there appear to not be there at all).

If you're doing something where this becomes a problem, and if you're looking at sound from a CD, the best way to avoid the combined microphone and speaker innacuracies is by playing the CD on your PC, and using the sound card as the input device. (Thus, everything happens within the sound card.)

Alternatively, you can play the CD on your stereo system, and run an audio cable from a low-level output to your sound card. (Young people: I am old and have never used an iPod, but I am sure you can do a direct feed to your sound card from it.)

If you're looking at the output of an electronic keyboard, similarly, if you need the greatest accuracy, run a sound cable from your keyboard to your sound card, and use the sound card as the audio input device. (In my screenshots, I did NOT do this--I used my webcam mic, but I do get more accurate results when I switch to the direct feed.)

If you need the best audio quality using a mic (say for singing or monitoring an instrument), an electrostatic mic for about $40 ought to do it. They run about $40. Typically, this is just needed if you want the overtones shown with the most accurate strengths.

In order to facilitate the above, a trip to your neighborhood Radio Shack for a few cables would be your best bet. (Don't forget the appropriate "Y" connectors so you can listen at the same time as you analyze!)

60 cycle hum / 50 cycle hum: I've noticed one of my microphones picks up a slight amount of 60 cycle hum, which is negligible compared to the size of my sound waves. In case you see this and wonder what it is, this is coming from the AC household power and imperfectly shielded wire. In the US, the AC household power is at 60 hz, and thus appears a little more than one half-step above the A 2 octaves below A220. In parts of Europe, you will have 50 cycle hum, which will be a little more than one half step below that same A. If it gets in the way of what you are doing, there are a lot of odd electrical engineering causes, and you can search on the internet for "60 cycle hum", "ground loops", etc., to see how to get rid of it.


For Psychoacoustic Fun: A Decent Free Companion Tool: UNESCO Sinusoidal Synthesizer

People interested particularly in the science of sound may want to look at various of your own combinations of pure sines. (For example, to explore the causes of consonance/dissonance, and perhaps determine your own dissonance critical bandwidth function.) Again, the reason for focusing on pure sines is that they are actually what the hardware in the ear picks up -- the basilar membrane vibrates in the positions of all the pure sines present.

I've discovered a decent free tool in the internet for generating pure sines. Its the UNESCO sponsored software WaveModeler. It's in French, but there is a manual in English. It generates one note and overtones at whatever level you choose. But, you can open the program again and generate a different note simultaneously, and experiment with consonance and dissonance. (I've checked that this simultaneous running does generate simultaneous notes -- they must have engineered that into the program). (You can simultaneously have the Spectratune open, and see all the sines being generated.) There seem to be a few bugs: I've noticed the synthesizer program crashes on me sometimes when I try to change the attack/decay/sustain/release parameters, but you don't need to change those features, and if you do, it seems the problem is just when you have at least one of them down to 0. More importantly, I've confirmed with an external chromatic tuner that the default settings on the UNESCO program are out of tune (under) by a little more than half a step. You have to use their tuning sliders to compensate for this. (It seems to be a bug, but their program is free and allows you to take sines in and out nicely.) Also, I noted the UNESCO page says you need to download Winzip to unzip their package, but nowadays that is built in to Windows XP and Vista.

I like using the UNESCO synthesis program because it shows off the veracity of the spectrogram's picking up sines. That is, that program allows you to separately turn on and off each of the first 16 overtones, and when you do, one by one they disappear from my spectrogram. (Be careful if you try this to note that the UNESCO harmonic sliders are 0 in the middle of the scale, not at the bottom. If you take them away from 0, and want to get them back to 0, enter 0 rather than trying to get them to zero by sliding. Also, if you're going to look at all those harmonics, you have to use Spectratune in its high-fi mode. That is, bring up device sampling to 44.1 khz, and no undersampling. See the picture at the BOTTOM of this page, where I generated a sound with a pattern of overtones not seen in nature.


Support for the claim that, at low frequencies, Spectratune shows Higher Precision

Try this .wav file,, of the almost sub-audible D near 36Hz (about 3 octaves below middle C -- pure sine) on your current spectral analysis software. Because it probably uses the Fast Fourier Transform, the resolution around this note will probably be very course -- perhaps each half-step will be divided into just two zones. Not so with the Spectratune, which has equal granularity throughout the scale, because I use a different algorithm. The Spectratune output is here:

.

----------------------------------------------- -----------------------------------------------

Questions:

norm@nastechservices.com,


MY OLD SITE (Spectrogram videos: Mostly classical Music)

Some of you coming to this page may not know about my old site, which might interest you. In about the year 2000 I made some spectral videos of certain classical music and put them on the web. They are like what the Spectratune does, except I use a spiral representation only. They play using Windows Media or Apple Quicktime viewers, so that they are not restricted to Windows OS folks. They were not done in real time. It took about 10 hours of processing for every hour of video. If you want to look at them, they are here. They are mostly clips of about half a minute, except one is about an hour.

The site continues to get a good number of viewers. I have been continually getting a good number of people viewing the clips each day. (They are from all over the world, and at the moment, 2007/2008, there is a spurt of popularity in China. This is a good sign, as China is an up-and-coming country!)

In the intervening years, computer speed has gotten higher, and as well I got a few ideas as to how to make my computations more efficient. So I've adapted the basic spectral video technique into something that runs in real time, and lets people look at whatever sounds or music they want to. Thus, the Spectratune.




TONAL VISION MUSIC COMPREHENSION TOOL (Interactively play and display MIDI files with chord-view octave overlay to aid comprehension of tonal aspects of the music, with optional overlaid pitch feedback for learning to sing along with the melody, or in harmony. )


Click here.


TONEGEN (A modest little tone generator allowing fine pitch variation for intonation perception exercises, and testing your friends at parties.)

Click here.


Ear Training Software:

I have, have built up my musical perception from, and recommend, EarMaster ear training software. These links, through Amazon, seem to be for the same product that I have: EarMaster 5. The prices are different: one through Amazon direct, one through a sub-vendor.

About me, Norm Spier: I am a free-lance mathematical statistician and computer programmer, living near Binghamton, New York, U.S.

My U.S. Health Insurance and Pre-Existing Conditions Site: , for US citizens (to know what peril they are in), and for others from developed democracies who just want a good laugh (at our expense I am afraid).

SUPPORT SPECTRATUNE AND RELATED ACOUSTICAL PROJECTS: DONATE HERE (suggested: 5 or 10 or Dollars or Euros for an appreciative user who finds the software useful). If you do make a donation via PayPal, why not send me a quick email at norm@nastechservices.com, so I can acknowledge you contribution, and take in any feedback you may have on the software?

Historical Psychoacoustics Book

Helmholtz, On The Sensations of Tone--A Physiological Basis for the Theory of Music

Distribution of Musical Talent:

Psychology of Musical Talent, Seashore

Classic Ear-Training Book (Public Domain):

Wedge, Ear Training and Sight Singing as Applied to the Study of Harmony