Re: Audio to pitch is not a solved problem. Don't expect it to work for general cases anytime soon!

From: Amara Graps (amara@amara.com)
Date: Sat May 27 2000 - 13:50:01 MDT


From: Eirikur Hallgrimsson <eh@mad.scientist.com>

>I see a lot of people pointing optimistically to some of the roadkill
>on the way to the fantasy of being able to hum and hear trumpets.
>It's almost all roadkill. It'll happen eventually, but it will be
>the result of using on the order of four different "pitch-guessing"
>processes, and voting. And to get that to work will require
>recognizing the instrument used as input.

I tend to agree, but I have not followed the music technology in
the last several years, so I don't know the current state-of-the-art.

Three years ago I gave a really fun seminar to a group of smart
folks who were interested in learning about wavelets; they had
the ultimate goal of wanting to use wavelets for mixing instruments
in music (for example, symphonic music). They wanted to analyze
the sound of an orchestra, and arrive at a score for each
instrument, then re-score it, and play it back.

Since I was just giving a "beginners" seminar, and I'm not a wavelet
researcher, myself, I couldn't help them with their ultimate goal of
isolating musical instruments with wavelets. But I did give their
idea some thought, and I looked into the wavelets literature a
little bit for them.

I believe that the wavelets could do very well with isolating individual
syllables (as in speech), and wavelets are exceptional at isolating
octaves of frequencies to remove, denoise, etc.

However to identify all trumpet sounds, for example, one would
need to combine wavelets with a pattern recognition scheme, like
with a neural net or some other classifying algorithm.

One response to your message said that this problem has been "solved".
If that's true, then I'd be very interested in the technical details.

Here is an abstract that I found 3 years ago that gave me some
hints that the wavelet researchers were actively working on the problem.
(I don't know the results of this particular piece of research.)

Have fun!

Amara

P.S. for those of you who have no idea about what are wavelets, you
can go here: http://www.amara.com/current/wavelet.html

-------------------
http://www.spie.org/web/abstracts/2800/2825.html
SPIE Proceedings Vol. 2825
Wavelet Applications in Signal and Image Processing IV
$134
ISBN: 0-8194-2213-4, 1044 pages. Published 1996
Meeting Date: 08/04 - 08/09/96, Denver, CO, USA

Editor(s): Michael A. Unser, National Institutes of Health, Bethesda, MD,
USA; Akram Aldroubi,
National Institutes of Health, Bethesda, MD, USA; Andrew F. Laine, Univ. of
Florida,
Gainesville, FL, USA.

Paper #: 2825-98
Wavelets in music analysis and synthesis: timbre analysis and
perspectives, pp.950-961
Author(s): Regis R. Alves Faria, Univ. of Sao Paulo, Sao Paulo,
            Brazil;
            Ruggero A. Ruschioni, Univ. of Sao Paulo,
            Sao Paulo, SP, Brazil;
            Joao A. Zuffo, Univ. of Sao Paulo, Sao Paulo,
            Brazil.

Abstract: Music is a vital element in the process of
       comprehending the world where we live and interact
       with. Frequently it exerts a subtle but expressive
       influence over a society's evolution line. Analysis and
       synthesis of music and musical instruments has always
       been associated with forefront technologies available
       at each period of human history, and there is no
       surprise in witnessing now the use of digital
       technologies and sophisticated mathematical tools
       supporting its development. Fourier techniques have
       been employed for years as a tool to analyze timbres'
       spectral characteristics, and re-synthesize them from
       these extracted parameters. Recently many modern
       implementations, based on spectral modeling techniques,
       have been leading to the development of new generations
       of music synthesizers, capable of reproducing natural
       sounds with high fidelity, and producing novel timbres
       as well. Wavelets are a promising tool on the
       development of new generations of music synthesizers,
       counting on its advantages over the Fourier techniques
       in representing non-periodic and transient signals,
       with complex fine textures, as found in music. In this
       paper we propose and introduce the use of wavelets
       addressing its perspectives towards musical
       applications. The central idea is to investigate the
       capacities of wavelets in analyzing, extracting
       features and altering fine timbre components in a
       multiresolution time- scale, so as to produce high
       quality synthesized musical sounds.

********************************************************************
Amara Graps email: amara@amara.com
Computational Physics vita: finger agraps@shell5.ba.best.com
Multiplex Answers URL: http://www.amara.com/
********************************************************************
"It works better if you plug it in." -- Sattinger's Law



This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:28:51 MST