[go: up one dir, main page]

WO2007119221A2 - Method and apparatus for extracting musical score from a musical signal - Google Patents

Method and apparatus for extracting musical score from a musical signal Download PDF

Info

Publication number
WO2007119221A2
WO2007119221A2 PCT/IB2007/051378 IB2007051378W WO2007119221A2 WO 2007119221 A2 WO2007119221 A2 WO 2007119221A2 IB 2007051378 W IB2007051378 W IB 2007051378W WO 2007119221 A2 WO2007119221 A2 WO 2007119221A2
Authority
WO
WIPO (PCT)
Prior art keywords
musical
signal
instrument
notes
identifying
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IB2007/051378
Other languages
French (fr)
Other versions
WO2007119221A3 (en
Inventor
Pranav Singh
Sriram Krishnan
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
US Philips Corp
Original Assignee
Koninklijke Philips Electronics NV
US Philips Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics NV, US Philips Corp filed Critical Koninklijke Philips Electronics NV
Publication of WO2007119221A2 publication Critical patent/WO2007119221A2/en
Publication of WO2007119221A3 publication Critical patent/WO2007119221A3/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10HELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00Details of electrophonic musical instruments
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10HELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H7/00Instruments in which the tones are synthesised from a data store, e.g. computer organs
    • G10H7/08Instruments in which the tones are synthesised from a data store, e.g. computer organs by calculating functions or polynomial approximations to evaluate amplitudes at successive sample points of a tone waveform
    • G10H7/10Instruments in which the tones are synthesised from a data store, e.g. computer organs by calculating functions or polynomial approximations to evaluate amplitudes at successive sample points of a tone waveform using coefficients or parameters stored in a memory, e.g. Fourier coefficients
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10HELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/031Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
    • G10H2210/066Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10HELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/031Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
    • G10H2210/086Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for transcription of raw audio or music data to a displayed or printed staff representation or to displayable MIDI-like note-oriented data, e.g. in pianoroll format

Definitions

  • the present invention generally relates to music signal processing, and more specifically relates to extracting the musical score from a multi-track musical signal comprising multiple instruments.
  • FIG. 1 is an illustration of prior art representations of musical score.
  • One of the most popular forms of representing music is the staff and clef notation 101.
  • Other forms of standardized musical note-sequence representations include tablatures 102.
  • tablatures and other forms of musical scores for several pieces popular amongst musicians are available. These tablatures are created manually, mostly by skilled musicians, by listening to the pieces repeatedly and using tacit knowledge and music theory.
  • the musical notes directly correspond to fixed frequencies, for example the musical note A4 corresponds to 440Hz.
  • Spectral analysis can, in theory, be used to find out the frequency of single musical notes. Techniques based on this have been used to assist in tuning of instruments, where the signal is relatively clean and only a single musical note on a single musical instrument is being played. However, simple spectral analysis techniques cannot work in cases of noisy, multi-instrument and rapidly changing music signals, which is most often the case. [0004] Research on the extraction of musical note-sequences is being carried out at a few universities, notably, the University of Dortmund (http://www.statauer.uni-êtmund.de/sfb475/dienst/de/content/veroeff-d/tr04-d.html).
  • FIG.2 is an illustration of spectral analysis of a song carried out at uniform time intervals as in the prior art 201. This method involves fitting a curve through the highest density clusters obtained using spectral analysis of the musical signal.
  • US patent application publication no. 2004/0060424 describes a method for converting a music signal into a note -based signal and for referencing a music signal in a data bank. This published patent application aims at extracting the melody (vaguely defined as the dominant tune observed in a song by an individual) from a musical signal. Melody, however, is far from a complete description of a song and is actually one of the components of the musical score. For example, a song could also have a bass guitar and piano while an electric guitar plays the melody. In such cases, the method described in the above patent can also fail as the musical notes played on the bass and the piano can interfere and/or be mistaken for the melody notes.
  • the proposed invention makes use of an adaptive musical knowledgebase, which captures music theory and other tacit knowledge, in order to intelligently extract components from multi-instrument music pieces.
  • the note-sequence extracted from music signals is used to mine a music database significantly improving similarity calculation and facilitating search.
  • the method of present invention provides printable musical score of the musical signal which includes staff and tablature notations with beamed notes, dynamic marks, ornaments, velocity marks and accidentals.
  • a method of extracting musical note sequence from a musical signal includes the steps of receiving a replica (temporary working copy) of the musical signal, identifying musical notes and instrument frequency envelopes of the musical signal using spectral analysis and an adaptive knowledgebase, and generating a subtractive signal based on the musical notes and instrument frequency envelopes and subtracting the subtractive signal from the replica of the musical signal for identifying the musical note sequence.
  • a system for extracting musical note sequence from a musical signal.
  • the system includes a replica of the musical signal, a spectral analysis tool for identifying musical notes and instrument frequency envelopes of the musical signal, and an adaptive knowledgebase for identifying and storing the musical notes and instrument frequency envelopes.
  • the musical scales and musical instruments are identified using a musical note and instrument identifier module.
  • the musical note and instrument identifier module includes a filtering tool to identify the contribution of a particular musical instrument after identifying the musical notes.
  • a subtractive signal generation module is provided for generating a subtractive signal. The subtractive signal is subtracted from the replica of the musical signal.
  • a confidence level identifier module is provided for calculating confidence level of identification of musical notes and identifying signal component of highest confidence level.
  • the confidence level is identified using the power distribution of spectrum of the musical signal, and a corrective module is provided for making retrospective correction to the previously identified musical notes, musical scales, beat patterns based on a new identified confidence level.
  • a method of searching in a musical notation database includes the steps of receiving an incomplete musical signal from a user, extracting musical note sequence and musical characteristics from the incomplete musical signal using an adaptive knowledgebase and tacit knowledge, matching the musical note sequence and musical characteristics against musical score stored in the musical notation database, and displaying search results in a ranked and clustered manner.
  • FIG. 1 is an illustration of prior art representations of musical score.
  • FIG. 2 is an illustration of spectral analysis of a song carried out at uniform time intervals as in the prior art.
  • FIG. 3 is a block diagram representation of an embodiment of the system for extracting musical note sequence from a musical signal.
  • FIG. 4 is a block diagram representation of generating a subtractive signal and subtracting the subtractive signal from the temporary working copy of the music signal.
  • FIG. 5 illustrates a possible order of extraction of signal components from the musical signal according to the confidence levels.
  • FIG. 6 is a block diagram illustrating the identification of musical scales and musical instruments from the musical signal.
  • FIG. 7 is a flow diagram illustrating an embodiment of the method of searching in a musical notation database.
  • FIG. 8 displays the spectral analysis plot and the musical notes extracted from different time intervals of a musical signal in an experiment.
  • FIG. 9 is a graphical depiction showing the confidence level estimation of each musical note identified from the spectral analysis displayed in FIG. 8.
  • FIG. 10 displays the output musical note sequence of the musical signal in the extraction experiment.
  • FIG. 3 is a block diagram representation of an embodiment of the system for extracting musical note sequence from a musical signal.
  • a temporary copy 302 of the original music signal 301 is fed into the musical note & instrument identifier module 303 where spectral analysis is performed.
  • Music and tacit knowledge from the adaptive knowledgebase 305 is used to assist in the identification.
  • Tacit knowledge includes information about the group/individual performing the song, the genre, origin, era etc.
  • the knowledge of the musical scales, beats patterns and instruments usually used by the particular group/individual, other groups/individuals from the same genre or origin is also used to assist in disambiguation. For example, the musical scales and modes used in jazz music and rock music are different. So are the musical instruments and beat patterns.
  • the proposed invention uses the adaptive knowledgebase 305 seeded with appropriate music theory and tacit knowledge.
  • a musical piece generally follows a fixed scale (a subset of all musical notes possible) and deviations are infrequent, except in special cases like jazz music.
  • Information about the song's scale and modes are used for predicting or disambiguating musical notes that are difficult to estimate.
  • the musical notes and instrument frequency envelopes and waveforms are identified from the spectral analysis, with the adaptive knowledgebase 305 helping in disambiguation.
  • Each instrument has a characteristic signature in the form of frequency and time envelopes. For example, when the musical note A4 (440Hz) is played on an acoustic guitar, it also produces harmonics A5 (880Hz), A6 (1760) etc.
  • Relative amplitudes of the harmonics and other frequencies generated by the musical instrument form its frequency envelope and characterize its tonal qualities.
  • the secondary frequencies produced by the musical instrument interfere with the identification of other musical notes, possibly played on other musical instruments.
  • the knowledge of the frequency and time response of a musical instrument significantly helps in the identification of other components besides helping to identify the primary musical note being played (A4 in this case).
  • the confidence level of the identification of musical notes and instruments is also calculated (0.6 for G and 0.8 for E in FIG.3). Confidence level of each musical note is identified by a confidence level identifier module 304. The confidence level is calculated based on the power distribution in the signal spectrum and how well the musical note/instrument identified corresponds with the estimation of the current adaptive knowledgebase 305. A musical note that is played in isolation is likely to have a higher confidence level than a musical note played in the presence of other background notes or noise. For example, the musical note G would have been extracted with a much higher confidence if it is known that G is in the scale of the song while F# and G# are not.
  • a subtractive signal is generated from subtractive signal generation module 307.
  • This subtractive signal is subtracted from the temporary copy of the musical signal 302. This clears the signal spectrum and waveform, assisting in the identification of other components in the musical signal.
  • the method of this invention extracts the highest confidence components first. This approach clears the signal spectrum and waveform and can assist in the extraction of other musical notes with lower confidence.
  • the adaptive knowledgebase 305 aids in disambiguation in case of low-confidence components by capturing estimates of the musical scales, beat patterns, musical instruments etc. A new identification may reveal errors in the previous assumptions (musical scales, beat patterns etc.).
  • a corrective module 306 makes retrospective correction to the previously identified musical notes, instruments, scales and subtractive signals.
  • the output from the system gives printable musical score 308 of the musical signal which includes staff and tablature notations with beamed notes, dynamic marks, ornaments, velocity marks and accidentals.
  • FIG. 4 is a block diagram representation of generating a subtractive signal 403 and subtracting the subtractive signal 403 from the temporary copy 302 of the music signal.
  • the generation of subtractive signal 403 is explained under the description of the FIG. 3.
  • Instrumental music signals are well behaved when compared to speech or vocal music signals. It is possible to reconstruct the signal if one has information on the musical notes, the intensity of each musical note, the time and frequency characteristics of the instrument on which the musical note has been played and certain other information about the way the musical note has been played (E.g.: staccato, natural harmonic, grace notes, tied-notes etc.). Unlike vocal music, it is easier to characterize the musical instrument's time and frequency response.
  • Every musical instrument suppresses or accentuates harmonics to a characteristic degree, causes spread of power around the actual frequency played (in the power spectrum) and adds frequencies that were not played. These characteristics are captured by spectral estimation of musical notes played by the instrument (preferably when other instruments are not playing).
  • the subtractive techniques described above can also be extended to vocal music and speech extraction if sufficiently accurate vocal tract models for estimating the spectral characteristics of a person's voice are available.
  • a subtractive signal 403 is generated that estimates the contribution of the musical note to the spectrum and the waveform.
  • the subtractive signal 403 is generated based on the musical notes 401 and instruments identified 402 and subtracted from the temporary copy (replica) of the musical signal 302. This subtraction results in a residual signal 404.
  • the subtraction in the frequency domain clears the spectrum by removing or reducing the harmonics, false frequencies and help in a higher confidence identification of other musical notes.
  • Subtraction of the waveform from the original music signal, in time domain reduces the interference with musical notes played later or earlier. This is very useful in identifying musical notes that are played before the subtracted musical note dies out.
  • a much more powerful approach is to perform a time varying frequency subtraction.
  • the power spectrum of an instrument also varies with time. For example, the resonance with the musical instrument's natural resonant frequencies comes into effect gradually, causing a change in the spectrum.
  • the subtractive signal 403 is a series of frequency spectrums estimated at very short intervals.
  • the time varying frequency characteristics of an instrument is obtained by performing short term fast fourier transform (FFT) at short time intervals. This characteristic, along with the frequency of musical note played and other information on playing style (intensity, staccato, natural harmonics etc.) is used to generate an estimation of the time varying spectrum of the musical note played.
  • the subtractive signal 403 thus generated is subtracted from the time varying spectrum of the musical signal.
  • FIG. 5 illustrates a possible order of extraction of signal components from the musical signal according to the confidence levels 501.
  • the confidence level of each extractable component is calculated and the highest confidence level component is extracted.
  • Confidence level of identification of a musical note is estimated based on the 'clarity' of the power spectrum (the absence of other frequencies, especially nearby frequencies), and also on the presence or absence of its harmonics and sub-harmonics.
  • the window size of the short term FFT might be so small that a single actual musical note is spanned by multiple windows. In each of these windows, a musical note is identified. In some windows, this may be different from the actual musical note played. The number of windows that agree with a particular musical note determines the confidence level.
  • FIG. 6 is a block diagram illustrating the assistance of musical scale and instrument knowledge in identification of musical scales and musical instruments from the musical signal.
  • the musical notes identified by musical note identifier block 603 are stored in the adaptive knowledgebase 305. Based on these musical notes, the possibility of certain scales is eliminated.
  • the musical notes identified are C#, E, G, G# and B 601.
  • the probability of the scale being C-major is low as this scale does not have the musical notes C# and G#.
  • the confidence level of the identification of C# and G# was very low and they were in reality C and G respectively. So the probability of C-major scale cannot be entirely ruled out. Instead the confidence level with which the musical notes have been identified is calculated and the probability of a scale being the actual scale of the song is estimated. So the probability of C-major scale might have a very low but finite probability.
  • a list of possible scales with corresponding probabilities is maintained and continually updated in the adaptive knowledgebase 305.
  • the probability of the scale being "Blues in C#” is 58% and the probability of "Bhairav (a musical scale in Indian classical music) in C" is 23%.
  • the calculation of probability of a scale can also take into account any available tacit knowledge like the origin of the song, the era and the genre. For example, for Indian songs, there is higher probability that the scale is "Bhairav in C".
  • the scale probability list is used to help in disambiguation in cases where two nearby musical notes like G and G# have comparable spectral power. After the extraction of each note, the probabilities of various possible scales are updated.
  • the confidence of extraction depends not only on the power of the musical note in the spectrum but also on its harmonics. Sometimes when the power of the musical notes is comparable, the musical note actually being played is determined by checking the harmonics produced. To be more general, the confidence level of a musical note is estimated by checking whether the actual spectrum of the musical signal matches with the spectrum predicted by looking at the frequency response of the instrument at that particular note.
  • the adaptive knowledgebase 305 has a library of known instruments with their probability being extracted from the frequency of occurrences and other tacit knowledge.
  • the waveform and frequency envelope information is also available.
  • the probability that an instrument is being played depends on the instrument waveforms and frequency response extracted till now, the genre, region and other tacit knowledge.
  • spectral estimation methods are used to focus on the contribution of the particular instrument.
  • a rough waveform is extracted and then matched against the waveforms of various instruments. The degree of match is used in conjunction with the instrument probability list in order to identify the instrument 602.
  • the instrument identifier block 604 matches the rough waveform against instrument probability list stored in the adaptive knowledgebase 305.
  • the instrument probability list is updated based on the instrument identified. In FIG. 6, the probability of the musical instrument being an acoustic steel guitar is 83%.
  • the frequency response and waveform of the instrument is updated with the newly obtained information.
  • FIG. 7 is a flow diagram illustrating an embodiment of the method of searching in a musical notation database.
  • User inputs an incomplete music signal for the purpose of searching the database 701. This could be in the form of a tune whistled/hummed to the system, an instrument played or music notation of a song.
  • Music notation has to be extracted if the input is a tune whistled/hummed and when the musical instrument is playing 702.
  • a pitch and tempo shifter is used to make corrections in the pitch and tempo of the input musical signal. In the case of a musical notation as input, the above mentioned techniques are not needed.
  • the user may provide information about the artist, genre, and instruments played in the song, the specific instrument for which the user provided the signal and other tacit information.
  • the musical notation (extracted or provided) is used to match against notations stored in the musical notation database 703.
  • the extraction algorithm For each musical signal from the song database, the extraction algorithm extracts musical note sequence, instruments, tempo, scale etc. Musical features such as usage of staccato in the song, usage of harmonics, tempo variations (accelerando, rallentando) in the song and other dynamic variations in intensity/notes and range of notes are also extracted. In addition, the tacit knowledge available such as artist name, year, genre, country and origin of artist and instruments typically used is also added.
  • the song database is clustered according to the various features. Various existing pattern matching techniques could be employed for this purpose. This in conjunction with the matching of additional features gives a method for ranking the songs that are found to closely match the song user is interested in searching for. The results are ranked and/or provided in a clustered form 704. The user can further narrow down the search results by selecting or eliminating various choices for different features.
  • FIG. 8 displays the spectral analysis plot and the musical notes extracted from different time intervals of a musical signal in an experiment 801.
  • the musical signal used in the experiment included an acoustic guitar and a bass guitar playing simultaneously. For each time interval, two musical notes were extracted.
  • the higher confidence musical note in a time interval is displayed on the top and the lower confidence level musical note below. All the high confidence musical notes were found to be correct.
  • the low confidence musical notes in the 2 nd and 3 rd intervals have a false identification (G and E). In these time intervals, the acoustic guitar and the bass guitar were playing the same musical notes, albeit on different octaves.
  • FIG. 9 is a graphical depiction showing the confidence level estimation of each musical note identified from the spectral analysis displayed in FIG. 8, 901.
  • the confidence level of each musical note is calculated based on the presence of its harmonics.
  • the 15 th , 20 th , 30 th and 33 rd musical notes in FIG. 9 have a much lower confidence level because their harmonics are not present.
  • the 25 th and 37 th musical notes have the highest confidence levels.
  • FIG. 10 displays the output musical note sequence of the musical signal in extraction experiment 1001.
  • the extraction experiment was able to extract correct musical notes and musical instruments with dynamics, time signature and tempo indications.
  • the output of the extraction was incorrect in the case of the 2 nd and 3 rd notes played on the bass guitar. This is not due to a flaw in the technique but due to an attempt to forcefully extract two unique musical notes in each time interval.
  • the acoustic guitar and the bass guitar play the same musical note at different octaves in the 2 nd and the 3 rd interval and therefore an attempt to forcefully extract another unique musical note returns a false musical note.
  • the present invention will find its applications in searching song databases, extracting user profiles, calculating similarities of songs and clustering similar songs. These applications are extremely useful for search engines like GoogleTM and Yahoo!®, e-shopping sites like AmazonTM and digital audio player manufacturers like PhilipsTM, CreativeTM and Apple®. [0043] While the present invention has been described with reference to several particular example embodiments, those skilled in the art will recognize that many changes may be made thereto without departing from the spirit and scope of the present invention, which is set forth in the following claims.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Multimedia (AREA)
  • Acoustics & Sound (AREA)
  • General Physics & Mathematics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Analysis (AREA)
  • Algebra (AREA)
  • Auxiliary Devices For Music (AREA)
  • Electrophonic Musical Instruments (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present invention provides a system and a method for extracting musical note sequence from a musical signal by incorporating music theory and instrument knowledge. The method includes receiving a replica of a musical signal, identifying musical notes and instrument frequency envelopes of the musical signal by means of spectral analysis and an adaptive knowledgebase, generating a subtractive signal based on the musical notes and instrument frequency envelopes, and subtracting the subtractive signal from the replica of the musical signal. The note-sequence extracted from music signals is used to mine a music database significantly improving search and similarity calculation. The method of present invention provides printable musical score of the musical signal which includes staff and tablature notations with beamed notes, dynamic marks, ornaments, velocity marks and accidentals.

Description

METHOD AND APPARATUS FOR EXTRACTING MUSICAL SCORE
FROM A MUSICAL SIGNAL
FIELD OF THE INVENTION
[0001] The present invention generally relates to music signal processing, and more specifically relates to extracting the musical score from a multi-track musical signal comprising multiple instruments.
BACKGROUND
[0002] For many centuries music has been accurately represented using different notations, also called the musical score. FIG. 1 is an illustration of prior art representations of musical score. One of the most popular forms of representing music is the staff and clef notation 101. Other forms of standardized musical note-sequence representations include tablatures 102. As of today, tablatures and other forms of musical scores for several pieces popular amongst musicians are available. These tablatures are created manually, mostly by skilled musicians, by listening to the pieces repeatedly and using tacit knowledge and music theory. The musical notes directly correspond to fixed frequencies, for example the musical note A4 corresponds to 440Hz.
[0003] Spectral analysis can, in theory, be used to find out the frequency of single musical notes. Techniques based on this have been used to assist in tuning of instruments, where the signal is relatively clean and only a single musical note on a single musical instrument is being played. However, simple spectral analysis techniques cannot work in cases of noisy, multi-instrument and rapidly changing music signals, which is most often the case. [0004] Research on the extraction of musical note-sequences is being carried out at a few universities, notably, the University of Dortmund (http://www.statistik.uni- dortmund.de/sfb475/dienst/de/content/veroeff-d/tr04-d.html). The methods described in the publications from University of Dortmund, http://www.sfb475.uni- dortmund.de/berichte/tr77-04.pdfand http://www.sfb475.uni-dortmund.de/berichte/tr39- 05.pdf, use time-series techniques for pitch extraction. However, they do not make use of instrument characteristics and music theory to generate confidence levels. They also use statistical techniques such as Hidden Markov Model (HMM) etc.
[0005] FIG.2 is an illustration of spectral analysis of a song carried out at uniform time intervals as in the prior art 201. This method involves fitting a curve through the highest density clusters obtained using spectral analysis of the musical signal. US patent application publication no. 2004/0060424 describes a method for converting a music signal into a note -based signal and for referencing a music signal in a data bank. This published patent application aims at extracting the melody (vaguely defined as the dominant tune observed in a song by an individual) from a musical signal. Melody, however, is far from a complete description of a song and is actually one of the components of the musical score. For example, a song could also have a bass guitar and piano while an electric guitar plays the melody. In such cases, the method described in the above patent can also fail as the musical notes played on the bass and the piano can interfere and/or be mistaken for the melody notes.
[0006] Commercial products are available for automatic transcription of music (e.g. AmazingMidi (http://www.pluto.dti.ne.jp/~araki/amazingmidi), Audio to score (http://www.emagic.de) and Widi (http://www.widisoft.com)). These commercially available soft wares typically track fundamental frequency (melody for instance) and essentially attempt to create a musical instrument digital interface (MIDI) sequence estimating the dominant melody. They often use multiple notes (which are often not even in the same scale) to estimate single musical notes, the overall feeling of which is often similar to the single musical note that was played. This is suitable for making ring tones.
[0007] Hence, it would be advantageous to provide a system and a method for extracting exact musical notes from multi-track music pieces. The present invention has been developed to meet these needs in the art.
SUMMARY OF THE INVENTION
[0008] The proposed invention makes use of an adaptive musical knowledgebase, which captures music theory and other tacit knowledge, in order to intelligently extract components from multi-instrument music pieces. The note-sequence extracted from music signals is used to mine a music database significantly improving similarity calculation and facilitating search. The method of present invention provides printable musical score of the musical signal which includes staff and tablature notations with beamed notes, dynamic marks, ornaments, velocity marks and accidentals.
[0009] In an example embodiment of the present invention, a method of extracting musical note sequence from a musical signal is provided. The method includes the steps of receiving a replica (temporary working copy) of the musical signal, identifying musical notes and instrument frequency envelopes of the musical signal using spectral analysis and an adaptive knowledgebase, and generating a subtractive signal based on the musical notes and instrument frequency envelopes and subtracting the subtractive signal from the replica of the musical signal for identifying the musical note sequence.
[0010] In another example embodiment of the present invention, a system is provided for extracting musical note sequence from a musical signal. The system includes a replica of the musical signal, a spectral analysis tool for identifying musical notes and instrument frequency envelopes of the musical signal, and an adaptive knowledgebase for identifying and storing the musical notes and instrument frequency envelopes. The musical scales and musical instruments are identified using a musical note and instrument identifier module. The musical note and instrument identifier module includes a filtering tool to identify the contribution of a particular musical instrument after identifying the musical notes. A subtractive signal generation module is provided for generating a subtractive signal. The subtractive signal is subtracted from the replica of the musical signal. A confidence level identifier module is provided for calculating confidence level of identification of musical notes and identifying signal component of highest confidence level. The confidence level is identified using the power distribution of spectrum of the musical signal, and a corrective module is provided for making retrospective correction to the previously identified musical notes, musical scales, beat patterns based on a new identified confidence level.
[0011] In another example embodiment of the present invention, a method of searching in a musical notation database is provided. The method includes the steps of receiving an incomplete musical signal from a user, extracting musical note sequence and musical characteristics from the incomplete musical signal using an adaptive knowledgebase and tacit knowledge, matching the musical note sequence and musical characteristics against musical score stored in the musical notation database, and displaying search results in a ranked and clustered manner.
[0012] The above summary of the present invention is not intended to represent each disclosed embodiment, or every aspect, of the present invention. Other aspects and example embodiments are provided in the figures and the detailed description that follows. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The invention may be more completely understood in consideration of the following detailed description of various embodiments of the invention in connection with the accompanying drawings, in which:
[0014] FIG. 1 is an illustration of prior art representations of musical score.
[0015] FIG. 2 is an illustration of spectral analysis of a song carried out at uniform time intervals as in the prior art.
[0016] FIG. 3 is a block diagram representation of an embodiment of the system for extracting musical note sequence from a musical signal.
[0017] FIG. 4 is a block diagram representation of generating a subtractive signal and subtracting the subtractive signal from the temporary working copy of the music signal.
[0018] FIG. 5 illustrates a possible order of extraction of signal components from the musical signal according to the confidence levels.
[0019] FIG. 6 is a block diagram illustrating the identification of musical scales and musical instruments from the musical signal.
[0020] FIG. 7 is a flow diagram illustrating an embodiment of the method of searching in a musical notation database.
[0021] FIG. 8 displays the spectral analysis plot and the musical notes extracted from different time intervals of a musical signal in an experiment.
[0022] FIG. 9 is a graphical depiction showing the confidence level estimation of each musical note identified from the spectral analysis displayed in FIG. 8. [0023] FIG. 10 displays the output musical note sequence of the musical signal in the extraction experiment.
[0024] While the invention is amenable to various modifications and alternative forms, specifics thereof have been shown by way of example in the drawings and will be described in detail. It should be understood, however, that the intention is not to limit the invention to the particular embodiments described. On the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims.
DETAILED DESCRIPTION
[0025] FIG. 3 is a block diagram representation of an embodiment of the system for extracting musical note sequence from a musical signal. A temporary copy 302 of the original music signal 301 is fed into the musical note & instrument identifier module 303 where spectral analysis is performed. Musical and tacit knowledge from the adaptive knowledgebase 305 is used to assist in the identification. Tacit knowledge includes information about the group/individual performing the song, the genre, origin, era etc. The knowledge of the musical scales, beats patterns and instruments usually used by the particular group/individual, other groups/individuals from the same genre or origin is also used to assist in disambiguation. For example, the musical scales and modes used in jazz music and rock music are different. So are the musical instruments and beat patterns. Similarly, there is a significant difference in the musical instruments, musical scales and beats used in western classical, Indian classical and African songs.
[0026] The proposed invention uses the adaptive knowledgebase 305 seeded with appropriate music theory and tacit knowledge. A musical piece generally follows a fixed scale (a subset of all musical notes possible) and deviations are infrequent, except in special cases like jazz music. Information about the song's scale and modes are used for predicting or disambiguating musical notes that are difficult to estimate. The musical notes and instrument frequency envelopes and waveforms are identified from the spectral analysis, with the adaptive knowledgebase 305 helping in disambiguation. Each instrument has a characteristic signature in the form of frequency and time envelopes. For example, when the musical note A4 (440Hz) is played on an acoustic guitar, it also produces harmonics A5 (880Hz), A6 (1760) etc. Relative amplitudes of the harmonics and other frequencies generated by the musical instrument form its frequency envelope and characterize its tonal qualities. The secondary frequencies produced by the musical instrument interfere with the identification of other musical notes, possibly played on other musical instruments. The knowledge of the frequency and time response of a musical instrument significantly helps in the identification of other components besides helping to identify the primary musical note being played (A4 in this case).
[0027] The confidence level of the identification of musical notes and instruments is also calculated (0.6 for G and 0.8 for E in FIG.3). Confidence level of each musical note is identified by a confidence level identifier module 304. The confidence level is calculated based on the power distribution in the signal spectrum and how well the musical note/instrument identified corresponds with the estimation of the current adaptive knowledgebase 305. A musical note that is played in isolation is likely to have a higher confidence level than a musical note played in the presence of other background notes or noise. For example, the musical note G would have been extracted with a much higher confidence if it is known that G is in the scale of the song while F# and G# are not.
[0028] Based on the musical note and musical instruments identified, a subtractive signal is generated from subtractive signal generation module 307. This subtractive signal is subtracted from the temporary copy of the musical signal 302. This clears the signal spectrum and waveform, assisting in the identification of other components in the musical signal. The method of this invention extracts the highest confidence components first. This approach clears the signal spectrum and waveform and can assist in the extraction of other musical notes with lower confidence. The adaptive knowledgebase 305 aids in disambiguation in case of low-confidence components by capturing estimates of the musical scales, beat patterns, musical instruments etc. A new identification may reveal errors in the previous assumptions (musical scales, beat patterns etc.). A corrective module 306 makes retrospective correction to the previously identified musical notes, instruments, scales and subtractive signals. The output from the system gives printable musical score 308 of the musical signal which includes staff and tablature notations with beamed notes, dynamic marks, ornaments, velocity marks and accidentals.
[0029] FIG. 4 is a block diagram representation of generating a subtractive signal 403 and subtracting the subtractive signal 403 from the temporary copy 302 of the music signal. The generation of subtractive signal 403 is explained under the description of the FIG. 3. Instrumental music signals are well behaved when compared to speech or vocal music signals. It is possible to reconstruct the signal if one has information on the musical notes, the intensity of each musical note, the time and frequency characteristics of the instrument on which the musical note has been played and certain other information about the way the musical note has been played (E.g.: staccato, natural harmonic, grace notes, tied-notes etc.). Unlike vocal music, it is easier to characterize the musical instrument's time and frequency response. Every musical instrument suppresses or accentuates harmonics to a characteristic degree, causes spread of power around the actual frequency played (in the power spectrum) and adds frequencies that were not played. These characteristics are captured by spectral estimation of musical notes played by the instrument (preferably when other instruments are not playing). However, the subtractive techniques described above can also be extended to vocal music and speech extraction if sufficiently accurate vocal tract models for estimating the spectral characteristics of a person's voice are available.
[0030] Based on the musical note identified, the manner in which the musical note was played (intensity, dynamics, natural harmonics etc.) and the instrument used to play the musical note, a subtractive signal 403 is generated that estimates the contribution of the musical note to the spectrum and the waveform. The subtractive signal 403 is generated based on the musical notes 401 and instruments identified 402 and subtracted from the temporary copy (replica) of the musical signal 302. This subtraction results in a residual signal 404. The subtraction in the frequency domain clears the spectrum by removing or reducing the harmonics, false frequencies and help in a higher confidence identification of other musical notes. Subtraction of the waveform from the original music signal, in time domain reduces the interference with musical notes played later or earlier. This is very useful in identifying musical notes that are played before the subtracted musical note dies out.
[0031] A much more powerful approach is to perform a time varying frequency subtraction. The power spectrum of an instrument also varies with time. For example, the resonance with the musical instrument's natural resonant frequencies comes into effect gradually, causing a change in the spectrum. In this case the subtractive signal 403 is a series of frequency spectrums estimated at very short intervals. The time varying frequency characteristics of an instrument is obtained by performing short term fast fourier transform (FFT) at short time intervals. This characteristic, along with the frequency of musical note played and other information on playing style (intensity, staccato, natural harmonics etc.) is used to generate an estimation of the time varying spectrum of the musical note played. The subtractive signal 403 thus generated is subtracted from the time varying spectrum of the musical signal.
[0032] FIG. 5 illustrates a possible order of extraction of signal components from the musical signal according to the confidence levels 501. The confidence level of each extractable component is calculated and the highest confidence level component is extracted. Confidence level of identification of a musical note is estimated based on the 'clarity' of the power spectrum (the absence of other frequencies, especially nearby frequencies), and also on the presence or absence of its harmonics and sub-harmonics. The window size of the short term FFT might be so small that a single actual musical note is spanned by multiple windows. In each of these windows, a musical note is identified. In some windows, this may be different from the actual musical note played. The number of windows that agree with a particular musical note determines the confidence level. Working in this manner, extraction is not necessarily carried out at the beginning of the musical signal or in a pre-defined sequential manner. A possible order of extraction of signal components according to the confidence levels is shown in FIG.5. The numerals in the figure (1, 2, 3, 4, and 5) indicate the order of extraction.
[0033] FIG. 6 is a block diagram illustrating the assistance of musical scale and instrument knowledge in identification of musical scales and musical instruments from the musical signal. The musical notes identified by musical note identifier block 603 are stored in the adaptive knowledgebase 305. Based on these musical notes, the possibility of certain scales is eliminated. In FIG. 6, the musical notes identified are C#, E, G, G# and B 601. The probability of the scale being C-major is low as this scale does not have the musical notes C# and G#. However, it is possible that the confidence level of the identification of C# and G# was very low and they were in reality C and G respectively. So the probability of C-major scale cannot be entirely ruled out. Instead the confidence level with which the musical notes have been identified is calculated and the probability of a scale being the actual scale of the song is estimated. So the probability of C-major scale might have a very low but finite probability.
[0034] A list of possible scales with corresponding probabilities is maintained and continually updated in the adaptive knowledgebase 305. In FIG. 6, the probability of the scale being "Blues in C#" is 58% and the probability of "Bhairav (a musical scale in Indian classical music) in C" is 23%. The calculation of probability of a scale can also take into account any available tacit knowledge like the origin of the song, the era and the genre. For example, for Indian songs, there is higher probability that the scale is "Bhairav in C". The scale probability list is used to help in disambiguation in cases where two nearby musical notes like G and G# have comparable spectral power. After the extraction of each note, the probabilities of various possible scales are updated.
[0035] The confidence of extraction depends not only on the power of the musical note in the spectrum but also on its harmonics. Sometimes when the power of the musical notes is comparable, the musical note actually being played is determined by checking the harmonics produced. To be more general, the confidence level of a musical note is estimated by checking whether the actual spectrum of the musical signal matches with the spectrum predicted by looking at the frequency response of the instrument at that particular note.
[0036] The adaptive knowledgebase 305 has a library of known instruments with their probability being extracted from the frequency of occurrences and other tacit knowledge. The waveform and frequency envelope information is also available. The probability that an instrument is being played depends on the instrument waveforms and frequency response extracted till now, the genre, region and other tacit knowledge. After the identification of the musical note, spectral estimation methods are used to focus on the contribution of the particular instrument. A rough waveform is extracted and then matched against the waveforms of various instruments. The degree of match is used in conjunction with the instrument probability list in order to identify the instrument 602. The instrument identifier block 604 matches the rough waveform against instrument probability list stored in the adaptive knowledgebase 305. The instrument probability list is updated based on the instrument identified. In FIG. 6, the probability of the musical instrument being an acoustic steel guitar is 83%. The frequency response and waveform of the instrument is updated with the newly obtained information.
[0037] FIG. 7 is a flow diagram illustrating an embodiment of the method of searching in a musical notation database. User inputs an incomplete music signal for the purpose of searching the database 701. This could be in the form of a tune whistled/hummed to the system, an instrument played or music notation of a song. Music notation has to be extracted if the input is a tune whistled/hummed and when the musical instrument is playing 702. There is a recording device provided to receive the signal input. Errors in pitch and tempo are likely in the case of voice input. A pitch and tempo shifter is used to make corrections in the pitch and tempo of the input musical signal. In the case of a musical notation as input, the above mentioned techniques are not needed. In addition to this, the user may provide information about the artist, genre, and instruments played in the song, the specific instrument for which the user provided the signal and other tacit information. The musical notation (extracted or provided) is used to match against notations stored in the musical notation database 703.
[0038] For each musical signal from the song database, the extraction algorithm extracts musical note sequence, instruments, tempo, scale etc. Musical features such as usage of staccato in the song, usage of harmonics, tempo variations (accelerando, rallentando) in the song and other dynamic variations in intensity/notes and range of notes are also extracted. In addition, the tacit knowledge available such as artist name, year, genre, country and origin of artist and instruments typically used is also added. The song database is clustered according to the various features. Various existing pattern matching techniques could be employed for this purpose. This in conjunction with the matching of additional features gives a method for ranking the songs that are found to closely match the song user is interested in searching for. The results are ranked and/or provided in a clustered form 704. The user can further narrow down the search results by selecting or eliminating various choices for different features.
[0039] FIG. 8 displays the spectral analysis plot and the musical notes extracted from different time intervals of a musical signal in an experiment 801. The musical signal used in the experiment included an acoustic guitar and a bass guitar playing simultaneously. For each time interval, two musical notes were extracted. In FIG. 8, the higher confidence musical note in a time interval is displayed on the top and the lower confidence level musical note below. All the high confidence musical notes were found to be correct. The low confidence musical notes in the 2nd and 3 rd intervals have a false identification (G and E). In these time intervals, the acoustic guitar and the bass guitar were playing the same musical notes, albeit on different octaves.
[0040] FIG. 9 is a graphical depiction showing the confidence level estimation of each musical note identified from the spectral analysis displayed in FIG. 8, 901. The confidence level of each musical note is calculated based on the presence of its harmonics. Thus, the 15th, 20th, 30th and 33rd musical notes in FIG. 9 have a much lower confidence level because their harmonics are not present. And the 25th and 37th musical notes have the highest confidence levels.
[0041] FIG. 10 displays the output musical note sequence of the musical signal in extraction experiment 1001. The extraction experiment was able to extract correct musical notes and musical instruments with dynamics, time signature and tempo indications. The output of the extraction was incorrect in the case of the 2 nd and 3 rd notes played on the bass guitar. This is not due to a flaw in the technique but due to an attempt to forcefully extract two unique musical notes in each time interval. The acoustic guitar and the bass guitar play the same musical note at different octaves in the 2nd and the 3rd interval and therefore an attempt to forcefully extract another unique musical note returns a false musical note.
APPLICATION OF THE INVENTION
[0042] The present invention will find its applications in searching song databases, extracting user profiles, calculating similarities of songs and clustering similar songs. These applications are extremely useful for search engines like Google™ and Yahoo!®, e-shopping sites like Amazon™ and digital audio player manufacturers like Philips™, Creative™ and Apple®. [0043] While the present invention has been described with reference to several particular example embodiments, those skilled in the art will recognize that many changes may be made thereto without departing from the spirit and scope of the present invention, which is set forth in the following claims.

Claims

[0044] CLAIMSWhat is claimed is:
1. A method of extracting musical note sequence from a musical signal comprising the steps of:
receiving a replica of said musical signal;
identifying musical notes and instrument frequency envelopes of the musical signal using spectral analysis and an adaptive knowledgebase; and
generating a subtractive signal based on said musical notes and instrument frequency envelopes and subtracting said subtractive signal from said replica of the musical signal for identifying said musical note sequence.
2. The method as in claim 1, wherein said identifying musical scale further comprising the steps of:
storing said musical notes and said instrument frequency envelopes in said adaptive knowledgebase;
calculating a confidence level of identification of musical notes, said confidence level comprises power distribution of spectrum of the musical signal, and matching of actual spectrum of the musical note identified with predicted spectrum by referring the frequency response of the musical instrument at the particular musical note stored in the adaptive knowledgebase;
extracting signal components of the musical signal in the order of descending confidence levels; and
calculating probability of occurrence of a particular musical scale by means of musical notes identified and tacit knowledge.
3. The method as in claim 1, wherein said identifying musical instrument further comprising the steps of:
maintaining and updating a list of known musical instruments and an instrument probability list in the adaptive knowledgebase, said instrument probability list comprises list of musical instruments based on the frequency of occurrence and said tacit knowledge;
calculating the probability of occurrence of a particular musical instrument based on tacit knowledge and the instrument waveforms and frequency envelopes; and
extracting a rough waveform of the musical signal and matching said rough waveform against waveforms of various musical instruments; and
using degree of said matching in conjunction with the instrument probability list for identifying the musical instruments.
4. The method of claim 1, wherein said adaptive knowledgebase comprises updated knowledgebase of musical scales, musical theory concepts, beat patterns, musical instrument characteristics, tacit knowledge and statistical knowledge.
5. The method of claim 4, wherein the tacit knowledge comprises information about the group or individual performing the song, genre, origin of the song, era, and also knowledge of musical scales, beat patterns and musical instruments used by said group or individual performing the song.
6. The method of claim 1, wherein said subtracting the subtractive signal from the replica of the musical signal comprises subtracting in frequency domain and clearing the spectrum by removing the harmonics, whereby identifying other musical notes in the musical signal.
7. The method of claim 1, wherein said subtracting the subtractive signal from the replica of the musical signal comprises subtracting in time domain, whereby reducing the interferences of other musical notes in the musical signal.
8. The method of claim 1, wherein said musical signal comprises musical signal produced from individual instruments, combined instruments, a plurality of musical instruments played simultaneously and multi track musical pieces.
9. The method of claim 1, wherein output of the method comprises printable musical score of the musical signal which includes staff and tablature notations with beamed notes, dynamic marks, ornaments, velocity marks and accidentals.
10. A system for extracting musical note sequence from a musical signal comprising of:
a replica of said musical signal;
a spectral analysis tool for identifying musical notes and instrument frequency envelopes of the musical signal; and
an adaptive knowledgebase for identifying and storing said musical notes and instrument frequency envelopes.
11. The system as in claim 10, further comprising of:
a musical note and instrument identifier module for identifying musical scales and musical instruments constituting the musical signal, said musical note and instrument identifier module further comprising a filtering tool for identifying the contribution of a particular musical instrument after identifying the musical notes;
a subtractive signal generation module for generating a subtractive signal and subtracting said subtractive signal from said replica of the musical signal;
a confidence level identifier module for calculating confidence level of identification of musical notes and identifying signal component of highest confidence level, said confidence level comprises power distribution of spectrum of the musical signal; a corrective module for making retrospective correction to the previously identified musical notes, musical scales, beat patterns based on a new identified confidence level.
12. The system as in claim 10, wherein said adaptive knowledgebase comprises constantly updated knowledgebase of musical scales, musical theory concepts, beat patterns, musical instrument characteristics, tacit knowledge and statistical knowledge.
13. The system as in claim 12, wherein said tacit knowledge comprises information about the group or individual performing the song, genre, origin of the song, era, and also knowledge of musical scales, beat patterns and musical instruments used by said group or individual performing the song.
14. The system as in claim 10, wherein said subtracting the subtractive signal from the replica of the musical signal, comprises subtracting in frequency domain and clearing the spectrum by removing the harmonics, whereby identifying other musical notes in the musical signal.
15. The system as in claim 10, wherein said subtracting the subtractive signal from the replica of the musical signal, comprises subtracting in time domain whereby reducing the interferences of other musical notes in the musical signal.
16. The system as in claim 10, wherein said musical signal comprises musical signal produced from individual instruments, combined instruments, a plurality of musical instruments played simultaneously and multi track musical pieces.
17. The system as in claim 10, wherein output of the method comprises printable musical score of the musical signal which includes staff and tablature notations with beamed notes, dynamic marks, ornaments, velocity marks and accidentals.
18. A method of searching in a musical notation database comprising the steps of:
receiving an incomplete musical signal from a user;
extracting musical note sequence and musical characteristics from said incomplete musical signal using an adaptive knowledgebase and tacit knowledge;
matching said musical note sequence and musical characteristics against musical score stored in said musical notation database; and
displaying search results in a ranked and clustered manner.
18. The method of claim 18, wherein said receiving an incomplete musical signal comprises voice input, a signal from a musical instrument and musical notation.
19. The method of claim 18, wherein said extracting musical note sequence and musical characteristics comprises the steps of: identifying musical scale, musical instruments providing an input; and receiving additional tacit information comprising information about the group or individual performing the song, genre, origin of the song, era from said user.
PCT/IB2007/051378 2006-04-18 2007-04-17 Method and apparatus for extracting musical score from a musical signal Ceased WO2007119221A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US79326306P 2006-04-18 2006-04-18
US60/793,263 2006-04-18

Publications (2)

Publication Number Publication Date
WO2007119221A2 true WO2007119221A2 (en) 2007-10-25
WO2007119221A3 WO2007119221A3 (en) 2007-12-27

Family

ID=38421518

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2007/051378 Ceased WO2007119221A2 (en) 2006-04-18 2007-04-17 Method and apparatus for extracting musical score from a musical signal

Country Status (1)

Country Link
WO (1) WO2007119221A2 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102008013172A1 (en) * 2008-03-07 2009-09-10 Neubäcker, Peter Method for sound-object-oriented analysis and notation-oriented processing of polyphonic sound recordings
CN108986841A (en) * 2018-08-08 2018-12-11 百度在线网络技术(北京)有限公司 Audio-frequency information processing method, device and storage medium
CN110675849A (en) * 2019-09-11 2020-01-10 东北大学 A method for generating Bossa Nova style music rhythm based on Bayesian network
CN112183658A (en) * 2020-10-14 2021-01-05 小叶子(北京)科技有限公司 Music score identification method and device, electronic equipment and storage medium
CN112784099A (en) * 2021-01-29 2021-05-11 山西大学 Sampling counting audio retrieval method resisting tonal modification interference

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5210366A (en) * 1991-06-10 1993-05-11 Sykes Jr Richard O Method and device for detecting and separating voices in a complex musical composition
DE102004049457B3 (en) * 2004-10-11 2006-07-06 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Method and device for extracting a melody underlying an audio signal

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102008013172A1 (en) * 2008-03-07 2009-09-10 Neubäcker, Peter Method for sound-object-oriented analysis and notation-oriented processing of polyphonic sound recordings
DE102008013172B4 (en) * 2008-03-07 2010-07-08 Neubäcker, Peter Method for sound-object-oriented analysis and notation-oriented processing of polyphonic sound recordings
US8022286B2 (en) 2008-03-07 2011-09-20 Neubaecker Peter Sound-object oriented analysis and note-object oriented processing of polyphonic sound recordings
CN108986841A (en) * 2018-08-08 2018-12-11 百度在线网络技术(北京)有限公司 Audio-frequency information processing method, device and storage medium
CN110675849A (en) * 2019-09-11 2020-01-10 东北大学 A method for generating Bossa Nova style music rhythm based on Bayesian network
CN110675849B (en) * 2019-09-11 2022-11-15 东北大学 Method for generating Bossa Nova style music rhythm based on Bayesian network
CN112183658A (en) * 2020-10-14 2021-01-05 小叶子(北京)科技有限公司 Music score identification method and device, electronic equipment and storage medium
CN112183658B (en) * 2020-10-14 2024-01-26 小叶子(北京)科技有限公司 Music score identification method and device, electronic equipment and storage medium
CN112784099A (en) * 2021-01-29 2021-05-11 山西大学 Sampling counting audio retrieval method resisting tonal modification interference
CN112784099B (en) * 2021-01-29 2022-11-11 山西大学 Sampling counting audio retrieval method resisting tonal modification interference

Also Published As

Publication number Publication date
WO2007119221A3 (en) 2007-12-27

Similar Documents

Publication Publication Date Title
Lee et al. Acoustic chord transcription and key extraction from audio using key-dependent HMMs trained on synthesized audio
Muller et al. Signal processing for music analysis
CN100527222C (en) A device for analyzing music using the sound of an instrument
Durrieu et al. Source/filter model for unsupervised main melody extraction from polyphonic audio signals
US20100198760A1 (en) Apparatus and methods for music signal analysis
EP3929921B1 (en) Melody detection method for audio signal, device, and electronic apparatus
CN101116134A (en) Information processing apparatus, method, and program
US20050234366A1 (en) Apparatus and method for analyzing a sound signal using a physiological ear model
Bittner et al. Multitask learning for fundamental frequency estimation in music
Benetos et al. Automatic transcription of Turkish microtonal music
JP2012506061A (en) Analysis method of digital music sound signal
WO2007119221A2 (en) Method and apparatus for extracting musical score from a musical signal
Heydarian Automatic recognition of Persian musical modes in audio musical signals
Scherbaum et al. Tuning systems of traditional Georgian singing determined from a new corpus of field recordings
CN116034421A (en) Musical composition analysis device and musical composition analysis method
CN105244021B (en) The conversion method of humming melody to MIDI melody
Holzapfel et al. The sousta corpus: Beat-informed automatic transcription of traditional dance tunes
Dobre et al. Automatic music transcription software based on constant Q transform
Lee A system for automatic chord transcription from audio using genre-specific hidden Markov models
Dittmar et al. A toolbox for automatic transcription of polyphonic music
Chuan et al. The KUSC classical music dataset for audio key finding
Cazau et al. An automatic music transcription system dedicated to the repertoires of the marovany zither
Gulati A tonic identification approach for Indian art music
Koduri et al. Computational approaches for the understanding of melody in carnatic music
Duggan et al. Compensating for expressiveness in queries to a content based music information retrieval system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07735523

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 07735523

Country of ref document: EP

Kind code of ref document: A2