US6405166B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Multimedia search apparatus and method for searching multimedia content using speaker detection by audio data
Assignee: Nuance Communications Inc
•
Filing: 2001-10-15
•
Granted: 2002-06-11
•
Expires: 2021-10-15
Abstract
Abstract A multimedia search apparatus and method for searching multimedia content using speaker detection to segment the multimedia content. The multimedia search apparatus receives a search request from a user device. The search request identifies the target speaker for which the search is to be conducted. Based on the search request, the multimedia search apparatus retrieves multimedia content from a multimedia database. The multimedia search apparatus retrieves models, such as Gaussian Mixture Models (GMMs), from a model storage device, corresponding to the target speaker and background data. Based on the retrieved models, the multimedia search device searches the multimedia data of the multimedia content and segments the multimedia data. The segments are identified by calculating an average normalized score for a block of frames of the multimedia data and determining if the average normalized score for the block of frames exceeds one or more predetermined thresholds.
Inventors
- 👤 Qian Huang
- 👤 Ivan Magrin-Chagnolleau
- 👤 Sarangarajan Parthasarathy
- 👤 Aaron Edward Rosenberg
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-27 07:30
CPC Classifications
G; G10; G10L; G10L17/00
Claims Analysis
Total Claims: 36
Claim 1
Independent Claim
125 words
A method of segmenting multimedia data using audio information, comprising: receiving a search request identifying at least one target speaker; retrieving at least one model for the at least one target speaker; and segmenting the multimedia data into one or more target speaker segments and background segments based on feature vectors of the multimedia data and the at least one model for the at least one target speaker, wherein the step of segmenting comprises: reading a first block of frames of the multimedia data; determining a score for the first block of frames based on the at least one model for the at least one target speaker; and determining if the score for the first block of frames is above or below a first threshold.
Claim 2
Independent Claim
125 words
A method of segmenting multimedia data using audio information, comprising: receiving a search request identifying at least one target speaker; retrieving at least one model for the at least one target speaker; and segmenting the multimedia data into one or more target speaker segments and background segments based on feature vectors of the multimedia data and the at least one model for the at least one target speaker, wherein the step of segmenting comprises: reading a first block of frames of the multimedia data; determining a score for the first block of frames based on the at least one model for the at least one target speaker; and determining if the score for the first block of frames is above or below a first threshold.
Claim 3
Dependent Claim
60 words
The method of claim 1 , further comprising: identifying the first block of frames as part of a target speaker segment if the score for the block of frames is above the predetermined threshold; and identifying the first block of frames as part of a background segment if the score for the block of frames is below the predetermined threshold.
Claim 4
Dependent Claim
57 words
The method of claim 1 , further comprising: identifying a tentative start point of a target speaker segment if the score for the first block of frames is above the first threshold; and identifying a tentative end point of a target speaker segment if the score for the first block of frames is below the first threshold.
Claim 5
Dependent Claim
84 words
The method of claim 3 , further comprising: reading a second block of frames of the audio data; determining a score for the second block of frames based on the model for the target speaker; verifying the tentative start point of the target speaker segment if the score for the second block of frames is above a second threshold; and verifying the tentative end point of the target speaker segment if the score for the second block of frames is below a third threshold.
Claim 6
Dependent Claim
13 words
The method of claim 1 , wherein the score is a normalized score.
Claim 7
Dependent Claim
27 words
The method of claim 5 , wherein the normalized score is calculated based on the model for the target speaker and one or more background data models.
Claim 8
Dependent Claim
20 words
The method of claim 1 , wherein the score is an averaged normalized score for the first block of frames.
Claim 9
Dependent Claim
81 words
The method of claim 1 , further comprising: sending at least one of (a) at least a portion of the target speaker segments and (b) at least a portion of the background segments to a user device from which the search request was received to enable the user device to reproduce a multimedia presentation incorporating the at least one of (a) the at least a portion of target speaker segments and (b) the at least a portion of the background segments.
Claim 10
Dependent Claim
30 words
The method of claim 8 , wherein the user device is one of a computer, a wired telephone, a wireless telephone, a Web TV™ terminal, and a Personal Digital Assistant.
Claim 11
Dependent Claim
24 words
The method of claim 1 , wherein the at least one model for the at least one target speaker is a Gaussian Mixture Model.
Claim 12
Dependent Claim
25 words
The method of claim 1 , wherein the at least one model for the at least one target speaker is a vector quantization codebook model.
Claim 13
Dependent Claim
24 words
The method of claim 1 , wherein the at least one model for the at least one target speaker is a hidden Markov model.
Claim 14
Dependent Claim
47 words
The method of claim 1 , further comprising retrieving at least one model for background, wherein the step of segmenting includes segmenting the multimedia data into the one or more target speaker segments and the background segments based on the at least one model for the background.
Claim 15
Dependent Claim
20 words
The method of claim 13 , wherein the at least one model for the background is a Gaussian Mixture Model.
Claim 16
Dependent Claim
21 words
The method of claim 13 , wherein the at least one model for the background is a vector quantization codebook model.
Claim 17
Dependent Claim
20 words
The method of claim 13 , wherein the at least one model for the background is a hidden Markov model.
Claim 18
Dependent Claim
71 words
A user device that receives at least one of (a) at least a portion of the target speaker segments and (b) at least a portion of the background segments that are segmented by the method of claim 1 and reproduces a multimedia presentation incorporating the at least one of (a) the art least a portion of the target speaker segments and (b) the at least a portion of the background segments.
Claim 19
Dependent Claim
30 words
The user device of claim 17 , wherein the user device is one of a computer, a wired telephone, a wireless telephone, a WebTV™ terminal, and a Personal Digital Assistant.
Claim 20
Independent Claim
140 words
An apparatus that identifies segments of multimedia data for retrieval, comprising: a controller; a network interface; and a memory, wherein the controller receives a search request via the network interface identifying at least one target speaker, retrieves at least one model for the at least one target speaker from the memory, and segments the multimedia data into one or more target speaker segments and background segments based on feature vectors of the multimedia data and the at least one model for the at least one target speaker; wherein the controller segments the multimedia data by reading a first block of frames of the multimedia data, determining a score for the first block of frames based on the at least one model for the at least one target speaker, and determining if the score is above or below a first threshold.
Claim 21
Independent Claim
140 words
An apparatus that identifies segments of multimedia data for retrieval, comprising: a controller; a network interface; and a memory, wherein the controller receives a search request via the network interface identifying at least one target speaker, retrieves at least one model for the at least one target speaker from the memory, and segments the multimedia data into one or more target speaker segments and background segments based on feature vectors of the multimedia data and the at least one model for the at least one target speaker; wherein the controller segments the multimedia data by reading a first block of frames of the multimedia data, determining a score for the first block of frames based on the at least one model for the at least one target speaker, and determining if the score is above or below a first threshold.
Claim 22
Dependent Claim
51 words
The apparatus of claim 19 , wherein the controller identifies the first block of frames as part of a target speaker segment if the score is above the predetermined threshold and identifies the first block of frames as part of a background segment if the score is below the predetermined threshold.
Claim 23
Dependent Claim
46 words
The apparatus of claim 19 , wherein the controller identifies a tentative start point of a target speaker segment if the score is above the first threshold and identifies a tentative end point of a target speaker segment if the score is below the first threshold.
Claim 24
Dependent Claim
85 words
The apparatus of claim 21 , wherein the controller reads a second block of frames of the audio data, determines a score for the second block of frames based on the model for the target speaker, verifies the tentative start point of the target speaker segment if the score for the second block of frames is above a second threshold, and verifies the tentative end point of the target speaker segment if the score for the second block of frames is below a third threshold.
Claim 25
Dependent Claim
13 words
The apparatus of claim 19 , wherein the score is a normalized score.
Claim 26
Dependent Claim
27 words
The apparatus of claim 23 , wherein the normalized score is calculated based on the model for the target speaker and one or more background data models.
Claim 27
Dependent Claim
20 words
The apparatus of claim 19 , wherein the score is an averaged normalized score for the first block of frames.
Claim 28
Dependent Claim
81 words
The apparatus of claim 19 , wherein the controller sends at least one of (a) at least a portion of the target speaker segments and (b) at least a portion of the background segments to a user device from which the search request was received to enable the user device to reproduce a multimedia presentation incorporating the at least one of (a) the at least a portion of target speaker segments and (b) the at least a portion of background segments.
Claim 29
Dependent Claim
30 words
The apparatus of claim 26 , wherein the user device is one of a computer, a wired telephone, a wireless telephone, a Web TV™ terminal, and a Personal Digital Assistant.
Claim 30
Dependent Claim
24 words
The apparatus of claim 19 , wherein the at least one model for the at least one target speaker is a Gaussian Mixture Model.
Claim 31
Dependent Claim
25 words
The apparatus of claim 19 , wherein the at least one model for the at least one target speaker is a vector quantization codebook model.
Claim 32
Dependent Claim
24 words
The apparatus of claim 19 , wherein the at least one model for the at least one target speaker is a hidden Markov model.
Claim 33
Dependent Claim
43 words
The apparatus of claim 19 , wherein the controller retrieves at least one model for background and segments the multimedia data into the one or more target speaker segments and the background segments based on the at least one model for the background.
Claim 34
Dependent Claim
20 words
The apparatus of claim 31 , wherein the at least one model for the background is a Gaussian Mixture Model.
Claim 35
Dependent Claim
21 words
The apparatus of claim 31 , wherein the at least one model for the background is a vector quantization codebook model.
Claim 36
Dependent Claim
20 words
The apparatus of claim 31 , wherein the at least one model for the background is a hidden Markov model.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2005-06-10 | 2005-12-10 | 2006-06-11 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2009-06-10 | 2009-12-10 | 2010-06-11 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2013-06-10 | 2013-12-10 | 2014-06-11 | $7700.00 | Window Closed / Lapsed |