US6345252B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Methods and apparatus for retrieving audio information using content and speaker information
Assignee: Nuance Communications Inc
•
Filing: 1999-04-09
•
Granted: 2002-02-05
•
Expires: 2019-04-09
Abstract
Abstract Methods and apparatus are provided for retrieving audio information based on the audio content as well as the identity of the speaker. The results of content and speaker-based audio information retrieval methods are combined to provide references to audio information (and indirectly to video). A query search system retrieves information responsive to a textual query containing a text string (one or more key words), and the identity of a given speaker. An indexing system transcribes and indexes the audio information to create time-stamped content index file(s) and speaker index file(s). An audio retrieval system uses the generated content and speaker indexes to perform query-document matching based on the audio content and the speaker identity. Documents satisfying the user-specified content and speaker constraints are identified by comparing the start and end times of the document segments in both the content and speaker domains. Documents satisfying the user-specified content and speaker constraints are assigned a combined score that can be used in accordance with the present invention to rank-order the identified documents returned to the user, with the best-matched segments at the top of the list.
Inventors
- 👤 Homayoon Sadr Mohammad Beigi
- 👤 Alain Charles Louis Tritschler
- 👤 Mahesh Viswanathan
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-24 16:34
CPC Classifications
G; G10; G10L; G10L15/00; G10L15/26; G06; G06F; G06F16/00; G06F16/60; G06F16/68; G06F16/683; G06F16/685; G10L17/00
Claims Analysis
Total Claims: 37
Claim 1
Independent Claim
56 words
A method for retrieving audio information from one or more audio sources, said method comprising the steps of: receiving a user query specifying at least one content and one speaker constraint; and comparing said user query with a content index and a speaker index of said audio source to identify audio information satisfying said user query.
Claim 2
Independent Claim
56 words
A method for retrieving audio information from one or more audio sources, said method comprising the steps of: receiving a user query specifying at least one content and one speaker constraint; and comparing said user query with a content index and a speaker index of said audio source to identify audio information satisfying said user query.
Claim 3
Dependent Claim
42 words
The method of claim 1 , wherein said content index and said speaker index are time-stamped and said comparing step further comprises the step of comparing the start and end times of the document segments in both the content and speaker domains.
Claim 4
Dependent Claim
20 words
The method of claim 1 , wherein said content index includes the frequency of each word in said audio source.
Claim 5
Dependent Claim
23 words
The method of claim 1 , wherein said content index includes the inverse document frequency (IDF) of each word in said audio source.
Claim 6
Dependent Claim
17 words
The method of claim 1 , wherein said content index includes the length of said audio source.
Claim 7
Dependent Claim
24 words
The method of claim 1 , wherein said content index includes a set of chain linked pointers to each document containing a given word.
Claim 8
Dependent Claim
26 words
The method of claim 1 , wherein said speaker index includes a score indicating the distance from an enrolled speaker model to the audio test segment.
Claim 9
Dependent Claim
20 words
The method of claim 1 , wherein said speaker index includes the start and end times of each audio segment.
Claim 10
Dependent Claim
20 words
The method of claim 1 , wherein said speaker index includes a label identifying the speaker associated with the segment.
Claim 11
Dependent Claim
31 words
The method of claim 1 , wherein said comparing step further comprises the step of comparing documents satisfying the content-based query with documents satisfying the speaker-based query to identify relevant documents.
Claim 12
Dependent Claim
26 words
The method of claim 1 , further comprising the step of transcribing and indexing said audio source to create said content index and said speaker index.
Claim 13
Dependent Claim
35 words
The method of claim 11 , wherein said step of creating said speaker index comprises the steps of automatically detecting turns in said audio source and assigning a speaker label to each of said turns.
Claim 14
Dependent Claim
24 words
The method of claim 1 , further comprising the step of returning at least a portion of said identified audio information to a user.
Claim 15
Dependent Claim
37 words
The method of claim 1 , further comprising the step of assigning a combined score to each segment of said identified audio information and returning at least a portion of said identified audio information in a ranked-list.
Claim 16
Dependent Claim
22 words
The method of claim 14 , wherein said combined score evaluates the extent of the overlap between the content and speaker domains.
Claim 17
Dependent Claim
20 words
The method of claim 14 , wherein said combined score evaluates a ranked document score ranking the content-based information retrieval.
Claim 18
Dependent Claim
26 words
The method of claim 14 , wherein said combined score evaluates a speaker segment score measuring the proximity between a speaker segment and enrolled speaker information.
Claim 19
Dependent Claim
16 words
The method of claim 1 , wherein said speaker constraint includes the identity of a speaker.
Claim 20
Dependent Claim
15 words
The method of claim 1 , wherein said content constraint includes one or more keywords.
Claim 21
Dependent Claim
98 words
An audio retrieval system for retrieving audio information from one or more audio sources, comprising: a memory that stores a content index and a speaker index of said audio source and computer-readable code; and a processor operatively coupled to said memory, said processor configured to implement said computer-readable code, said computer-readable code configured to: receive a user query specifying one or more words and the identity of a speaker; and combine the results of a content-based and a speaker-based audio information retrieval to provide references to said audio source based on the audio content and the speaker identity.
Claim 22
Dependent Claim
98 words
An audio retrieval system for retrieving audio information from one or more audio sources, comprising: a memory that stores a content index and a speaker index of said audio source and computer-readable code; and a processor operatively coupled to said memory, said processor configured to implement said computer-readable code, said computer-readable code configured to: receive a user query specifying one or more words and the identity of a speaker; and combine the results of a content-based and a speaker-based audio information retrieval to provide references to said audio source based on the audio content and the speaker identity.
Claim 23
Dependent Claim
42 words
The audio retrieval system of claim 20 , wherein said content index and said speaker index are time-stamped and said processor is further configured to compare the start and end times of the document segments in both the content and speaker domains.
Claim 24
Dependent Claim
22 words
The audio retrieval system of claim 20 , wherein said content index includes the frequency of each word in said audio source.
Claim 25
Dependent Claim
25 words
The audio retrieval system of claim 20 , wherein said content index includes the inverse document frequency (IDF) of each word in said audio source.
Claim 26
Dependent Claim
28 words
The audio retrieval system of claim 20 , wherein said speaker index includes a score indicating the distance from an enrolled speaker model to the audio test segment.
Claim 27
Dependent Claim
22 words
The audio retrieval system of claim 20 , wherein said speaker index includes a label identifying the speaker associated with the segment.
Claim 28
Dependent Claim
31 words
The audio retrieval system of claim 20 , wherein said processor is further configured to compare documents satisfying the content-based query with documents satisfying the speaker-based query to identify relevant documents.
Claim 29
Dependent Claim
30 words
The audio retrieval system of claim 20 , wherein said processor is further configured to transcribe and index said audio source to create said content index and said speaker index.
Claim 30
Dependent Claim
41 words
The audio retrieval system of claim 20 , wherein said processor is further configured to assign a combined score to each segment of said identified audio information and return at least a portion of said identified audio information in a ranked-list.
Claim 31
Dependent Claim
24 words
The audio retrieval system of claim 28 , wherein said combined score evaluates the extent of the overlap between the content and speaker domains.
Claim 32
Dependent Claim
22 words
The audio retrieval system of claim 29 , wherein said combined score evaluates a ranked document score ranking the content-based information retrieval.
Claim 33
Dependent Claim
28 words
The audio retrieval system of claim 29 , wherein said combined score evaluates a speaker segment score measuring the proximity between a speaker segment and enrolled speaker information.
Claim 34
Dependent Claim
84 words
An article of manufacture for retrieving audio information from one or more audio sources, comprising: a computer readable medium having computer readable program code means embodied thereon, said computer readable program code means comprising: a step to receive a user query specifying one or more words and the identity of a speaker; and a step to combine the results of a content-based and a speaker-based audio information retrieval to provide references to said audio source based on the audio content and the speaker identity.
Claim 35
Dependent Claim
84 words
An article of manufacture for retrieving audio information from one or more audio sources, comprising: a computer readable medium having computer readable program code means embodied thereon, said computer readable program code means comprising: a step to receive a user query specifying one or more words and the identity of a speaker; and a step to combine the results of a content-based and a speaker-based audio information retrieval to provide references to said audio source based on the audio content and the speaker identity.
Claim 36
Dependent Claim
78 words
An article of manufacture for retrieving audio information from one or more audio sources, comprising: a computer readable medium having computer readable program code means embodied thereon, said computer readable program code means comprising: a step to receive a user query specifying at least one content and one speaker constraint; and a step to compare said user query with a content index and a speaker index of said audio source to identify audio information satisfying said user query.
Claim 37
Dependent Claim
78 words
An article of manufacture for retrieving audio information from one or more audio sources, comprising: a computer readable medium having computer readable program code means embodied thereon, said computer readable program code means comprising: a step to receive a user query specifying at least one content and one speaker constraint; and a step to compare said user query with a content index and a speaker index of said audio source to identify audio information satisfying said user query.
References Cited (Backward Citations)
US6185527B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2005-02-04 | 2005-08-06 | 2006-02-05 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2009-02-04 | 2009-08-06 | 2010-02-05 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2013-02-04 | 2013-08-06 | 2014-02-05 | $7700.00 | Window Closed / Lapsed |