Patent US2454339A successfully refreshed from Google Patents.
Patent US7219054B1 successfully refreshed from Google Patents.
US7219054B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Systems and methods for generating an annotation guide
Assignee: Nuance Communications Inc
•
Filing: 2003-04-04
•
Granted: 2007-05-15
•
Expires: 2023-04-04
Abstract
Abstract Systems and methods for generating an annotation guide. Speech data is organized and presented to a user. After the user selects some of the utterances in the speech data, the selected utterances are included in a class and/or call type. Additional utterances that belong to the class and/or call type can be found in the speech data using relevance feedback, data mining, data clustering, support vector machines, and the like. After a call type is complete, it is committed to the annotation guide. After all call types are completed, the annotation guide is generated.
Inventors
- 👤 Lee Begeja
- 👤 Harris Drucker
- 👤 David Crawford Gibbon
- 👤 Allen Louis Gorin
- 👤 Patrick Guy Haffner
- 👤 Steven H Lewis
- 👤 Zhu Liu
- 👤 Mazin G Rahim
- 👤 Bernard S. Renger
- 👤 Behzad Shahraray
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-01 08:10
CPC Classifications
G; G10; G10L; G10L15/00; G10L15/22; G10L15/08; G10L15/18; G10L15/1822
Claims Analysis
Total Claims: 37
Claim 1
Independent Claim
89 words
A method for generating an annotation guide, wherein the annotation guide is used by labelers to annotate speech data, the method comprising: analyzing speech data to present a user with a plurality of utterances; receiving user-selected utterances from the plurality of utterances, the user-selected utterances associated with a call type; and analyzing the speech data to identify additional utterances that are similar to the user-selected utterances, wherein the identified additional utterances and the user-selected utterances are used to generate an annotation guide used by labelers to annotate speech data.
Claim 2
Independent Claim
89 words
A method for generating an annotation guide, wherein the annotation guide is used by labelers to annotate speech data, the method comprising: analyzing speech data to present a user with a plurality of utterances; receiving user-selected utterances from the plurality of utterances, the user-selected utterances associated with a call type; and analyzing the speech data to identify additional utterances that are similar to the user-selected utterances, wherein the identified additional utterances and the user-selected utterances are used to generate an annotation guide used by labelers to annotate speech data.
Claim 3
Dependent Claim
86 words
A method as defined in claim 1 , wherein analyzing speech data to present a user with a plurality of utterances further comprises one or more of: organizing the speech data into clusters, wherein a particular cluster is selected by the user and displayed to the user and wherein the particular cluster contains the plurality of utterances; searching the speech data using a string or a variant of the string to identify the plurality of utterances; and categorizing the speech data using a data mining module.
Claim 4
Dependent Claim
37 words
A method as defined in claim 1 , wherein analyzing the speech data to find additional utterances that are similar to the user-selected utterances further comprises using the user-selected utterances as feedback to identify the additional utterances.
Claim 5
Dependent Claim
97 words
A method as defined in claim 1 , wherein analyzing the speech data to identify additional utterances that are similar to the user-selected utterances further comprises at least one of: identifying at least one utterance to be included in the call type; identifying at least one utterance to be excluded from the call type; identifying the additional utterances using relevance feedback; identifying the additional utterances using a string search; identifying the additional utterances using data clustering; mining the speech data to identify the additional utterances; and using active learning to identify marginal utterances in the speech data.
Claim 6
Dependent Claim
23 words
A method as defined in claim 1 , further comprising selecting some of the additional utterances to be included in the call type.
Claim 7
Dependent Claim
18 words
A method as defined in claim 5 , further comprising committing the call type to an annotation guide.
Claim 8
Dependent Claim
26 words
A method as defined in claim 6 , further comprising: generating additional call types to be included in the annotation guide; and generating the annotation guide.
Claim 9
Independent Claim
96 words
A method for generating an annotation guide, wherein the annotation guide is used by labelers to annotate speech data, the method comprising: presenting a user with a plurality of utterances from speech data; receiving one or more user-selected utterances from the plurality of utterances, wherein the user-selected utterances are in a working class; identifying additional utterances in the speech data that are similar to the user-selected utterances, wherein some of the additional utterances are included in the working class; and committing the working class to an annotation guide used by labelers to annotate the speech data.
Claim 10
Independent Claim
96 words
A method for generating an annotation guide, wherein the annotation guide is used by labelers to annotate speech data, the method comprising: presenting a user with a plurality of utterances from speech data; receiving one or more user-selected utterances from the plurality of utterances, wherein the user-selected utterances are in a working class; identifying additional utterances in the speech data that are similar to the user-selected utterances, wherein some of the additional utterances are included in the working class; and committing the working class to an annotation guide used by labelers to annotate the speech data.
Claim 11
Dependent Claim
30 words
A method as defined in claim 8 , wherein presenting a user with a plurality of utterances from the speech data further comprises clustering the speech data into N clusters.
Claim 12
Dependent Claim
27 words
A method as defined in claim 8 , wherein presenting a user with a plurality of utterances further comprises performing a string search on the speech data.
Claim 13
Dependent Claim
33 words
A method as defined in claim 8 , wherein presenting a user with a plurality of utterances further comprises mining the speech data to retrieve a group of utterances from the speech data.
Claim 14
Dependent Claim
27 words
A method as defined in claim 8 , wherein presenting a user with a plurality of utterances further comprises displaying the plurality of utterances to the user.
Claim 15
Dependent Claim
83 words
A method as defined in claim 8 , wherein receiving one or more user-selected utterances from the plurality of utterances further comprises one or more of: identifying at least one utterance to be included in the working class; identifying at least one utterance to be excluded from the working class; identifying at least one utterance to be included in the annotation guide as a positive example; and identifying at least one utterance to be included in the annotation guide as a negative example.
Claim 16
Dependent Claim
33 words
A method as defined in claim 8 , wherein receiving one or more user-selected utterances from the plurality of utterances, further comprises using the user-selected utterances as feedback to identify the additional utterances.
Claim 17
Dependent Claim
26 words
A method as defined in claim 8 , wherein committing the working class to an annotation guide further comprises providing a description of the working class.
Claim 18
Dependent Claim
14 words
A method as defined in claim 8 , further comprising generating the annotation guide.
Claim 19
Dependent Claim
15 words
A computer program product having computer-executable instructions for performing the method of claim 8 .
Claim 20
Independent Claim
127 words
A method for generating an annotation guide for use in annotating speech data, wherein the annotation guide includes one or more call types, the method comprising: creating a call type for inclusion in an annotation guide, wherein creating a call type comprises: retrieving a plurality of utterances from speech data, wherein the plurality of utterances are displayed to a user; identifying user-selected utterances from the plurality of utterances, wherein the user-selected utterances are included in the call type; identifying additional utterances from the speech data based on the user-selected utterances, wherein some of the additional utterances are included in the call type; and committing the call type to the annotation guide; and generating the annotation guide, wherein the annotation guide includes at least each committed call type.
Claim 21
Independent Claim
127 words
A method for generating an annotation guide for use in annotating speech data, wherein the annotation guide includes one or more call types, the method comprising: creating a call type for inclusion in an annotation guide, wherein creating a call type comprises: retrieving a plurality of utterances from speech data, wherein the plurality of utterances are displayed to a user; identifying user-selected utterances from the plurality of utterances, wherein the user-selected utterances are included in the call type; identifying additional utterances from the speech data based on the user-selected utterances, wherein some of the additional utterances are included in the call type; and committing the call type to the annotation guide; and generating the annotation guide, wherein the annotation guide includes at least each committed call type.
Claim 22
Dependent Claim
31 words
A method as defined in claim 18 , wherein creating a call type further comprises including a description of the call type, wherein the description is included in the annotation guide.
Claim 23
Dependent Claim
23 words
A method as defined in claim 18 , wherein retrieving a plurality of utterances from speech data further comprises analyzing the speech data.
Claim 24
Dependent Claim
63 words
A method as defined in claim 20 , wherein analyzing the speech data further comprises at least one of: clustering the speech data into N clusters; performing relevance feedback using the user-selected utterances; searching the speech data for utterances that include a particular string or a variant of the particular string; and mining the speech data to identify a group of related utterances.
Claim 25
Dependent Claim
37 words
A method as defined in claim 20 , wherein identifying user-selected utterances from the plurality of utterances further comprises using the selected one or more utterances as feedback to identify the additional utterances from the speech data.
Claim 26
Dependent Claim
39 words
A method as defined in claim 20 , wherein identifying user-selected utterances from the plurality of utterances further comprises identifying at least one utterance to be included in the annotation guide for the call type as a positive example.
Claim 27
Dependent Claim
39 words
A method as defined in claim 20 , wherein identifying user-selected utterances from the plurality of utterances further comprises identifying at least one utterance to be included in the annotation guide for the call type as a negative example.
Claim 28
Dependent Claim
18 words
A method as defined in claim 20 , further comprising iteratively retrieving additional utterances based on selected utterances.
Claim 29
Dependent Claim
28 words
A method as defined in claim 18 , wherein committing the call type further comprises retrieving new utterances from the speech data for generating a new call type.
Claim 30
Dependent Claim
45 words
A method as defined in claim 18 , wherein retrieving a plurality of utterances from speech data further comprising linking each of the utterances to an audio file, wherein the audio file associated with a particular utterance is played if the particular utterance is selected.
Claim 31
Dependent Claim
15 words
A computer program product having computer-executable instructions for implementing the method of claim 18 .
Claim 32
Independent Claim
175 words
A method for generating an annotation guide from speech data, the method comprising: analyzing speech data to present a user with a first group of utterances; analyzing the speech data to present a user with a second group of utterances that are related to the first group of utterances; and defining a first call type in the first group of utterances and defining a second call type in the second group of utterances by performing one or more of: removing one or more utterances from the first group of utterances or the second group of utterances; copying a particular utterance from the first group of utterances to the second group of utterances; moving a particular utterance from the first group of utterances to the second group of utterances; analyzing the speech data to find additional utterances for the first group of utterances using selected utterances from the first group of utterances; and analyzing the speech data to find additional utterances for the second group of utterances using selected utterances from the second group of utterances.
Claim 33
Independent Claim
175 words
A method for generating an annotation guide from speech data, the method comprising: analyzing speech data to present a user with a first group of utterances; analyzing the speech data to present a user with a second group of utterances that are related to the first group of utterances; and defining a first call type in the first group of utterances and defining a second call type in the second group of utterances by performing one or more of: removing one or more utterances from the first group of utterances or the second group of utterances; copying a particular utterance from the first group of utterances to the second group of utterances; moving a particular utterance from the first group of utterances to the second group of utterances; analyzing the speech data to find additional utterances for the first group of utterances using selected utterances from the first group of utterances; and analyzing the speech data to find additional utterances for the second group of utterances using selected utterances from the second group of utterances.
Claim 34
Dependent Claim
47 words
A method as defined in claim 29 , wherein analyzing the speech data to find additional utterances for the first group of utterances further comprises using the selected utterances as feedback to identify the additional utterances in the speech data that are similar to the selected utterances.
Claim 35
Dependent Claim
89 words
A method as defined in claim 29 , wherein analyzing the speech data to present a user with a first group of utterances further comprises one or more of: clustering the speech data into N clusters; searching the speech data using a string or a variant of the string; mining the speech data using speech parameters; using selected utterances as feedback to identify similar utterances in the speech data; and identifying marginal utterances of a particular call type using both positive and negative examples of the particular call type.
Claim 36
Dependent Claim
33 words
A method as defined in claim 29 , further comprising committing the first group of utterances to the first call type and committing the second group of utterances to the second call type.
Claim 37
Dependent Claim
14 words
A method as defined in claim 32 , further comprising generating an annotation guide.
References Cited (Backward Citations)
US6378132B1
Cited
US7149687B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2010-05-14 | 2010-11-13 | 2011-05-15 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2014-05-14 | 2014-11-13 | 2015-05-15 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2018-05-14 | 2018-11-13 | 2019-05-15 | $7700.00 | Window Closed / Lapsed |