Patent US6377995B2 successfully refreshed from Google Patents.
US6377995B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Indexing multimedia communications
Assignee: AT&T Inc
•
Filing: 1998-02-19
•
Granted: 2002-04-23
•
Expires: 2018-02-19
Abstract
Abstract A network based platform uses face recognition, speech recognition, background change detection and key scene events to index multimedia communications. Before the multimedia communication begins, active participants register their speech and face models with a server. The process consists of creating a speech sample, capturing a sample image of the participant and storing the data in a database. The server provides an indexing function for the multimedia communication. During the multimedia communication, metadata including time stamping is retained along with the multimedia content. The time stamping information is used for synchronizing the multimedia elements. The multimedia communication is then processed through the server to identify the multimedia communication participants based on speaker and face recognition models. This allows the server to create an index table that becomes an index of the multimedia communication. In addition, through scene change detection and background recognition, certain backgrounds and key scene information can be used for indexing. Therefore, through this indexing apparatus and method, a specific participant can be recognized as speaking and the content that the participant discussed can also be used for indexing.
Inventors
- 👤 Sanjay Agraharam
- 👤 Robert Edward Markowitz
- 👤 Kenneth H. Rosen
- 👤 David Hilton Shur
- 👤 Joel A. Winthrop
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-29 11:22
CPC Classifications
H; H04; H04N; H04N7/00; H04N7/14; H04N7/141; H04N7/142; G; G06; G06F; G06F16/00; G06F16/50; G06F16/58; G06F16/583; G06F16/5838; G10; G10L; G10L17/00; H04L; H04L65/00; H04L65/1066; H04L65/1101; H04L65/40; H04L65/403; H04L65/60; H04L65/61; H04L65/611; H04N7/147; H04N7/15; H04N7/152
Claims Analysis
Total Claims: 21
Claim 1
Independent Claim
152 words
A method for indexing a multimedia communication, comprising: receiving the multimedia communication, the multimedia communication including a plurality of multimedia data packets; processing the plurality of multimedia data packets to identify distinguishing features and associating each of the plurality of multimedia data packets with one of a plurality of objects within the multimedia communication; comparing the distinguishing features with representations of the plurality of objects stored within a database to create verified distinguishing features; associating the verified distinguishing features with each one of a plurality of objects; indexing the plurality of multimedia data packets based on the verified distinguishing features; and rebroadcasting in near real-time, during the multimedia communication, the processed plurality of multimedia data packets, wherein when the object is a speaker participating in the multimedia communication, combination of the speaker's audio speech patterns and video statistical sampling of a face of the speaker is a portion of the distinguishing features.
Claim 2
Independent Claim
152 words
A method for indexing a multimedia communication, comprising: receiving the multimedia communication, the multimedia communication including a plurality of multimedia data packets; processing the plurality of multimedia data packets to identify distinguishing features and associating each of the plurality of multimedia data packets with one of a plurality of objects within the multimedia communication; comparing the distinguishing features with representations of the plurality of objects stored within a database to create verified distinguishing features; associating the verified distinguishing features with each one of a plurality of objects; indexing the plurality of multimedia data packets based on the verified distinguishing features; and rebroadcasting in near real-time, during the multimedia communication, the processed plurality of multimedia data packets, wherein when the object is a speaker participating in the multimedia communication, combination of the speaker's audio speech patterns and video statistical sampling of a face of the speaker is a portion of the distinguishing features.
Claim 3
Dependent Claim
26 words
The method of claim 1 , wherein the distinguishing features are based on at least one of audio data and video data of the multimedia communication.
Claim 4
Dependent Claim
24 words
The method of claim 1 , wherein the objects include at least one of a person, an animal, a plant, and an inanimate object.
Claim 5
Dependent Claim
46 words
The method of claim 3 , wherein a multimedia data packet of the plurality of multimedia data packets includes a payload having one of the audio and the video data that corresponds to an object, the associating step attaching a header identifier that identifies the object.
Claim 6
Dependent Claim
33 words
The method of claim 4 , wherein the speaker is associated with the multimedia data packet if the portion of the distinguishing features is included in the payload of the multimedia data packet.
Claim 7
Dependent Claim
63 words
The method of claim 4 , further comprising: identifying background changes in the video data; identifying key scene events based on the video data; and attaching a second header identifier to each multimedia data packet containing the background change and the key scene event, the second header identifier identifying the multimedia data packet as containing a background change and a key scene event.
Claim 8
Dependent Claim
27 words
The method of claim 6 , wherein the multimedia communication is a multicast multimedia communication, the rebroadcasting step including multicasting the processed plurality of multimedia data packets.
Claim 9
Dependent Claim
20 words
The method of claim 7 , wherein a time stamp is provided to synchronize the audio and the video data.
Claim 10
Dependent Claim
40 words
The method of claim 8 , further comprising storing the indexed plurality of multimedia data packets, wherein the indexed plurality of multimedia data packets can be searched to retrieve audio and video multimedia data packets corresponding to selected distinguishing features.
Claim 11
Dependent Claim
20 words
The method of claim 9 , wherein the indexed plurality of multimedia data packets can be searched using key words.
Claim 12
Dependent Claim
17 words
The method of claim 10 , wherein the multimedia communication is conducted using a local area network.
Claim 13
Dependent Claim
19 words
The method of claim 1 , wherein the indexing and the processing steps are performed at a multicast network.
Claim 14
Independent Claim
122 words
An apparatus for indexing a multimedia communication, comprising: a server that receives multimedia communication in multimedia data packets including audio, visual and data communications and identifies distinguishing features in the multimedia communication based on at least one of audio and video recognition and a source of the multimedia communications; a header function module connected to the server, the header function module entering metadata in a header segment corresponding to the multimedia data packets received by the server, the metadata being related to the distinguishing features; an index server that rebroadcasts in near real-time the multimedia communication; a storage device that stores the multimedia data packets, wherein the distinguishing features include audio, voice and video face patterns of participants in the multimedia communication.
Claim 15
Independent Claim
122 words
An apparatus for indexing a multimedia communication, comprising: a server that receives multimedia communication in multimedia data packets including audio, visual and data communications and identifies distinguishing features in the multimedia communication based on at least one of audio and video recognition and a source of the multimedia communications; a header function module connected to the server, the header function module entering metadata in a header segment corresponding to the multimedia data packets received by the server, the metadata being related to the distinguishing features; an index server that rebroadcasts in near real-time the multimedia communication; a storage device that stores the multimedia data packets, wherein the distinguishing features include audio, voice and video face patterns of participants in the multimedia communication.
Claim 16
Dependent Claim
17 words
The apparatus of claim 13 , wherein the metadata includes voice and face identifiers of the participants.
Claim 17
Dependent Claim
46 words
The apparatus of claim 14 , wherein the server identifies background changes in the video multimedia data packets and wherein the header function module enters second metadata in the header segment corresponding to the multimedia data packets having background changes, the second metadata including scene identifiers.
Claim 18
Dependent Claim
13 words
The apparatus of claim 15 , wherein the server is an multicast server.
Claim 19
Dependent Claim
15 words
The apparatus of claim 16 , wherein the server comprises audio, video and data bridges.
Claim 20
Independent Claim
92 words
A method of identifying participants to a multimedia communication that is rebroadcast in near real-time, comprising: comparing audio speech patterns for each participant to speech models; comparing video face patterns for each participant to face models; determining an identity of a particular participant when both the audio speech patterns and the video face patterns match the speech and the face models for the particular participant; and creating an index of the participants based on identification of speech and face patterns of the participants, the index being used to segment the multimedia communication.
Claim 21
Independent Claim
92 words
A method of identifying participants to a multimedia communication that is rebroadcast in near real-time, comprising: comparing audio speech patterns for each participant to speech models; comparing video face patterns for each participant to face models; determining an identity of a particular participant when both the audio speech patterns and the video face patterns match the speech and the face models for the particular participant; and creating an index of the participants based on identification of speech and face patterns of the participants, the index being used to segment the multimedia communication.
References Cited (Backward Citations)
US5835667A
Cited
US5666466A
Cited
US5710591A
Cited
US5835153A
Cited
US6038368A
Cited
US5894480A
Cited
US6035304A
Cited
US5928330A
Cited
US5926624A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2005-04-22 | 2005-10-22 | 2006-04-23 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2009-04-22 | 2009-10-22 | 2010-04-23 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2013-04-22 | 2013-10-22 | 2014-04-23 | $7700.00 | Window Closed / Lapsed |