US6892351B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Creating a multimedia presentation from full motion video using significance measures
Assignee: Wirefree Services Belgium SA
•
Filing: 2002-01-17
•
Granted: 2005-05-10
•
Expires: 2022-01-17
Abstract
Abstract Methods apparatus and computer programs products for creating a multimedia presentation having video frames and video segments with synchronized audio are disclosed. Audio-video data is separated into an audio stream and the video sequence. The video sequence is divided into video segments, with each video segment comprising a group of frames. For each video segment an audio significance measure is calculated using the audio stream. Using the audio significance measure, the video segment is selected in its entirety or, alternatively, one or more slide frames are extracted from the corresponding group of frames. The selection of video segments or slide frames can also be performed on the basis of an additional video significance measure. The audio stream and selected video segment and slide frames are synchronized, then synchronously reproduced as the multimedia presentation.
Inventors
- 👤 Vinod V. Vasudevan
- 👤 Gurminder Singh
- 👤 Manoranjan D. Jesudoss
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-24 21:20
CPC Classifications
H; H04; H04N; H04N19/00; H04N19/50; H04N19/587
Claims Analysis
Total Claims: 29
Claim 1
Independent Claim
151 words
A method for creating a multimedia presentation having video frames and video segments with synchronized audio, said method comprising the steps of: (g) receiving audio-video data; (h) separating said audio-video data into an audio stream and a video sequence; (i) dividing said video sequence into video segments, each of said video segments comprising a group of slide frames; (j) for each of said video segments; (d1) calculating an audio significance measure using said audio stream related to each of said video segments; (d2) using at least said audio significance measure, selecting either of each said video segments in its entirety or extracting at least one slide frame from said corresponding group of frames; (k) synchronizing said audio stream and selected video segments and said at least one slide frame; and (l) synchronously reproducing selected video segments and said at least one slide frame and said audio stream as said multimedia presentation.
Claim 2
Independent Claim
151 words
A method for creating a multimedia presentation having video frames and video segments with synchronized audio, said method comprising the steps of: (g) receiving audio-video data; (h) separating said audio-video data into an audio stream and a video sequence; (i) dividing said video sequence into video segments, each of said video segments comprising a group of slide frames; (j) for each of said video segments; (d1) calculating an audio significance measure using said audio stream related to each of said video segments; (d2) using at least said audio significance measure, selecting either of each said video segments in its entirety or extracting at least one slide frame from said corresponding group of frames; (k) synchronizing said audio stream and selected video segments and said at least one slide frame; and (l) synchronously reproducing selected video segments and said at least one slide frame and said audio stream as said multimedia presentation.
Claim 3
Dependent Claim
26 words
The method according to claim 1 , wherein, in step (d1), said audio significance measure is calculated from a measure of activity within said audio stream.
Claim 4
Dependent Claim
23 words
The method according to claim 2 , wherein said measure of activity is determined from a frequency domain representation of said audio stream.
Claim 5
Dependent Claim
24 words
The method according to claim 2 , wherein said measure of activity is calculated from one or more of power, peak-frequency and frequency spread.
Claim 6
Dependent Claim
17 words
The method according to claim 4 , wherein said measure of activity is calculated as power+peak frequency+spread.
Claim 7
Dependent Claim
17 words
The method according to claim 4 , wherein said measure of activity is calculated as 2×power+peak frequency+0.5×spread.
Claim 8
Dependent Claim
51 words
The method of claim 1 , wherein, in step (d2), said selection is based upon a comparison of said audio significance measure with an activity threshold, such that if said threshold is exceeded, then said video segment is selected, else if not exceeded, said at least one slide frame is selected.
Claim 9
Independent Claim
176 words
A method for creating a multimedia presentation having video frames and video segments with synchronized audio, said method comprising the steps of: (g) receiving audio-video data; (h) separating said audio-video data into an audio stream and a video sequence; (i) dividing said video sequence into video segments, each of said video segments comprising a group of slide frames; (j) for each of said video segments; (d1) extracting at least one representative frame from the corresponding said group of frames; (d2) calculating a video significance measure using the extracted at least one frame; (d3) calculating an audio significance measure using the audio stream related to said each of said video segment; (d4) using said video and audio significance measures, selecting either each of the video segments in its entirely or extracting at least one slide frame from said group of frames; (k) synchronizing said audio stream and the selected video segments and the extracted said at least one slide frame; and (l) synchronously reproducing the selected video segment and the extracted slide frame and said audio stream.
Claim 10
Independent Claim
176 words
A method for creating a multimedia presentation having video frames and video segments with synchronized audio, said method comprising the steps of: (g) receiving audio-video data; (h) separating said audio-video data into an audio stream and a video sequence; (i) dividing said video sequence into video segments, each of said video segments comprising a group of slide frames; (j) for each of said video segments; (d1) extracting at least one representative frame from the corresponding said group of frames; (d2) calculating a video significance measure using the extracted at least one frame; (d3) calculating an audio significance measure using the audio stream related to said each of said video segment; (d4) using said video and audio significance measures, selecting either each of the video segments in its entirely or extracting at least one slide frame from said group of frames; (k) synchronizing said audio stream and the selected video segments and the extracted said at least one slide frame; and (l) synchronously reproducing the selected video segment and the extracted slide frame and said audio stream.
Claim 11
Dependent Claim
26 words
The method according to claim 8 , wherein, in step (d1), said audio significance measure is calculated from a measure of activity within said audio stream.
Claim 12
Dependent Claim
23 words
The method according to claim 9 , wherein said measure of activity is determined from a frequency domain representation of said audio stream.
Claim 13
Dependent Claim
24 words
The method according to claim 8 , wherein said measure of activity is calculated from one or more of power, peak-frequency and frequency spread.
Claim 14
Dependent Claim
17 words
The method according to claim 11 , wherein said measure of activity is calculated as power+peak frequency+spread.
Claim 15
Dependent Claim
17 words
The method according to claim 11 , wherein said measure of activity is calculated as 2×power+peak frequency+0.5×spread.
Claim 16
Dependent Claim
23 words
The method according to claim 8 , wherein said video significance measure is determined from a level of relative movement between said frames.
Claim 17
Dependent Claim
47 words
The method according to claim 14 , wherein said frames comprise objects and said level of relative movement is determined from a direction and magnitude of motion (u i , v i ) of each object in said frames to derive an activity value of the frame.
Claim 18
Dependent Claim
31 words
The method according to claim 15 , wherein said activity value is determined from the standard deviation of the direction and magnitude (u i , v i ) of each object.
Claim 19
Dependent Claim
60 words
The method of claim 16 , wherein said activity value is determined according to the expression: Activity value == ∑ i = 1 i = k [ u i - u ] 2 + [ v i - v ] 2 wherein (u,v) is the average direction and magnitude of motion of all objects in said frame.
Claim 20
Dependent Claim
51 words
The method of claim 8 , wherein, in step (d4), said selection is based upon a comparison of said audio significance measure with an activity threshold, such that if said threshold is exceeded, then said video segment is selected, else if not exceeded, said at least one slide frame is selected.
Claim 21
Dependent Claim
43 words
The method of claim 18 , wherein said combined significance measure is equal to one of the audio significance measure, the video significance measure, or an average of said audio and video significance measures depending upon the level of audio and video activity.
Claim 22
Dependent Claim
147 words
Apparatus for creating a multimedia presentation having video frames and video segments with synchronized audio, said apparatus comprising: input means for receiving audio-video data; means for separating said audio-video data into an audio stream and a video sequence; means for dividing said video sequence into video segments, each of said video segments comprising a group of slide frames; processor means which, for each said video segments, calculates an audio significance measure using the audio stream related to each of the video segments, and, using at least said audio significance measure, selects either each of the video segments in its entity or extracts at least one slide frame from said corresponding group of frames; means for synchronizing said audio stream and selected video segments and extracted slide frames; and means for synchronously reproducing selected video segments and extracted slide frames and said audio stream as said multimedia presentation.
Claim 23
Dependent Claim
147 words
Apparatus for creating a multimedia presentation having video frames and video segments with synchronized audio, said apparatus comprising: input means for receiving audio-video data; means for separating said audio-video data into an audio stream and a video sequence; means for dividing said video sequence into video segments, each of said video segments comprising a group of slide frames; processor means which, for each said video segments, calculates an audio significance measure using the audio stream related to each of the video segments, and, using at least said audio significance measure, selects either each of the video segments in its entity or extracts at least one slide frame from said corresponding group of frames; means for synchronizing said audio stream and selected video segments and extracted slide frames; and means for synchronously reproducing selected video segments and extracted slide frames and said audio stream as said multimedia presentation.
Claim 24
Dependent Claim
168 words
Apparatus for creating a multimedia presentation having video frames and video segments with synchronized audio, comprising: means for receiving audio-video data; means for separating said audio-video data into an audio stream and a video sequence; means for dividing said video sequence into video segments, each of said video segments comprising a group of slide frames; processor means which, for each of said video segments, extracts at least one representative frame from the corresponding group of frames, calculates a video significance measure using the extracted at least one frame, calculates an audio significance measure using the audio stream related to said each of said video segments, and, using said video and audio significance measures, selects either said each of said video segments in its entirely or extracts at least one slide frame from the group of frames; means for synchronizing said audio stream and selected video segments and extracted slide frames; and means for synchronously reproducing the selected video segments and the extracted slide frames and said audio stream.
Claim 25
Dependent Claim
168 words
Apparatus for creating a multimedia presentation having video frames and video segments with synchronized audio, comprising: means for receiving audio-video data; means for separating said audio-video data into an audio stream and a video sequence; means for dividing said video sequence into video segments, each of said video segments comprising a group of slide frames; processor means which, for each of said video segments, extracts at least one representative frame from the corresponding group of frames, calculates a video significance measure using the extracted at least one frame, calculates an audio significance measure using the audio stream related to said each of said video segments, and, using said video and audio significance measures, selects either said each of said video segments in its entirely or extracts at least one slide frame from the group of frames; means for synchronizing said audio stream and selected video segments and extracted slide frames; and means for synchronously reproducing the selected video segments and the extracted slide frames and said audio stream.
Claim 26
Dependent Claim
169 words
A computer program product including a computer readable medium incorporating a computer program for creating a multimedia presentation having video frames and video segments with synchronized audio, said computer program having: input code means for receiving audio-video data; code means for separating said audio-video data into an audio stream and a video sequence; code means for dividing said video sequence into video segments, each of said video segments comprising a group of slide frames; processing code means which, for each of said video segments, calculates an audio significance measure using the audio stream related to said each of said video segments, and, using at least said audio significance measure, selects either said each of said video segments in its entity or extracts at least one slide frame from said corresponding group of frames; code means for synchronizing said audio stream and selected video segments and extracted slide frames; and code means for synchronously reproducing selected video segments and extracted slide frames and said audio stream as said multimedia presentation.
Claim 27
Dependent Claim
169 words
A computer program product including a computer readable medium incorporating a computer program for creating a multimedia presentation having video frames and video segments with synchronized audio, said computer program having: input code means for receiving audio-video data; code means for separating said audio-video data into an audio stream and a video sequence; code means for dividing said video sequence into video segments, each of said video segments comprising a group of slide frames; processing code means which, for each of said video segments, calculates an audio significance measure using the audio stream related to said each of said video segments, and, using at least said audio significance measure, selects either said each of said video segments in its entity or extracts at least one slide frame from said corresponding group of frames; code means for synchronizing said audio stream and selected video segments and extracted slide frames; and code means for synchronously reproducing selected video segments and extracted slide frames and said audio stream as said multimedia presentation.
Claim 28
Dependent Claim
190 words
A computer program product including a computer readable medium incorporating a computer program for creating a multimedia presentation having video frames and video segments with synchronized audio, said computer program having: code means for receiving audio-video data; code means for separating said audio-video data into an audio stream and a video sequence; code means for dividing said video sequence into video segments, each of said video segments comprising a group of slide frames; processing code means which, for said each of said video segments, extracts at least one representative frame from the corresponding said group of frames, calculates a video significance measure using the at least one extracted frame, calculates an audio significance measure using the audio stream related to said each of said video segments, and, using said video and audio significance measures, selects either said each of said video segments in its entirety or extracts at least one slide frame from said group of frames; code means for synchronizing said audio stream and selected video segments and extracted slide frames; and code means for synchronously reproducing the selected video segments and extracted slide frames and said audio stream.
Claim 29
Dependent Claim
190 words
A computer program product including a computer readable medium incorporating a computer program for creating a multimedia presentation having video frames and video segments with synchronized audio, said computer program having: code means for receiving audio-video data; code means for separating said audio-video data into an audio stream and a video sequence; code means for dividing said video sequence into video segments, each of said video segments comprising a group of slide frames; processing code means which, for said each of said video segments, extracts at least one representative frame from the corresponding said group of frames, calculates a video significance measure using the at least one extracted frame, calculates an audio significance measure using the audio stream related to said each of said video segments, and, using said video and audio significance measures, selects either said each of said video segments in its entirety or extracts at least one slide frame from said group of frames; code means for synchronizing said audio stream and selected video segments and extracted slide frames; and code means for synchronously reproducing the selected video segments and extracted slide frames and said audio stream.
References Cited (Backward Citations)
US5179449A
Cited
US5253069A
Cited
US5686961A
Cited
US5489949A
Cited
US5999173A
Cited
US5537528A
Cited
US5459517A
Cited
US5550965A
Cited
US5521841A
Cited
US5606655A
Cited
US5642171A
Cited
US5635982A
Cited
US5664227A
Cited
US5708767A
Cited
US5793429A
Cited
US5640560A
Cited
US5760767A
Cited
US6122668A
Cited
US6141001A
Cited
US5949410A
Cited
US5893062A
Cited
US6172675B1
Cited
US6404978B1
Cited
US6154771A
Cited
US6504990B1
Cited
US6738075B1
Cited
US6616700B1
Cited
US6636238B1
Cited
US6369835B1
Cited
US6636237B1
Cited
US6621502B1
Cited
US20040114904A1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2008-05-09 | 2008-11-08 | 2009-05-10 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2012-05-09 | 2012-11-08 | 2013-05-10 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2016-05-09 | 2016-11-08 | 2017-05-10 | $7700.00 | Window Closed / Lapsed |