US5943445A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Dynamic sprites for encoding video data
Assignee: Hewlett Packard Development Co LP
•
Filing: 1996-12-19
•
Granted: 1999-08-24
•
Expires: 2016-12-19
Abstract
Abstract Video data representing a scene are processed to improve encoding efficiencies. The video data are segmented into rigidly and non-rigidly moving video objects. The segmentation is preformed by estimating local motion vectors for the video data of a sequence of frames. The local motion vectors are clustered to determine dominant motions, and video data having motion vectors similar to the dominant motions are segmented out as rigidly moving video objects. For these objects, motion parameters are robustly estimated. Using the motion parameters, the rigid video objects of the frames are integrated in one or more corresponding sprites stored in a long-term memory. The sprites can be used in a two-way motion compensated prediction technique for encoding video data where blocks of video data are encoded either from sprites based on rigid motion parameters, or from a previous frame based on local motion parameters. In addition, the sprites can be used to construct a high resolution panoramic view of the background of a scene.
Inventors
- 👤 Frederic A. Dufaux
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 14:03
CPC Classifications
H; H04; H04N; H04N19/00; H04N19/50; H04N19/503; H04N19/51; H04N19/58; H04N19/20; H04N19/23; H04N19/537; H04N19/543
Claims Analysis
Total Claims: 28
Claim 1
Independent Claim
52 words
A computer implemented method for processing video data representing a scene, comprising the steps of: segmenting the video data into rigidly and non-rigidly moving video objects; estimating motion information for the rigidly moving objects; and integrating the rigidly moving objects, using the motion information, incrementally into corresponding dynamic sprites stored in memory.
Claim 2
Dependent Claim
52 words
A computer implemented method for processing video data representing a scene, comprising the steps of: segmenting the video data into rigidly and non-rigidly moving video objects; estimating motion information for the rigidly moving objects; and integrating the rigidly moving objects, using the motion information, incrementally into corresponding dynamic sprites stored in memory.
Claim 3
Dependent Claim
45 words
The method of claim 1 further comprising: estimating local motion vectors for the video data; clustering the local motion vectors to determine dominant motions; and segmenting the video data having local motion vectors substantially similar to the dominant motions as the rigidly moving video objects.
Claim 4
Dependent Claim
67 words
The method of claim 2 further comprising: determining first residual information present in a prediction error of the dominant motions; determining second residual information present in a prediction error of the local motion vectors; segmenting the video data as the rigidly moving video objects if the first residual information are substantially similar to the second residual information, otherwise, segmenting the video data as non-rigidly moving video objects.
Claim 5
Dependent Claim
16 words
The method of claim 1 further comprising: estimating the dominant motions using a parametric motion model.
Claim 6
Dependent Claim
15 words
The method of claim 4 wherein the parametric motion model is a perspective motion model.
Claim 7
Dependent Claim
19 words
The method of claim 2 wherein there is one video object and a corresponding sprite for each dominant motion.
Claim 8
Dependent Claim
29 words
The method of claim 1 wherein the video data are organized as a temporal sequence of frames, and further comprising: aligning the frames with respect to a coordinate system.
Claim 9
Dependent Claim
32 words
The method of claim 7 further comprising: aligning the frames with respect to a first frame of the temporal sequence of frames to establish a fixed reference coordinate system for the sprites.
Claim 10
Dependent Claim
57 words
The method of claim 8 wherein each frame and each sprite includes a plurality of pixels, each pixel representing a light intensity value of the scene, and further comprising: warping pixels of a current frame of the temporal sequence of frames to the fixed reference coordinate system of the sprites using a backward transform prior to integrating.
Claim 11
Dependent Claim
31 words
The method of claim 7 further comprising: aligning the sprites with respect to a current frame of the temporal sequence of frames to establish a time-varying coordinate system for the sprites.
Claim 12
Dependent Claim
51 words
The method of claim 10 wherein each frame and each sprite includes a plurality of pixels, each pixel representing a light intensity value of the scene, and further comprising: warping pixels of the, sprites to a current frame of the temporal sequence of frames using a forward transform prior to integrating.
Claim 13
Dependent Claim
88 words
The method of claim 1 further comprising: encoding the video data, and motion and shape information as encoded data using an encoder of a first processing system; decoding the encoded data using a first decoder to build a first sprite in a first memory of the first processing system; transmitting the encoded data to a second processing system; decoding the encoded data using a second decoder to build a second sprite in a second memory of the second processing system, the first and second sprites being substantially identical.
Claim 14
Dependent Claim
63 words
The method of claim 12 wherein the video data are organized as a temporal sequence of frames, and each frame includes a plurality of block of pixels, and further comprising: selecting a particular block of pixels of a current frame of the temporal sequence of frames for encoding from the first sprite or from a previous frame of the temporal sequence of frames.
Claim 15
Dependent Claim
49 words
The method of claim 13 further comprising: encoding the particular block using the motion information if the particular block is selected for encoding from the first sprite; and encoding the particular block using the local motion vectors if the particular block is selected for encoding from the previous frame.
Claim 16
Dependent Claim
79 words
The method of claim 13 further comprising: determining a first prediction error for the particular block based on encoding the particular block from the first sprite; determining a second prediction error for the particular block based on encoding the particular block from the previous frame; and selecting the particular block for encoding from the first sprite if the first prediction error is less than the second prediction error, otherwise selecting the particular block for encoding from the previous frame.
Claim 17
Dependent Claim
44 words
The method of claim 8 further comprising: selecting one of the sprites having the fixed reference coordinate system to construct a panoramic view of a background portion of the scene represented by the video data, the panoramic view only including rigidly moving video objects.
Claim 18
Independent Claim
60 words
An apparatus for processing video data representing a scene, comprising: a camera acquiring video data representing light intensity values of a scene; means for segmenting the video data into rigidly and non-rigidly moving video objects; means for estimating motion information for the rigidly moving objects; and a memory incrementally storing an integration of the rigidly moving objects as dynamic sprites.
Claim 19
Independent Claim
60 words
An apparatus for processing video data representing a scene, comprising: a camera acquiring video data representing light intensity values of a scene; means for segmenting the video data into rigidly and non-rigidly moving video objects; means for estimating motion information for the rigidly moving objects; and a memory incrementally storing an integration of the rigidly moving objects as dynamic sprites.
Claim 20
Dependent Claim
117 words
A computer-implemented method for processing video data representing a scene comprising the steps of: segmenting the video data into rigidly and non-rigidly moving video objects; estimating motion information for the rigidly moving objects; integrating the rigidly moving objects, using the motion information into at least one corresponding sprite in a memory; wherein the video data are organized as a temporal sequence of frames, each frame includes a plurality of blocks of pixels, and a particular block of pixels of a first of the temporal sequence of frames is selected for encoding from one of the sprite and a second of the temporal sequence of frames which occurs prior to the first of the temporal sequence of frames.
Claim 21
Dependent Claim
117 words
A computer-implemented method for processing video data representing a scene comprising the steps of: segmenting the video data into rigidly and non-rigidly moving video objects; estimating motion information for the rigidly moving objects; integrating the rigidly moving objects, using the motion information into at least one corresponding sprite in a memory; wherein the video data are organized as a temporal sequence of frames, each frame includes a plurality of blocks of pixels, and a particular block of pixels of a first of the temporal sequence of frames is selected for encoding from one of the sprite and a second of the temporal sequence of frames which occurs prior to the first of the temporal sequence of frames.
Claim 22
Dependent Claim
50 words
The method of claim 18, wherein, if the particular block of pixels is selected from the sprite, the encoding uses the motion information and, if the particular block is selected from the second of the temporal sequence of frames, the encoding uses other motion information different than the motion information.
Claim 23
Independent Claim
74 words
An apparatus for processing video data representing a scene comprising: a processor configured to receive the video data, to segment the video data into rigidly and non-rigidly moving objects, and to estimate motion information for the rigidly moving objects; a memory configured to incrementally store an integration of the rigidly moving objects as at least one dynamic sprite; and an encoder configured to encode the video data using the at least one dynamic sprite.
Claim 24
Independent Claim
74 words
An apparatus for processing video data representing a scene comprising: a processor configured to receive the video data, to segment the video data into rigidly and non-rigidly moving objects, and to estimate motion information for the rigidly moving objects; a memory configured to incrementally store an integration of the rigidly moving objects as at least one dynamic sprite; and an encoder configured to encode the video data using the at least one dynamic sprite.
Claim 25
Dependent Claim
77 words
The apparatus of claim 20, wherein the processor is further configured to organize the video data as a temporal sequence of frames, each frame including a plurality of blocks of pixels, and to select a particular block of pixels of a first of the temporal sequence of frames for the encoder to encode from one of the at least one sprite and a second of the temporal sequence of frames which occurs prior to the first frame.
Claim 26
Dependent Claim
43 words
A computer-implemented method for processing video data representing a scene comprising the steps of: estimating motion information for a rigidly moving object within the scene; and building at least one dynamic sprite by incrementally integrating the rigidly moving object using the motion information.
Claim 27
Dependent Claim
43 words
A computer-implemented method for processing video data representing a scene comprising the steps of: estimating motion information for a rigidly moving object within the scene; and building at least one dynamic sprite by incrementally integrating the rigidly moving object using the motion information.
Claim 28
Dependent Claim
20 words
The method of claim 22, further comprising the step of: encoding the video data using the at least one sprite.
References Cited (Backward Citations)
US4689022A
Cited
US4639878A
Cited
US4906940A
Cited
US4969036A
Cited
US5734441A
Cited
US5144149A
Cited
US5448053A
Cited
US5428387A
Cited
US5585862A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2002-08-23 | 2003-02-22 | 2003-08-24 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2006-08-23 | 2007-02-22 | 2007-08-24 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2010-08-23 | 2011-02-22 | 2011-08-24 | $7700.00 | Window Closed / Lapsed |