US5630037A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method and apparatus for extracting and treating digital images for seamless compositing
Assignee: Schindler Imaging Inc
•
Filing: 1994-05-18
•
Granted: 1997-05-13
•
Expires: 2014-05-18
Abstract
Abstract An image processing system for extracting and treating a digitized color subject image for seamless compositing against an arbitrary background includes the generation of a final control image from the image of a subject in front of a screen consisting of a range of colors. A sample of the background screen is captured and analyzed using filter kernels to determine a range of red, green and blue values which are stored in separate histograms. A first control image is generated by comparing mean histogram data with the subject image. A final control image is generated by dividing the first control image into separate background, foreground and fringe regions. A treated subject image is generated by leaking color from the local background region into the fringe regions of the subject image using filter kernels. The treated image is suitable for compositing against an arbitrary background image. The composite image may be touched up by using a software brush which on each pass of the brush expands the background region by eliminating immediately adjacent fringe pixels, causes other nearby fringe pixels to increase in translucence, and allows the boundary of the fringe region to encroach on the foreground region. Simultaneously, the brush may also cause fringe pixels to absorb some of the color of neighboring foreground pixels.
Inventors
- 👤 Keith W. Schindler
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-01 23:57
CPC Classifications
G; G06; G06T; G06T5/00; G06T5/40; G06T5/90; G06T2207/00; G06T2207/10; G06T2207/10024
Claims Analysis
Total Claims: 27
Claim 1
Independent Claim
328 words
A method for extracting a digitized foreground image for compositing with an arbitrary background image, said method comprising the steps of: acquiring an image comprising a foreground subject and a background, obtaining data representing coloration of said background, said data comprising a plurality of values corresponding to color intensities, generating from said data a statistical range of color intensities for each of a selected group of colors, deriving an indication of average background color from said statistical range of color intensities, differentiating said foreground subject from said background by comparing color intensity values of said foreground subject with said indication of average background color, generating a control image comprised of values representing a distance in color of each color intensity value in said foreground subject from said average background color, and generating a second control image by dividing said first control image into a background region, a fringe region of variable width, and a foreground region, and mixing color into foreground pixels at locations corresponding to said fringe region of said second control image from nearby background pixels at locations corresponding to said background region of said second control image, wherein said second control image comprises a plurality of values corresponding to pixels at specific locations, and wherein said step of mixing color into foreground pixels at locations corresponding to said fringe region of said second control image from nearby background pixels at locations corresponding to said background region of said second control image comprises the steps of (a) identifying a pixel in said fringe region bordered by pixels from said background region, (b) changing a color of the corresponding pixel in said foreground image to reflect an average color of a plurality of background pixels in locations corresponding to said background region near said pixel, (c) associating said pixel no longer with said fringe region but instead with said background region, and (d) repeating steps (a) through (c) for a plurality of pixels in said fringe region.
Claim 2
Independent Claim
328 words
A method for extracting a digitized foreground image for compositing with an arbitrary background image, said method comprising the steps of: acquiring an image comprising a foreground subject and a background, obtaining data representing coloration of said background, said data comprising a plurality of values corresponding to color intensities, generating from said data a statistical range of color intensities for each of a selected group of colors, deriving an indication of average background color from said statistical range of color intensities, differentiating said foreground subject from said background by comparing color intensity values of said foreground subject with said indication of average background color, generating a control image comprised of values representing a distance in color of each color intensity value in said foreground subject from said average background color, and generating a second control image by dividing said first control image into a background region, a fringe region of variable width, and a foreground region, and mixing color into foreground pixels at locations corresponding to said fringe region of said second control image from nearby background pixels at locations corresponding to said background region of said second control image, wherein said second control image comprises a plurality of values corresponding to pixels at specific locations, and wherein said step of mixing color into foreground pixels at locations corresponding to said fringe region of said second control image from nearby background pixels at locations corresponding to said background region of said second control image comprises the steps of (a) identifying a pixel in said fringe region bordered by pixels from said background region, (b) changing a color of the corresponding pixel in said foreground image to reflect an average color of a plurality of background pixels in locations corresponding to said background region near said pixel, (c) associating said pixel no longer with said fringe region but instead with said background region, and (d) repeating steps (a) through (c) for a plurality of pixels in said fringe region.
Claim 3
Dependent Claim
22 words
The method of claim 1 wherein said plurality of pixels in step (d) comprises all of the pixels in the fringe region.
Claim 4
Dependent Claim
39 words
The method of claim 1 wherein said step of changing a color of the corresponding pixel in said foreground image comprises the step of calculating an average color of background pixels in a predefined square region around said pixel.
Claim 5
Dependent Claim
56 words
The method of claim 1 further comprising the step of generating a treated image by removing color pixels whose values correspond to the background coloration from said foreground subject, determining neighboring color pixel values for selected remaining portions of said foreground subject, and mixing local background coloration into said selected remaining portions on a pixel-by-pixel basis.
Claim 6
Dependent Claim
100 words
The method of claim 4 wherein said step of mixing color into foreground pixels at locations corresponding to said fringe region of said second control image from nearby background pixels at locations corresponding to said background region of said second control image comprises the step of generating an intermediate image by mixing color with selected fringe pixels from neighboring pixel locations, and wherein said step of removing color pixels whose values correspond to the background coloration from said foreground subject comprises the step of calculating a difference between fringe pixels in said intermediate image and fringe pixels in said image.
Claim 7
Dependent Claim
24 words
The method of claim 4 further comprising the step of generating a composite image by combining said treated image with a selected background image.
Claim 8
Dependent Claim
51 words
The method of claim 6 wherein said second control image comprises an alpha channel, and wherein said step of combining said treated image with a selected background image comprises the step of mixing portions of said treated image and said selected background image in a ratio determined by said alpha channel.
Claim 9
Dependent Claim
89 words
The method of claim 6 wherein said step of combining said treated image with a selected background image results in said composite image being the same as said selected background image in regions where said second control image has a minimum value, the same as said treated image in regions where said second control image has a maximum value, and a mixture of said selected background image and said treated image in regions where said second control image has a value between said minimum value and said maximum value.
Claim 10
Independent Claim
126 words
A method for touching up a digitally composited image comprising the steps of: selecting a region of a composited image, said composited image comprising a foreground image and a background image bordered by a fringe region of variable width, wherein said step of selecting a region comprises the step of interactively placing a brush on a screen displaying said composited image, in a portion of said selected region corresponding to said fringe region and immediately bordered at least in part by said foreground region, replacing some of the color pixel values in said portion with color pixel values that are closer in proximity, within a color space, to said foreground region, and mixing a remaining color in said portion with a color of said foreground region.
Claim 11
Independent Claim
126 words
A method for touching up a digitally composited image comprising the steps of: selecting a region of a composited image, said composited image comprising a foreground image and a background image bordered by a fringe region of variable width, wherein said step of selecting a region comprises the step of interactively placing a brush on a screen displaying said composited image, in a portion of said selected region corresponding to said fringe region and immediately bordered at least in part by said foreground region, replacing some of the color pixel values in said portion with color pixel values that are closer in proximity, within a color space, to said foreground region, and mixing a remaining color in said portion with a color of said foreground region.
Claim 12
Dependent Claim
21 words
The method of claim 9 wherein said step of placing a brush is carried out with a stylus, pen or mouse.
Claim 13
Independent Claim
215 words
A method for processing a digitized subject image, said method comprising the steps of: acquiring an image of a subject in front of a background, said image comprising a plurality of color values at each of a plurality of pixel locations, measuring the color of said background by obtaining a plurality of color values at each of a plurality of pixel locations in said background, generating a plurality of histograms, one for each color of a select group of colors, from said color values of said background, calculating an average background color from said plurality of histograms, generating a first control image by comparing said average background color and said color values of said image, dividing said first control image into a plurality of regions so as to generate a second control image, said second control image defining a background region, a fringe region, and a foreground region, and modifying color values in a portion of said image corresponding to said fringe region, wherein each of said histograms comprises a plurality of average color values, said step of generating a plurality of histograms comprising the step of deriving each of said average color values by averaging the color values at a particular pixel location with the color values of a predefined group of surrounding pixels.
Claim 14
Independent Claim
215 words
A method for processing a digitized subject image, said method comprising the steps of: acquiring an image of a subject in front of a background, said image comprising a plurality of color values at each of a plurality of pixel locations, measuring the color of said background by obtaining a plurality of color values at each of a plurality of pixel locations in said background, generating a plurality of histograms, one for each color of a select group of colors, from said color values of said background, calculating an average background color from said plurality of histograms, generating a first control image by comparing said average background color and said color values of said image, dividing said first control image into a plurality of regions so as to generate a second control image, said second control image defining a background region, a fringe region, and a foreground region, and modifying color values in a portion of said image corresponding to said fringe region, wherein each of said histograms comprises a plurality of average color values, said step of generating a plurality of histograms comprising the step of deriving each of said average color values by averaging the color values at a particular pixel location with the color values of a predefined group of surrounding pixels.
Claim 15
Independent Claim
193 words
A method for processing a digitized subject image, said method comprising the steps of: acquiring an image of a subject in front of a background, said image comprising a plurality of color values at each of a plurality of pixel locations, measuring the color of said background by obtaining a plurality of color values at each of a plurality of pixel locations in said background, generating a plurality of histograms, one for each color of a select group of colors, from said color values of said background, calculating an average background color from said plurality of histograms, generating a first control image by comparing said average background color and said color values of said image, dividing said first control image into a plurality of regions so as to generate a second control image, said second control image defining a background region, a fringe region, and a foreground region, modifying color values in a portion of said image corresponding to said fringe region, and adjusting a location of a first frontier between said background region and said fringe region, and a location of a second frontier between said fringe region and said foreground region.
Claim 16
Independent Claim
193 words
A method for processing a digitized subject image, said method comprising the steps of: acquiring an image of a subject in front of a background, said image comprising a plurality of color values at each of a plurality of pixel locations, measuring the color of said background by obtaining a plurality of color values at each of a plurality of pixel locations in said background, generating a plurality of histograms, one for each color of a select group of colors, from said color values of said background, calculating an average background color from said plurality of histograms, generating a first control image by comparing said average background color and said color values of said image, dividing said first control image into a plurality of regions so as to generate a second control image, said second control image defining a background region, a fringe region, and a foreground region, modifying color values in a portion of said image corresponding to said fringe region, and adjusting a location of a first frontier between said background region and said fringe region, and a location of a second frontier between said fringe region and said foreground region.
Claim 17
Independent Claim
276 words
A method for processing a digitized subject image, said method comprising the steps of: acquiring an image of a subject in front of a background, said image comprising a plurality of color values at each of a plurality of pixel locations, measuring the color of said background by obtaining a plurality of color values at each of a plurality of pixel locations in said background, generating a plurality of histograms, one for each color of a select group of colors, from said color values of said background, calculating an average background color from said plurality of histograms, generating a first control image by comparing said average background color and said color values of said image, dividing said first control image into a plurality of regions so as to generate a second control image, said second control image defining a background region, a fringe region, and a foreground region, and modifying color values in a portion of said image corresponding to said fringe region, said step of modifying a portion of said image comprising the step of generating an intermediate image by adding color from said background region to said fringe region, wherein said step of generating said intermediate image comprises the steps of (a) identifying a pixel in said fringe region bordered by at least one pixel in said background region, (b) changing a color of said fringe pixel to reflect an average color of a plurality of pixels in said background region, (c) associated said fringe pixel no longer with said fringe region but instead with said background region, and (d) repeating steps (a) through (c) for a plurality of pixels in said fringe region.
Claim 18
Independent Claim
276 words
A method for processing a digitized subject image, said method comprising the steps of: acquiring an image of a subject in front of a background, said image comprising a plurality of color values at each of a plurality of pixel locations, measuring the color of said background by obtaining a plurality of color values at each of a plurality of pixel locations in said background, generating a plurality of histograms, one for each color of a select group of colors, from said color values of said background, calculating an average background color from said plurality of histograms, generating a first control image by comparing said average background color and said color values of said image, dividing said first control image into a plurality of regions so as to generate a second control image, said second control image defining a background region, a fringe region, and a foreground region, and modifying color values in a portion of said image corresponding to said fringe region, said step of modifying a portion of said image comprising the step of generating an intermediate image by adding color from said background region to said fringe region, wherein said step of generating said intermediate image comprises the steps of (a) identifying a pixel in said fringe region bordered by at least one pixel in said background region, (b) changing a color of said fringe pixel to reflect an average color of a plurality of pixels in said background region, (c) associated said fringe pixel no longer with said fringe region but instead with said background region, and (d) repeating steps (a) through (c) for a plurality of pixels in said fringe region.
Claim 19
Dependent Claim
22 words
The method of claim 13 wherein said plurality of pixels in step (d) comprises all of the pixels in said fringe region.
Claim 20
Dependent Claim
36 words
The method of claim 13 wherein the step of changing a color of said fringe pixel comprises the step of calculating an average color of background pixels in a predefined square region around said fringe pixel.
Claim 21
Independent Claim
239 words
A method for processing a digitized subject image, said method comprising the steps of: acquiring an image of a subject in front of a background, said image comprising a plurality of color values at each of a plurality of pixel locations, measuring the color of said background by obtaining a plurality of color values at each of a plurality of pixel locations in said background, generating a plurality of histograms, one for each color of a select group of colors, from said color values of said background, calculating an average background color from said plurality of histograms, generating a first control image by comparing said average background color and said color values of said image, dividing said first control image into a plurality of regions so as to generate a second control image, said second control image defining a background region, a fringe region, and a foreground region, modifying color values in a portion of said image corresponding to said fringe region, said step of modifying a portion of said image comprising the step of generating an intermediate image by adding color from said background region to said fringe region, and generating a treated image by removing at least a portion of said background color from said image, determining neighboring color pixel values in the remaining portions of said image, and leaking a local background color from adjacent background pixels into boundary pixels of each of said remaining portions.
Claim 22
Independent Claim
239 words
A method for processing a digitized subject image, said method comprising the steps of: acquiring an image of a subject in front of a background, said image comprising a plurality of color values at each of a plurality of pixel locations, measuring the color of said background by obtaining a plurality of color values at each of a plurality of pixel locations in said background, generating a plurality of histograms, one for each color of a select group of colors, from said color values of said background, calculating an average background color from said plurality of histograms, generating a first control image by comparing said average background color and said color values of said image, dividing said first control image into a plurality of regions so as to generate a second control image, said second control image defining a background region, a fringe region, and a foreground region, modifying color values in a portion of said image corresponding to said fringe region, said step of modifying a portion of said image comprising the step of generating an intermediate image by adding color from said background region to said fringe region, and generating a treated image by removing at least a portion of said background color from said image, determining neighboring color pixel values in the remaining portions of said image, and leaking a local background color from adjacent background pixels into boundary pixels of each of said remaining portions.
Claim 23
Dependent Claim
36 words
The method of claim 16 wherein said step of removing at least a portion of the background color comprises the step of calculating a difference between values of said intermediate image and values of said image.
Claim 24
Dependent Claim
24 words
The method of claim 16 further comprising the step of generating a composite image by combining said treated image with a selected background image.
Claim 25
Dependent Claim
51 words
The method of claim 18 wherein said second control image comprises an alpha channel, and wherein said step of combining said treated image with a selected background image comprises the step of mixing portions of said treated image and said selected background image in a ratio determined by said alpha channel.
Claim 26
Dependent Claim
88 words
The method of claim 18 wherein said step of combining said treated image with a selected background image results in said composite image being the same as said selected background image in regions where said second control image has a minimum value, the same as said treated image in regions where said second control image has a maximum value, and a mixture of said selected background image and said treated image in regions where said second control image has value between said minimum value and said maximum value.
Claim 27
Dependent Claim
15 words
The method of claim 18 further comprising the step of touching up said composite image.
References Cited (Backward Citations)
US4811084A
Cited
US5014328A
Cited
US5185808A
Cited
US5382980A
Cited
US5345313A
Cited
US5251016A
Cited
US5262856A
Cited
US5355174A
Cited
US5313304A
Cited
US5386242A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2000-05-12 | 2000-11-11 | 2001-05-13 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2004-05-12 | 2004-11-11 | 2005-05-13 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2008-05-12 | 2008-11-11 | 2009-05-13 | $7700.00 | Window Closed / Lapsed |