US6141433A — System and method for segmenting image regions fr…

Claim 1

Independent Claim 336 words

A method for extracting image information from a video frame for regions of the video frame that likely are objects of interest in a scene, comprising steps of:. (a) generating a first set of regions based upon differences between image information for the video frame and image information for a background image of the scene; (b) generating a second set of regions from the first set of regions based upon edge information for regions in the first set and edge information for the background image, wherein the step of generating a second set of regions comprises steps of: (b)(1) extracting edge information from each region in the first set of regions; (b)(2) extracting edge information for the background image; (b)(3) comparing edge information for each region of the first set of regions with edge information for the background image; (b)(4) generating a confidence value for each region in the first set of regions depending on whether pixels of a region and corresponding pixels in the background image represent edge information, wherein the step of generating a confidence value for each region in the first set of regions comprises steps of: examining each pixel of a region; if the pixel in the region represents an edge and a corresponding pixel in the background image represents an edge, then reducing the confidence value for the region; if the pixel in the region does not represent an edge and a corresponding pixel in the background image represents an edge, then increasing the confidence value for the region; and if the pixel in the region represents an edge and a corresponding pixel in the background image does not represent an edge, then increasing the confidence value for the region; (b)(5) retaining regions from the first set of regions which have a confidence value greater than a predetermined confidence threshold; and (c) generating a third set of regions from the second set of regions by combining regions in the second set with each other if resulting combined regions satisfy predetermined criteria.

Claim 2

Independent Claim 336 words

A method for extracting image information from a video frame for regions of the video frame that likely are objects of interest in a scene, comprising steps of:. (a) generating a first set of regions based upon differences between image information for the video frame and image information for a background image of the scene; (b) generating a second set of regions from the first set of regions based upon edge information for regions in the first set and edge information for the background image, wherein the step of generating a second set of regions comprises steps of: (b)(1) extracting edge information from each region in the first set of regions; (b)(2) extracting edge information for the background image; (b)(3) comparing edge information for each region of the first set of regions with edge information for the background image; (b)(4) generating a confidence value for each region in the first set of regions depending on whether pixels of a region and corresponding pixels in the background image represent edge information, wherein the step of generating a confidence value for each region in the first set of regions comprises steps of: examining each pixel of a region; if the pixel in the region represents an edge and a corresponding pixel in the background image represents an edge, then reducing the confidence value for the region; if the pixel in the region does not represent an edge and a corresponding pixel in the background image represents an edge, then increasing the confidence value for the region; and if the pixel in the region represents an edge and a corresponding pixel in the background image does not represent an edge, then increasing the confidence value for the region; (b)(5) retaining regions from the first set of regions which have a confidence value greater than a predetermined confidence threshold; and (c) generating a third set of regions from the second set of regions by combining regions in the second set with each other if resulting combined regions satisfy predetermined criteria.

Claim 3

Dependent Claim 115 words

The method of claim 1, wherein the step of generating the first set of regions comprises steps of: (a)(1) generating a luminance difference image and a color difference image based upon image information for the video frame and image information for the background image; (a)(2) forming a composite image from the luminance difference image and the color difference image; (a)(3) comparing the composite image with a predetermined image difference threshold to generate a binary interest image; (a)(4) generating a gray interest image by masking the luminance image information for the video frame with the binary interest image; and (a)(5) extracting from the gray interest image those regions that are connected and have similar gray levels.

Claim 4

Dependent Claim 56 words

The method of claim 2, and further comprising the step of: (a)(6) adjusting values of pixels in the luminance difference image based upon values of corresponding pixels in the background image to generate an adjusted luminance difference image; wherein the composite image is formed based on the adjusted luminance difference image and the color difference image.

Claim 5

Dependent Claim 48 words

The method of claim 3, wherein the step of adjusting values of pixels in the luminance difference image comprises multiplying a value of each pixel in the luminance difference image by a factor which is proportional to a luminance intensity of the corresponding pixel in the background image.

Claim 6

Dependent Claim 163 words

The method of claim 2, wherein the luminance image information is represented by a Y component of YUV image information for the video frame and the color image information is represented by a U component and a V component of YUV image information for the video frame, and wherein the step of generating a luminance difference image and a color difference image comprises: (a)(1)(i) generating a Y difference image representing a difference between a Y component of the image information for the video frame and a Y component of the image information for the background image; (a)(1)(ii) generating a U difference image representing a difference between a U component of the image information for the video frame and a U component of the image information for the background image; and (a)(1)(iii) generating a V difference image representing a difference between a V component of the image information for the video frame and a V component of the image information for the background image.

Claim 7

Dependent Claim 46 words

The method of claim 5, wherein the step of forming the composite image comprises steps of: (a)(3)(i) forming a combined UV difference image from the U difference image and the V difference image; and (a)(3)(ii) combining the combined UV difference image with the Y difference image.

Claim 8

Dependent Claim 34 words

The method of claim 6, and further comprising the step of: (a)(3)(iii) weighting the Y difference image by a predetermined emphasis factor to emphasize either color differences or intensity differences in the composite image.

Claim 9

Dependent Claim 32 words

The method of claim 5, wherein the step of generating a gray interest image comprises masking the Y component of the image information for the video frame with the binary interest image.

Claim 10

Dependent Claim 96 words

The method of claim 1, wherein the step of generating the first set of regions comprises steps of: (a)(1) generating a luminance difference image based upon image information for the video frame and image information for the background image; (a)(2) comparing the luminance difference image with a predetermined image difference threshold to generate a binary interest image; (a)(3) generating a gray interest image by masking the luminance image information of the video frame with the binary interest image; and (a)(4) extracting from the gray interest image those regions that are connected and have similar gray levels.

Claim 11

Dependent Claim 58 words

The method of claim 9, and further comprising the step of: (a)(5) adjusting values of pixels in the luminance difference image based upon values of corresponding pixels in the background image to generate an adjusted luminance difference image; wherein the adjusted luminance difference image is compared with the predetermined image difference threshold to generate the binary interest image.

Claim 12

Dependent Claim 95 words

The method of claim 1, wherein the step of generating the first set of regions comprises steps of: (a)(1) generating a color difference image based upon image information for the video frame and image information for the background image; (a)(2) comparing the color difference image with a predetermined image difference threshold to generate a binary interest image; (a)(3) generating a gray interest image by masking color image information of the video frame with the binary interest image; and (a)(4) extracting from the gray interest image those regions that are connected and have similar gray levels.

Claim 13

Dependent Claim 101 words

The method of claim 1, wherein the step of generating the third set of regions comprises steps of: (c)(1) determining which regions in the second set can be combined with other regions to form a region pair if the region pair satisfies predetermined criteria including at least one of, a predetermined size limit, sufficient proximity of the regions in the region pair, and overlap of morphological dilated versions of the regions in the region pair; and (c)(2) comparing region pairs with each other and merging together region pairs to form groups of regions from region pairs which have a common region.

Claim 14

Independent Claim 388 words

A system for extracting image information from a video frame for regions of the video frame that likely are objects of interest in a scene, comprising: (a) a video camera positioned to monitor the scene and generating video signals representing activity within the scene; (b) a frame grabber coupled to the video camera to generate a stream of video frames from the video signal, each video frame comprising image information of the scene at an instant of time; (c) a processor coupled to the frame grabber, the processor being programmed to: (c)(1) generate a first set of regions based upon differences between image information for the video frame and image information for a background image of the scene; (c)(2) generate a second set of regions from the first set of regions based upon edge information for regions in the first set and edge information for the background image by: (c)(2)(i) extracting edge information from each region in the first set of regions; (c)(2)(ii) extracting edge information for the background image; (c)(2)(iii) comparing edge information for each region of the first set of regions with edge information for the background image; (c)(2)(iv) generating a confidence value for each region in the first set of regions depending on whether pixels of a region and corresponding pixels in the background image represent edge information, wherein the processor is programmed to generate a confidence value for each region in the first set of regions by: examining each pixel of a region; if the pixel in the region represents an edge and a corresponding pixel in the background image represents an edge, then reducing the confidence value for the region; if the pixel in the region does not represent an edge and a corresponding pixel in the background image represents an edge, then increasing the confidence value for the region; and if the pixel in the region represents an edge and a corresponding pixel in the background image does not represent an edge, then increasing the confidence value for the region; and (c)(2)(v) retaining regions from the first set of regions which have a confidence value greater than a predetermined confidence threshold; and (c)(3) generate a third set of regions from the second set of regions by combining regions in the second set with each other if resulting combined regions satisfy predetermined criteria.

Claim 15

Independent Claim 388 words

A system for extracting image information from a video frame for regions of the video frame that likely are objects of interest in a scene, comprising: (a) a video camera positioned to monitor the scene and generating video signals representing activity within the scene; (b) a frame grabber coupled to the video camera to generate a stream of video frames from the video signal, each video frame comprising image information of the scene at an instant of time; (c) a processor coupled to the frame grabber, the processor being programmed to: (c)(1) generate a first set of regions based upon differences between image information for the video frame and image information for a background image of the scene; (c)(2) generate a second set of regions from the first set of regions based upon edge information for regions in the first set and edge information for the background image by: (c)(2)(i) extracting edge information from each region in the first set of regions; (c)(2)(ii) extracting edge information for the background image; (c)(2)(iii) comparing edge information for each region of the first set of regions with edge information for the background image; (c)(2)(iv) generating a confidence value for each region in the first set of regions depending on whether pixels of a region and corresponding pixels in the background image represent edge information, wherein the processor is programmed to generate a confidence value for each region in the first set of regions by: examining each pixel of a region; if the pixel in the region represents an edge and a corresponding pixel in the background image represents an edge, then reducing the confidence value for the region; if the pixel in the region does not represent an edge and a corresponding pixel in the background image represents an edge, then increasing the confidence value for the region; and if the pixel in the region represents an edge and a corresponding pixel in the background image does not represent an edge, then increasing the confidence value for the region; and (c)(2)(v) retaining regions from the first set of regions which have a confidence value greater than a predetermined confidence threshold; and (c)(3) generate a third set of regions from the second set of regions by combining regions in the second set with each other if resulting combined regions satisfy predetermined criteria.

Claim 16

Dependent Claim 115 words

The system of claim 13, wherein the processor is programmed to generate the first set of regions by: (c)(1)(i) generating a luminance difference image and a color difference image based upon image information for the video frame and image information for the background image; (c)(1)(ii) forming a composite image from the luminance difference image and the color difference image; (c)(1)(iii) comparing the composite image with a predetermined image difference threshold to generate a binary interest image; (c)(1)(iv) generating a gray interest image by masking the luminance image information for the video frame with the binary interest image; and (c)(1)(v) extracting from the gray interest image those regions that are connected and have similar gray levels.

Claim 17

Dependent Claim 61 words

The system of claim 14, wherein the processor is further programmed to perform the step of: (c)(1)(ii) adjusting values of pixels in the luminance difference image based upon values of corresponding pixels in the background image to generate an adjusted luminance difference image; wherein the composite image is formed based on the adjusted luminance difference image and the color difference image.

Claim 18

Dependent Claim 51 words

The system of claim 15, wherein the processor is further programmed to adjust values of pixels in the luminance difference image comprises multiplying a value of each pixel in the luminance difference image by a factor which is proportional to a luminance intensity of the corresponding pixel in the background image.

Claim 19

Dependent Claim 170 words

The system of claim 14, wherein the video camera generates a color video signal comprising luminance image information and color image information, the luminance image information represented by a Y component of YUV image information and the color image information comprises a U component and a V component of the YUV image information, wherein the processor is programmed to generate the luminance difference image and the color difference image by: (c)(1)(i)(A) generating a Y difference image representing a difference between a Y component of the image information for the video frame and a Y component of the image information for the background image; (c)(1)(i)(B) generating a U difference image representing a difference between a U component of the image information for the video frame and a U component of the image information for the background image; and (c)(1)(i)(C) generating a V difference image representing a difference between a V component of the image information for the video frame and a V component of the image information for the background image.

Claim 20

Dependent Claim 46 words

The system of claim 17, wherein the processor is programmed to form the composite image by: (c)(1)(iii)(A) forming a combined UV difference image from the U difference image and the V difference image; and (c)(1)(iii)(B) combining the combined UV difference image with the Y difference image.

Claim 21

Dependent Claim 40 words

The system of claim 18, wherein the processor is further programmed to form the composite image by: (c)(1)(iii)(C) weighting the Y difference image by a predetermined emphasis factor to emphasize either color differences or intensity differences in the composite image.

Claim 22

Dependent Claim 34 words

The system of claim 17, wherein the processor is programmed to generate a gray interest image by masking the Y component of the image information for the video frame with the binary interest image.

Claim 23

Dependent Claim 102 words

The system of claim 13, wherein the processor is programmed to generate the first set of regions by: (c)(1)(i) generating a luminance difference image based upon image information for the video frame and image information for the background image; (c)(1)(ii) comparing the luminance difference image with a predetermined image difference threshold to generate a binary interest image; (c)(1)(iii) generating a gray interest image by masking the luminance image information of the video frame with the binary interest image; and (c)(1)(iv) extracting from the gray interest image those regions that are connected and have similar gray levels to form the initial region set.

Claim 24

Dependent Claim 65 words

The system of claim 21, wherein the processor is further programmed to adjust the luminance difference image by: (c)(1)(v) adjusting values of pixels in the luminance difference image based upon values of corresponding pixels in the background image to generate an adjusted luminance difference image; wherein the adjusted luminance difference image is compared with the predetermined image difference threshold to generate the binary interest image.

Claim 25

Dependent Claim 96 words

The system of claim 13, wherein the processor is programmed to generate the first set of regions by: (c)(1)(i) generating a luminance difference image based upon image information for the video frame and image information for the background image; (c)(1)(ii) comparing the luminance difference image with a predetermined image difference threshold to generate a binary interest image; (c)(1)(iii) generating a gray interest image by masking the luminance image information of the video frame with the binary interest image; and (c)(1)(iv) extracting from the gray interest image those regions that are connected and have similar gray levels.

Claim 26

Dependent Claim 59 words

The system of claim 23, wherein the processor is further programmed to: (c)(1)(v) adjust values of pixels in the luminance difference image based upon values of corresponding pixels in the background image to generate an adjusted luminance difference image; wherein the adjusted luminance difference image is compared with the predetermined image difference threshold to generate the binary interest image.

Claim 27

Dependent Claim 96 words

The system of claim 13, wherein the processor is programmed to generate the first set of regions by: (c)(1)(i) generating a color difference image based upon image information for the video frame and image information for the background image; (c)(1)(ii) comparing the color difference image with a predetermined image difference threshold to generate a binary interest image; (c)(1)(iii) generating a gray interest image by masking the color image information of the video frame with the binary interest image; and (c)(1)(iv) extracting from the gray interest image those regions that are connected and have similar gray levels.

Claim 28

Dependent Claim 93 words

The system of claim 13, wherein the processor is programmed to generate the third set of regions by: (c)(3)(i) determining which regions in the second set can be combined with other regions to form a region pair if the region pair satisfies predetermined criteria including at least one of, a predetermined size limit, sufficient proximity of the regions, and overlap of morphological dilated versions of the regions; and (c)(3)(ii) comparing region pairs with each other and merging together region pairs to form groups of regions from regions pairs which have a common region.

Claim 29

Dependent Claim 347 words

A computer-readable medium storing executable instructions which cause a computer to extract image information from a video frame for regions of the video frame that likely are objects of interest in the scene, by: (a) generating a first set of regions based upon differences between image information for the video frame and image information for a background image of the scene; (b) generating a second set of regions from the first set of regions based upon edge information for regions in the first set and edge information for the background image and generating a second set of regions, wherein the step of generating the second set of regions comprises steps of (b)(1) extracting edge information from each region in the first set of regions; (b)(2) extracting edge information for the background image; (b)(3) comparing edge information for each region of the first set of regions with edge information for the background image; (b)(4) generating a confidence value for each region in the first set of regions depending on whether pixels of a region and corresponding pixels in the background image represent edge information, wherein the instructions for generating a confidence value for each region in the first set comprise instructions for: examining each pixel of a region; if the pixel in the region represents an edge and a corresponding pixel in the background image represents an edge, then reducing the confidence value for the region; if the pixel in the region does not represent an edge and a corresponding pixel in the background image represents an edge, then increasing the confidence value for the region; and if the pixel in the region represents an edge and a corresponding pixel in the background image does not represent an edge, then increasing the confidence value for the region; (b)(5) retaining regions from the first set of regions which have a confidence value greater than a predetermined confidence threshold; and (c) generating a third set of regions from the second set of regions by combining regions in the second set with each other if resulting combined regions satisfy predetermined criteria.

Claim 30

Dependent Claim 347 words

A computer-readable medium storing executable instructions which cause a computer to extract image information from a video frame for regions of the video frame that likely are objects of interest in the scene, by: (a) generating a first set of regions based upon differences between image information for the video frame and image information for a background image of the scene; (b) generating a second set of regions from the first set of regions based upon edge information for regions in the first set and edge information for the background image and generating a second set of regions, wherein the step of generating the second set of regions comprises steps of (b)(1) extracting edge information from each region in the first set of regions; (b)(2) extracting edge information for the background image; (b)(3) comparing edge information for each region of the first set of regions with edge information for the background image; (b)(4) generating a confidence value for each region in the first set of regions depending on whether pixels of a region and corresponding pixels in the background image represent edge information, wherein the instructions for generating a confidence value for each region in the first set comprise instructions for: examining each pixel of a region; if the pixel in the region represents an edge and a corresponding pixel in the background image represents an edge, then reducing the confidence value for the region; if the pixel in the region does not represent an edge and a corresponding pixel in the background image represents an edge, then increasing the confidence value for the region; and if the pixel in the region represents an edge and a corresponding pixel in the background image does not represent an edge, then increasing the confidence value for the region; (b)(5) retaining regions from the first set of regions which have a confidence value greater than a predetermined confidence threshold; and (c) generating a third set of regions from the second set of regions by combining regions in the second set with each other if resulting combined regions satisfy predetermined criteria.

Claim 31

Dependent Claim 117 words

The computer-readable medium of claim 27, wherein the executable instructions for generating the first set of regions comprise instructions for: (a)(1) generating a luminance difference image and a color difference image based upon image information for the video frame and image information for the background image; (a)(2) forming a composite image from the luminance difference image and the color difference image; (a)(3) comparing the composite image with a predetermined image difference threshold to generate a binary interest image; (a)(4) generating a gray interest image by masking the luminance image information for the video frame with the binary interest image; and (a)(5) extracting from the gray interest image those regions that are connected and have similar gray levels.

Claim 32

Dependent Claim 56 words

The computer-readable medium of claim 28, and further comprising instructions for: (a)(6) adjusting values of pixels in the luminance difference image based upon values of corresponding pixels in the background image to generate an adjusted luminance difference image; wherein the composite image is formed based on the adjusted luminance difference image and the color difference image.

Claim 33

Dependent Claim 50 words

The computer-readable medium of claim 9, wherein the instructions for adjusting the pixels in the luminance difference image comprise instructions for multiplying a value of each pixel in the luminance difference image by a factor which is proportional to a luminance intensity of the corresponding pixel in the background image.

Claim 34

Dependent Claim 164 words

The computer-readable medium of claim 28, wherein the luminance image information comprises a Y component of YUV image information for the video frame and the color image information is represented by a U component and a V component of YUV image information for the video frame, and wherein the instructions for generating a luminance difference image and a color difference image comprise instructions for: (a)(1)(i) generating a Y difference image representing a difference between a Y component of the image information for the video frame and a Y component of the image information for the background image; (a)(1)(ii) generating a U difference image representing a difference between a U component of the image information for the video frame and a U component of the image information for the background image; and (a)(1)(iii) generating a V difference image representing a difference between a V component of the image information for the video frame and a V component of the image information for the background image.

Claim 35

Dependent Claim 47 words

The computer-readable medium of claim 31, wherein the instructions for forming a composite image comprise instructions for: (a)(3)(i) forming a combined UV difference image from the U difference image and the V difference image; and (a)(3)(ii) combining the combined UV difference image with the Y difference image.

Claim 36

Dependent Claim 41 words

The computer-readable medium of claim 32, wherein the instructions for forming the composite image further comprise instructions for: (a)(3)(iii) weighting the Y difference image by a predetermined emphasis factor to emphasize either color differences or intensity differences in the composite image.

Claim 37

Dependent Claim 33 words

The computer-readable medium of claim 32, wherein the instructions for generating a gray interest image comprise masking the Y component of the image information for the video frame with the binary interest image.

Claim 38

Dependent Claim 103 words

The computer-readable medium of claim 27, wherein the instructions for generating the first set of regions comprise instructions for: (a)(1) generating a luminance difference image based upon image information for the video frame and image information for the background image; (a)(2) comparing the luminance difference image with a predetermined image difference threshold to generate a binary interest image; (a)(3) generating a gray interest image by masking the luminance image information of the video frame with the binary interest image; and (a)(4) extracting from the gray interest image those regions that are connected and have similar gray levels to form the initial region set.

Claim 39

Dependent Claim 57 words

The computer-readable medium of claim 35, and further comprising instructions for: adjusting values of pixels in the luminance difference image based upon values of corresponding pixels in the background image to generate an adjusted luminance difference image; wherein the adjusted luminance difference image is compared with the predetermined image difference threshold to generate the binary interest image.

Claim 40

Dependent Claim 96 words

The computer-readable medium of claim 27, wherein the instructions for generate the first set of regions comprise instructions for: (a)(1) generating a color difference image based upon image information for the video frame and image information for the background image; (a)(2) comparing the color difference image with a predetermined image difference threshold to generate a binary interest image; (a)(3) generating a gray interest image by masking color image information of the video frame with the binary interest image; and (a)(4) extracting from the gray interest image those regions that are connected and have similar gray levels.

Claim 41

Dependent Claim 94 words

The computer-readable medium of claim 27, wherein the instructions for generating the third set of regions comprise instructions for: (c)(1) determining which regions in the second set can be combined with other regions to form a region pair if the region pair satisfies predetermined criteria including at least one of, a predetermined size limit, sufficient proximity of the regions, and overlap of morphological dilated versions of the regions; and (c)(2) comparing region pairs with each other and merging together region pairs to form groups or regions from region pairs which have a common region.

Annuity Stage	Window Opens	Payment Deadline	Surcharge Period Until	Estimated Fee	Status
3.5 Years (Year 4)	2003-10-31	2004-05-01	2004-10-31	$2000.00	Window Closed / Lapsed
7.5 Years (Year 8)	2007-10-31	2008-05-01	2008-10-31	$3760.00	Window Closed / Lapsed
11.5 Years (Year 12)	2011-10-31	2012-05-01	2012-10-31	$7700.00	Window Closed / Lapsed

System and method for segmenting image regions from a scene likely to represent particular objects in the scene

Abstract

Inventors

Bibliographic Details

Claims Analysis

References Cited (Backward Citations)

USPTO Annuity & Maintenance Windows

Assigned Categories & Tags

Assign Category Tag