US6160907A — Iterative three-dimensional process for creating …

Claim 1

Independent Claim 300 words

A method for developing finished media content from an input visual image sequence, the input visual image sequence containing a visual representation of one or more visual objects, the method comprising the steps of: (a) analyzing portions of the input visual image sequence to derive a scene model in three spatial dimensions with respect to a three-dimensional object space by performing the steps of: (i) defining a common three-dimensional object space; (ii) defining an image-based data object containing a segmented pixel representation corresponding to a portion of at least one frame of the input visual image sequence within the three-dimensional object space; (iii) defining an abstraction-based data object containing an abstract model for at least a portion of one of the visual objects contained in the input visual image sequence within the same three-dimensional object space; (b) defining a choreography specification for the finished media content that includes a specification for how the data objects are to be used in creating the finished media content; (c) selecting an image-based data object to be modified and accepting user input parameters modifying how the selected image-based data obiect is to be used in the choreography specification for generating the finished media content; (d) selecting an abstraction-based data object to be modified and accepting user input parameters modifying how the selected abstraction-based data object is to be used in the choreography specification for generating the finished media content; (e) generating a two-dimensional (2D) visual image of the finished media content from the choreography specification, scene model, at least one pixel representation, and at least one abstract model; and (f) iteratively refining the finished media content by performing certain selected ones of steps (a) through (e) in an order as selected by user input until the finished media content is of acceptable finish quality.

Claim 2

Independent Claim 300 words

A method for developing finished media content from an input visual image sequence, the input visual image sequence containing a visual representation of one or more visual objects, the method comprising the steps of: (a) analyzing portions of the input visual image sequence to derive a scene model in three spatial dimensions with respect to a three-dimensional object space by performing the steps of: (i) defining a common three-dimensional object space; (ii) defining an image-based data object containing a segmented pixel representation corresponding to a portion of at least one frame of the input visual image sequence within the three-dimensional object space; (iii) defining an abstraction-based data object containing an abstract model for at least a portion of one of the visual objects contained in the input visual image sequence within the same three-dimensional object space; (b) defining a choreography specification for the finished media content that includes a specification for how the data objects are to be used in creating the finished media content; (c) selecting an image-based data object to be modified and accepting user input parameters modifying how the selected image-based data obiect is to be used in the choreography specification for generating the finished media content; (d) selecting an abstraction-based data object to be modified and accepting user input parameters modifying how the selected abstraction-based data object is to be used in the choreography specification for generating the finished media content; (e) generating a two-dimensional (2D) visual image of the finished media content from the choreography specification, scene model, at least one pixel representation, and at least one abstract model; and (f) iteratively refining the finished media content by performing certain selected ones of steps (a) through (e) in an order as selected by user input until the finished media content is of acceptable finish quality.

Claim 3

Dependent Claim 24 words

A method as in claim 1 wherein step (b) additionally comprises: choreographing representations of objects within the context of a three-dimensional (3D) virtual stage.

Claim 4

Dependent Claim 23 words

A method as in claim 1 wherein at least one parameter of a virtual stage is defined by analyzing the input visual image.

Claim 5

Dependent Claim 38 words

A method as in claim 1 additionally comprising the step of, after step (a): storing a link in correlation mesh data object wherein the link indicates a correspondence between an image-based data object and an abstraction-based data object.

Claim 6

Dependent Claim 28 words

A method as in claim 1 wherein step (a) further comprises the step of: defining a spatial position for the pixel representation with respect to a virtual stage.

Claim 7

Dependent Claim 26 words

A method as in claim 1 wherein the input visual image is a visual image stream comprising a time sequence of two or more image frames.

Claim 8

Dependent Claim 40 words

A method as in claim 6 wherein step (b) additionally comprises the step of: defining a choreography specification that includes a specification of one or more time-sequenced manipulations of at least one of the abstract models in a virtual stage.

Claim 9

Dependent Claim 38 words

A method as in claim 7 wherein step (e) further comprises the step of: generating a time-based sequence of 2D output visual images from at least one of the choreography specification, the abstract models, and the pixel representations.

Claim 10

Dependent Claim 44 words

A method as in claim 1 wherein step (f) additionally comprises the step of: during a given iteration, accepting user input specifying which of a plurality of pixel representations and which of a plurality of abstract models are used in the output generating step.

Claim 11

Dependent Claim 31 words

A method as in claim 1 additionally comprising the step of: defining a camera model including a specification of a viewpoint of a camera used to generate the input visual image.

Claim 12

Dependent Claim 31 words

A method as in claim 10 wherein step (e) further comprises the step of: generating the output visual image from the viewpoint of the camera as specified by the camera model.

Claim 13

Dependent Claim 41 words

A method as in claim 10 wherein the camera model additionally specifies at least one camera parameter taken from the group consisting of camera position, movement, resolution, field of view, depth of field, focal length, lens f-stop, exposure interval and orientation.

Claim 14

Dependent Claim 19 words

A method as in claim 1 wherein the visual object in the input visual image comprises a physical object.

Claim 15

Dependent Claim 19 words

A method as in claim 1 wherein the visual object in the input visual image comprises a synthetic object.

Claim 16

Dependent Claim 16 words

A method as in claim 1 wherein the abstract models comprise models of multiple physical objects.

Claim 17

Dependent Claim 19 words

A method as in claim 1 wherein the abstract models comprise a model of at least one synthetic object.

Claim 18

Dependent Claim 57 words

A method as in claim 1 additionally comprising the step of: defining a light model specifying at least one light parameter of a light that can be used to generate the output visual image, with at least one light parameter taken from the group consisting of light position, light movement, light intensity, light color, and light orientation.

Claim 19

Dependent Claim 22 words

A method as in claim 1 additionally comprising the step of: generating an audio stream synchronized with the output visual image stream.

Claim 20

Dependent Claim 33 words

A method as in claim 1 wherein the step (e) of generating an output visual image additionally comprises the step of: determining at least one visual effect on at least one visual object.

Claim 21

Dependent Claim 25 words

A method as in claim 1 wherein step (b) further comprises additionally comprises specifying a visual effect upon at least one of the visual objects.

Claim 22

Dependent Claim 24 words

A method as in claim 20 wherein the visual effect comprises a visual effect selected from the group consisting of shadows, reflections, and refractions.

Claim 23

Dependent Claim 25 words

A method as in claim 20 wherein the visual effect comprises a visual effect selected from the group consisting of atmospheric and object geometry distortions.

Claim 24

Dependent Claim 15 words

A method as in claim 20 wherein the visual effect comprises an inter-object morphing effect.

Claim 25

Dependent Claim 19 words

A method as in claim 20 wherein the visual effect comprises changes to at least one object surface attribute.

Claim 26

Dependent Claim 23 words

A method as in claim 24 wherein the object surface attribute is selected from the group consisting of surface texture, reflectivity, and opacity.

Claim 27

Dependent Claim 30 words

A method as in claim 1 additionally comprising the step of: generating an input specification and parameters for an external 3D rendering system used in the output generating step (e).

Claim 28

Dependent Claim 41 words

A method as in claim 1 additionally comprising the step of: accepting a user input specifying how a visual image specified by more than one pixel representation and abstract model is to be blended into a single composite output visual image.

Claim 29

Dependent Claim 34 words

A method as in claim 1 wherein step (b) of defining a choreography specification additionally comprises the step of: specifying the blending of multiple pixel representations and abstract models of a given visual object.

Claim 30

Dependent Claim 50 words

A method as in claim 28 wherein the blending specification step additionally comprises the step of: rendering each of a plurality of abstract models first to a two-dimensional 2D pixel representation, each 2D pixel representation including a plurality of pixel data elements, and then blending the multiple 2D pixel representations.

Claim 31

Dependent Claim 25 words

A method as in claim 29 wherein step blending specification step additionally comprises the step of: determining an image blending attribute for the visual object.

Claim 32

Dependent Claim 30 words

A method as in claim 30 wherein the image blending attribute is at least one attribute selected from the group consisting of depth from camera, image transparency, and surface normal.

Claim 33

Dependent Claim 31 words

A method as in claim 30 wherein step (e) additionally comprises the step of: blending multiple 2D pixel representations by comparing the image blending attribute on a pixel by pixel basis.

Claim 34

Dependent Claim 33 words

A method as in claim 30 wherein step (e) additionally comprises the step of: blending multiple 2D pixel representations by comparing the image blending attribute on a pixel region by pixel region basis.

Claim 35

Dependent Claim 31 words

A method as in claim 30 wherein step (e) additionally comprises the step of: blending multiple 2D pixel representations by comparing the image blending attribute on a sub-pixel by sub-pixel basis.

Claim 36

Dependent Claim 36 words

A method as in claim 4 additionally comprising the step of: during a given iteration, accepting user input specifying changes to the correlation mesh in response to display of the 2D finish representation to the user.

Claim 37

Dependent Claim 31 words

A method as in claim 1 additionally comprising the step of: during a given iteration, accepting user input specifying changes to parameters of an automatic segmenting process that performs step (a)(i).

Claim 38

Dependent Claim 32 words

A method as in claim 1 additionally comprising the step of: during a given iteration, accepting user input specifying changes to parameters of an automatic abstract modeling process that performs step (a)(ii).

Claim 39

Dependent Claim 53 words

A method as in claim 1 wherein step (a)(i) additionally comprises the steps of: creating multiple pixel representation versions of a given input visual image, the multiple pixel representation versions being at different levels of image resolution; and using different ones of the pixel representation versions in given iterations of performing step (a)(i).

Claim 40

Dependent Claim 43 words

A method as in claim 38 additionally comprising the step of: presenting an interactive display of an intermediate finish representation from a given iteration whereby a user specifies which one of the pixel representation versions are to be used in a subsequent iteration.

Claim 41

Dependent Claim 18 words

A method as in claim 38 wherein the different levels of image resolution are for different spatial resolutions.

Claim 42

Dependent Claim 19 words

A method as in claim 38 wherein the different levels of image resolution are for different color space resolutions.

Claim 43

Dependent Claim 57 words

A method as in claim 1 wherein the step of defining an abstract model additionally comprises the steps of: creating multiple abstract model versions of a given input visual image, the abstract model versions being at different levels of modeling detail; and using different ones of the abstract model versions in given iterations of performing step (f).

Claim 44

Dependent Claim 43 words

A method as in claim 42 additionally comprising the step of: presenting an interactive display of an intermediate finish representation from a given iteration whereby a user specifies which one of the abstract model versions are to be used in a subsequent iteration.

Claim 45

Dependent Claim 19 words

A method as in claim 42 wherein the different levels of modeling detail are different levels of geometry detail.

Claim 46

Dependent Claim 19 words

A method as in claim 42 wherein the different levels of modeling detail are different representations of surface textures.

Claim 47

Dependent Claim 32 words

A method as in claim 7 additionally comprising the step of: providing an application programming interface that permits access to the pixel representation(s), abstract model(s) choreography specification, correlation mesh, or visual image(s).

Claim 48

Dependent Claim 34 words

A method as in claim 7 additionally comprising the step of: modifying the output audio stream in accordance with changes made to at least one object in the virtual stage through the choreography step.

Claim 49

Dependent Claim 223 words

A data processing system for generating finish quality media from one or more source images of a scene comprising: a scene model developed by analyzing portions of the source images, the scene model defining (i) a three-dimensional virtual space; (ii) an image-based data object containing a segmented pixel representation corresponding to a portion of at least one frame of the source images; (iii) an abstraction-based data object containing an abstract model for at least a portion of one of the visual objects contained in at least one of the source images, the abstraction-based data object being defined with respect to the three-dimensional virtual space; a choreography specification for defining how the finish quality media is to be developed from the scene model and both the image-passed and abstraction-based data objects; means for accepting user input parameters modifying how selected data objects are to be used with the choreography specification in generating the finish quality media; an image renderer, connected to receive the scene model, the data objects, and the choreography specification as inputs, for rendering a two-dimensional visual representation of each data object; means for accepting user inputs in response to the two-dimensional visual representation produced by the image renderer, the user inputs defining how the data objects and the choreography specification are to he used in iteratively refining the finish quality media.

Claim 50

Dependent Claim 223 words

A data processing system for generating finish quality media from one or more source images of a scene comprising: a scene model developed by analyzing portions of the source images, the scene model defining (i) a three-dimensional virtual space; (ii) an image-based data object containing a segmented pixel representation corresponding to a portion of at least one frame of the source images; (iii) an abstraction-based data object containing an abstract model for at least a portion of one of the visual objects contained in at least one of the source images, the abstraction-based data object being defined with respect to the three-dimensional virtual space; a choreography specification for defining how the finish quality media is to be developed from the scene model and both the image-passed and abstraction-based data objects; means for accepting user input parameters modifying how selected data objects are to be used with the choreography specification in generating the finish quality media; an image renderer, connected to receive the scene model, the data objects, and the choreography specification as inputs, for rendering a two-dimensional visual representation of each data object; means for accepting user inputs in response to the two-dimensional visual representation produced by the image renderer, the user inputs defining how the data objects and the choreography specification are to he used in iteratively refining the finish quality media.

Claim 51

Dependent Claim 16 words

A system as in claim 48 wherein the scene model is developed from two source images.

Claim 52

Dependent Claim 15 words

A system as in claim 49 wherein the two source images comprise two live/recorded images.

Claim 53

Dependent Claim 19 words

A system as in claim 49 wherein the two source images comprise a live/recorded image and a synthetic image.

Claim 54

Dependent Claim 15 words

A system as in claim 49 wherein the two source images comprise two synthetic images.

Claim 55

Dependent Claim 24 words

A system as in claim 48 wherein the image renderer additionally blends the rendered data objects using depth information derived from the source images.

Claim 56

Dependent Claim 19 words

A system as in claim 48 wherein the image renderer additionally blends the rendered data objects using transparency information.

Claim 57

Dependent Claim 18 words

A system as in claim 48 wherein the scene model includes image-based data objects and abstract data objects.

Annuity Stage	Window Opens	Payment Deadline	Surcharge Period Until	Estimated Fee	Status
3.5 Years (Year 4)	2003-12-12	2004-06-12	2004-12-12	$2000.00	Window Closed / Lapsed
7.5 Years (Year 8)	2007-12-12	2008-06-12	2008-12-12	$3760.00	Window Closed / Lapsed
11.5 Years (Year 12)	2011-12-12	2012-06-12	2012-12-12	$7700.00	Window Closed / Lapsed

Iterative three-dimensional process for creating finished media content

Abstract

Inventors

Bibliographic Details

Claims Analysis

References Cited (Backward Citations)

USPTO Annuity & Maintenance Windows

Assigned Categories & Tags

Assign Category Tag