US5946673A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Computer implemented machine learning and control system
Assignee: Individual
•
Filing: 1996-07-12
•
Granted: 1999-08-31
•
Expires: 2016-07-12
Abstract
Abstract In a computer implemented learning and/or process control system, a computer model is constituted by the most currently fit entity in a population of computer program entities. The computer model defines fitness as a function of inputs and outputs. A computing unit accesses the model with a set of inputs, and determines a set of outputs for which the fitness is highest. This associates a sensory-motor (input-output) state with a fitness in a manner that might be termed "feeling". The learning and/or control system preferably utilizes a compiling Genetic Programming system (CGPS) in which one or more machine code entities such as functions are created which represent solutions to a problem and are directly executable by a computer. The programs are created and altered by a program in a higher level language such as "C" which is not directly executable, but requires translation into executable machine code through compilation, interpretation, translation, etc. The entities are initially created as an integer array that can be altered by the program as data, and are executed by the program by recasting a pointer to the array as a function type. The entities are evaluated by executing them with training data as inputs, and calculating fitnesses based on a predetermined criterion. The entities are then altered based on their fitnesses using a genetic machine learning algorithm by recasting the pointer to the array as a data (e.g. integer) type. This process is iteratively repeated until an end criterion is reached.
Inventors
- 👤 Frank D. Francone
- 👤 Peter Nordin
- 👤 Wolfgang Banzhaf
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-31 11:14
CPC Classifications
B; B25; B25J; B25J9/00; B25J9/16; B25J9/1628; B25J9/163; G; G06; G06N; G06N20/00; G06N3/00; G06N3/12; G06N3/126
Claims Analysis
Total Claims: 76
Claim 1
Independent Claim
90 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; and a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; in which the computer model defines fitness as a function of input and output; further comprising a learning unit configured to derive the computer model; in which the learning unit is configured to derive the computer model using empirical fitness of input and output.
Claim 2
Dependent Claim
90 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; and a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; in which the computer model defines fitness as a function of input and output; further comprising a learning unit configured to derive the computer model; in which the learning unit is configured to derive the computer model using empirical fitness of input and output.
Claim 3
Dependent Claim
22 words
A system as in claim 1, in which the learning unit is configured to derive the computer model using a learning algorithm.
Claim 4
Dependent Claim
20 words
A system as in claim 1, in which the learning unit is configured to derive the computer model using regression.
Claim 5
Dependent Claim
21 words
A system as in claim 1, in which the learning unit is configured to derive the computer model using symbolic regression.
Claim 6
Dependent Claim
23 words
A system as in claim 1, in which the learning unit is configured to derive the computer model using a machine learning system.
Claim 7
Dependent Claim
26 words
A system as in claim 1, in which the learning unit is configured to derive the computer model using a fitting algorithm applied to training data.
Claim 8
Dependent Claim
18 words
A system as in claim 1, in which the computing unit is configured to receive applied inputs serially.
Claim 9
Dependent Claim
18 words
A system as in claim 1, in which the computing unit is configured to receive applied inputs periodically.
Claim 10
Dependent Claim
20 words
A system as in claim 1, in which the learning unit and the computing unit operate independently and interact intermittently.
Claim 11
Dependent Claim
25 words
A system as in claim 9, in which said intermittent interaction comprises the computing unit periodically accessing the learning unit to obtain the computer model.
Claim 12
Dependent Claim
23 words
A system as in claim 1, in which the learning unit is configured to derive the computer model by altering a single entity.
Claim 13
Dependent Claim
14 words
A system as in claim 11, in which the entity comprises a computer program.
Claim 14
Dependent Claim
27 words
A system as in claim 1, in which the learning unit is configured to derive the computer model by altering a plurality of a population of entities.
Claim 15
Dependent Claim
13 words
A system as in claim 13, in which the entities comprise computer programs.
Claim 16
Dependent Claim
21 words
A system as in claim 1, in which the learning unit is configured to derive different computer models at different times.
Claim 17
Dependent Claim
61 words
A system as in claim 1, in which: the computer model defines fitness as a function of input and output; and the computing unit is configured to calculate, using the computer model, for an applied input and at least a plurality of outputs, corresponding fitnesses, and select the output for which the corresponding fitness is best according to a predetermined criterion.
Claim 18
Dependent Claim
24 words
A system as in claim 1, in which the learning unit includes an entity which defines fitness as a function of input and output.
Claim 19
Dependent Claim
14 words
A system as in claim 17, in which the entity comprises a computer program.
Claim 20
Dependent Claim
29 words
A system as in claim 17, in which the learning unit is configured to derive the computer model by an algorithm that includes the step of altering the entity.
Claim 21
Dependent Claim
26 words
A system as in claim 1, in which the learning unit includes a population of entities which define fitness as a function of input and output.
Claim 22
Dependent Claim
13 words
A system as in claim 20, in which the entities include computer programs.
Claim 23
Dependent Claim
32 words
A system as in claim 20, in which the learning unit is configured to derive the computer model by an algorithm which includes the step of altering a plurality of the entities.
Claim 24
Dependent Claim
39 words
A system as in claim 1, further comprising an autonomous agent, in which: the computing unit is configured to apply said output to control the autonomous agent; and the applied input represents a parameter relating to the autonomous agent.
Claim 25
Dependent Claim
23 words
A system as in claim 23, further comprising a sensor configured to sense said parameter and apply said input to the computing unit.
Claim 26
Dependent Claim
61 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; and a learning unit configured to derive the computer model; using a compiling genetic programming system.
Claim 27
Dependent Claim
61 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; and a learning unit configured to derive the computer model; using a compiling genetic programming system.
Claim 28
Dependent Claim
67 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; and a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; in which the computing unit is configured to use the output determined by the computing unit to control a real-time process.
Claim 29
Dependent Claim
67 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; and a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; in which the computing unit is configured to use the output determined by the computing unit to control a real-time process.
Claim 30
Dependent Claim
21 words
A system as in claim 26, in which at least one applied input represents a parameter relating to the real-time process.
Claim 31
Dependent Claim
82 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; a learning unit configured to derive the computer model; and a fitness calculation module, which is configured to calculate the empirical fitness of an output previously determined by the computing unit in response to an applied input.
Claim 32
Dependent Claim
82 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; a learning unit configured to derive the computer model; and a fitness calculation module, which is configured to calculate the empirical fitness of an output previously determined by the computing unit in response to an applied input.
Claim 33
Dependent Claim
32 words
A system as in claim 28, in which the learning unit is configured to use at least one evaluation of empirical fitness by the fitness calculation module to derive the computer model.
Claim 34
Dependent Claim
32 words
A system as in claim 28, in which the learning unit is configured to use at least one evaluation of empirical fitness by the fitness calculation module to improve the computer model.
Claim 35
Dependent Claim
62 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; and a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; in which the computer model defines fitness as including a "pain" component and a "pleasure" component.
Claim 36
Dependent Claim
62 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; and a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; in which the computer model defines fitness as including a "pain" component and a "pleasure" component.
Claim 37
Dependent Claim
88 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; and a learning unit which includes: a memory for storing a plurality of vectors, each of which includes an input, an output, and a corresponding fitness; and a derivation unit configured to derive the computer model using the vectors as a training set.
Claim 38
Dependent Claim
88 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; and a learning unit which includes: a memory for storing a plurality of vectors, each of which includes an input, an output, and a corresponding fitness; and a derivation unit configured to derive the computer model using the vectors as a training set.
Claim 39
Dependent Claim
32 words
A system as in claim 32, in which: the fitness in at least one of the vectors is an empirical fitness; and the fitness defined by computer model is a predicted fitness.
Claim 40
Dependent Claim
28 words
A system as in claim 32, in which the learning unit is further configured to prevent at least one of the vectors stored in memory from being removed.
Claim 41
Dependent Claim
34 words
A system as in claim 32, in which the learning unit is further configured to temporarily prevent at least one of the vectors stored in memory from being removed according to a predetermined criterion.
Claim 42
Dependent Claim
38 words
A system as in claim 32, in which the learning unit is further configured to make at least one of the vectors stored in memory more difficult to remove than at least one other vector stored in memory.
Claim 43
Dependent Claim
37 words
A system as in claim 32, in which the derivation unit is configured to alter, using an algorithm, a population of entities, at least one of which defines fitness as an alterable function of input and output.
Claim 44
Dependent Claim
50 words
A system as in claim 32, in which the computing unit is further configured to calculate, using the computer model, for an applied input and at least a plurality of outputs, corresponding fitnesses, and to select the output for which the corresponding fitness is best according to a predetermined criterion.
Claim 45
Dependent Claim
20 words
A system as in claim 32, in which the derivation unit and the computing unit operate independently and interact intermittently.
Claim 46
Dependent Claim
39 words
A system as in claim 32, in which the system further comprises a fitness calculation module, which is configured to calculate the empirical fitness of an output previously determined by the computing unit in response to an applied input.
Claim 47
Dependent Claim
25 words
A system as in claim 40, in which the learning unit is configured to use said evaluation of empirical fitness to derive the computer model.
Claim 48
Dependent Claim
25 words
A system as in claim 40, in which the learning unit is configured to use said evaluation of empirical fitness to improve the computer model.
Claim 49
Dependent Claim
17 words
A system as in claim 32, in which at least one of the vectors is determined empirically.
Claim 50
Dependent Claim
19 words
A system as in claim 32, in which at least one of the vectors is determined from past experience.
Claim 51
Dependent Claim
16 words
A system as in claim 32, in which at least one of the vectors is preset.
Claim 52
Dependent Claim
15 words
A system as in claim 32, in which each vector further includes a state variable.
Claim 53
Dependent Claim
16 words
A system as in claim 32, in which each input includes at least one sensed parameter.
Claim 54
Dependent Claim
22 words
A system as in claim 32, in which the learning unit is further configured to periodically store new vectors in the memory.
Claim 55
Dependent Claim
32 words
A system as in claim 48, in which the learning unit is further configured to prevent at least one of the vectors which was first stored in the memory from being replaced.
Claim 56
Dependent Claim
64 words
A system as in claim 32, in which: the computer model includes an entity which defines fitness as a function of input and output; and the computing unit is configured to calculate, using the entity, for an applied input and at least a plurality of outputs, corresponding fitnesses, and select the output for which the corresponding fitness is best according to a predetermined criterion.
Claim 57
Dependent Claim
56 words
A system as in claim 32, in which the derivation unit is configured to alter, using an algorithm, a population of computer entities, each of which defines fitness as an alterable function of an input and an output, and produce the computer model as the entity which best fits the vectors according to a predetermined criterion.
Claim 58
Dependent Claim
27 words
A system as in claim 51, in which: the fitnesses included in the vectors are empirical fitnesses; and the fitnesses defined by the entities are predicted fitnesses.
Claim 59
Dependent Claim
13 words
A system as in claim 51, in which the algorithm includes genetic alteration.
Claim 60
Dependent Claim
13 words
A system as in claim 51, in which the algorithm includes tournament selection.
Claim 61
Dependent Claim
76 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; and an autonomous agent, in which: the computing unit is configured to apply said output to control the autonomous agent; and said fitness includes a "pain" component and a "pleasure" component.
Claim 62
Dependent Claim
76 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; and an autonomous agent, in which: the computing unit is configured to apply said output to control the autonomous agent; and said fitness includes a "pain" component and a "pleasure" component.
Claim 63
Dependent Claim
37 words
A system as in claim 55, in which: the "pain" component increases as the behavior of the autonomous agent becomes less desirable; and the "pleasure" component increases as the behavior of the autonomous agent becomes more desirable.
Claim 64
Dependent Claim
118 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; an autonomous agent, in which the computing unit is configured to apply said output to the autonomous agent; and the applied input represents a parameter relating to the autonomous agent; and a learning unit which includes: a memory for storing a plurality of vectors, each of which includes an input, an output, and a corresponding fitness; and a derivation unit configured to derive the computer model using the vectors as a training set.
Claim 65
Dependent Claim
118 words
A computer implemented system, comprising: a computer model which defines a relationship between inputs, outputs, and corresponding fitnesses; a computing unit for accessing the computer model to determine, in response to an applied input, an output for which a corresponding fitness meets a predetermined criterion; an autonomous agent, in which the computing unit is configured to apply said output to the autonomous agent; and the applied input represents a parameter relating to the autonomous agent; and a learning unit which includes: a memory for storing a plurality of vectors, each of which includes an input, an output, and a corresponding fitness; and a derivation unit configured to derive the computer model using the vectors as a training set.
Claim 66
Dependent Claim
20 words
A system as in claim 57, in which the derivation unit and the autonomous agent operate independently and interact intermittently.
Claim 67
Dependent Claim
29 words
A system as in claim 57, in which: the fitnesses included in the vectors are empirical fitnesses; and the fitness defined by the computer model is a predicted fitness.
Claim 68
Dependent Claim
77 words
A system as in claim 57, in which the learning unit is further configured to wait until a predetermined event occurs after the computing unit applies an output corresponding to a previous applied input to the autonomous agent, receive a new applied input, calculate an empirical fitness as a predetermined function of said new applied input and said output, and store said previous applied input, said output, and said empirical fitness as a vector in the memory.
Claim 69
Dependent Claim
20 words
A system as in claim 60, in which the predetermined event is a lapse of a predetermined length of time.
Claim 70
Dependent Claim
27 words
A system as in claim 60, in which the system further comprises a sensor configured to sense said parameter and apply said input to the computing unit.
Claim 71
Dependent Claim
19 words
A system as in claim 60, in which said predetermined function includes a "pain" component and a "pleasure" component.
Claim 72
Dependent Claim
38 words
A system as in claim 57, in which: the autonomous agent is a propelled object; the input represents proximity of the propelled object to external objects; and the output represents a propulsion control parameter for the propelled object.
Claim 73
Dependent Claim
23 words
A system as in claim 64, in which the fitnesses included in the vectors each include a "pain" component and a "pleasure" component.
Claim 74
Dependent Claim
33 words
A system as in claim 65, in which: the "pain" component increases with proximity to external objects; and the "pleasure" component increases with at least one of propulsion speed and straightness of movement.
Claim 75
Dependent Claim
44 words
A system as in claim 64, in which: the propelled object comprises a propulsion motor and a plurality of proximity sensors; the input comprises a vector including outputs of the proximity sensors; and the output comprises a speed control value for the propulsion motor.
Claim 76
Dependent Claim
14 words
A system as in claim 64, in which the propelled object is a robot.
References Cited (Backward Citations)
US4697242A
Cited
US5410634A
Cited
US5414865A
Cited
US4821333A
Cited
US4881178A
Cited
US5255345A
Cited
US5222192A
Cited
US5148513A
Cited
US5343554A
Cited
US4935877A
Cited
US4961152A
Cited
US5140530A
Cited
US5182793A
Cited
US5471593A
Cited
US5111531A
Cited
US5249259A
Cited
US5136686A
Cited
US5048095A
Cited
US5172253A
Cited
US5113482A
Cited
US5150289A
Cited
US5159660A
Cited
US5257343A
Cited
US5367612A
Cited
US5504841A
Cited
US5245696A
Cited
US5400436A
Cited
US5224176A
Cited
US5566275A
Cited
US5301256A
Cited
US5274744A
Cited
US5448681A
Cited
US5394509A
Cited
US5390282A
Cited
US5390283A
Cited
US5440723A
Cited
US5434796A
Cited
US5428709A
Cited
US5493631A
Cited
US5608843A
Cited
EP0696000A1
Cited
US5455938A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2002-08-30 | 2003-03-01 | 2003-08-31 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2006-08-30 | 2007-03-01 | 2007-08-31 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2010-08-30 | 2011-03-01 | 2011-08-31 | $7700.00 | Window Closed / Lapsed |