US5299285A — Neural network with dynamically adaptable neurons

Claim 1

Independent Claim 224 words

In a neural network employing a plurality of neurons each associated with a respective conductor in a network of conductors which are selectively interconnected by a plurality of problem-defining synpases each having an adjustable weighting factor whereby each neuron receives a weighted sum of inputs from plural conductors of a previous layer of said network and produces an output to a conductor in a following layer of said network, wherein the neural network is parameterized in an iterative learning process in which problem-defining signals are input to the neurons whereby to produce error signals between actual outputs from the network and expected outputs from the network and said error signals are used to incrementally change each weighting factor, an improvement for reducing the number of iterations required from the neural network to learn how to solve a problem of interest comprising: in each neuron between an input thereof for receiving said weighted sum of inputs and an output therefor for outputting an output signal value, including a neural conductive element having a variable gain defining said output signal value as a function of (a) said variable gain and (b) said weighted sum of inputs and gain adjustment logic means for dynamically adjusting the variable gain of the neuron independently of the other neurons in the network during a learning process of the neural network.

Claim 2

Dependent Claim 224 words

In a neural network employing a plurality of neurons each associated with a respective conductor in a network of conductors which are selectively interconnected by a plurality of problem-defining synpases each having an adjustable weighting factor whereby each neuron receives a weighted sum of inputs from plural conductors of a previous layer of said network and produces an output to a conductor in a following layer of said network, wherein the neural network is parameterized in an iterative learning process in which problem-defining signals are input to the neurons whereby to produce error signals between actual outputs from the network and expected outputs from the network and said error signals are used to incrementally change each weighting factor, an improvement for reducing the number of iterations required from the neural network to learn how to solve a problem of interest comprising: in each neuron between an input thereof for receiving said weighted sum of inputs and an output therefor for outputting an output signal value, including a neural conductive element having a variable gain defining said output signal value as a function of (a) said variable gain and (b) said weighted sum of inputs and gain adjustment logic means for dynamically adjusting the variable gain of the neuron independently of the other neurons in the network during a learning process of the neural network.

Claim 3

Dependent Claim 28 words

The improvement to a neural network of claim 1 wherein: said gain adjustment logic means dynamically adjusts said variable gain as a function of an instantaneous error signal.

Claim 4

Dependent Claim 42 words

The improvement to a neural network of claim 2 wherein: said gain adjustment logic means makes incremental changes in said variable gain which are proportional to the negative of the derivative of said instantaneous error signal with respect to said variable gain.

Claim 5

Dependent Claim 50 words

The improvement to a neutral network of claim 3 wherein an incremental change ΔT for an 1 th neuron on a layer n can be given as: ##EQU14## where η is the temperature learning rate, which is a pre-established constant and E is the sum of said instantaneous error signals.

Claim 6

Dependent Claim 90 words

A neural network comprising: a plurality of signal paths; a plurality of synapses each comprising means for coupling respective pairs of said signal paths together, each of said synapses having a signal weighting factor which is adjusted during an iterative learning process of said neural network; and a plurality of neurons each comprising means for providing a respective gain in respective ones of said signal paths and for individually adjusting said respective gain independently of the other neurons in the network during an iterative learning process of said neural network.

Claim 7

Dependent Claim 90 words

A neural network comprising: a plurality of signal paths; a plurality of synapses each comprising means for coupling respective pairs of said signal paths together, each of said synapses having a signal weighting factor which is adjusted during an iterative learning process of said neural network; and a plurality of neurons each comprising means for providing a respective gain in respective ones of said signal paths and for individually adjusting said respective gain independently of the other neurons in the network during an iterative learning process of said neural network.

Claim 8

Dependent Claim 55 words

The neural network of claim 5 wherein the learning process of said synapses and the learning process of said neurons are the same learning process whereby the weighting factors of each of said synapses and the gains of each of said neurons are adjusted simultaneously in said same iterative learning process of said neural network.

Claim 9

Dependent Claim 37 words

The neural network of claim 5 wherein the learning process of said synpases and the learning process of said neurons are independent learning processes, wherein changes to said gains and changes to said weights are not simultaneous.

Claim 10

Dependent Claim 76 words

The neural network of claim 5 wherein said neural network comprises first and second layers, each of said layers comprising a respective set of said signal paths and a respective set of said synapses which interconnect the signal paths in said first layer, the signal paths of said first layer having outputs connected to inputs of respective ones of said neurons and the signal paths of said second layer having inputs connected to said respective neurons.

Claim 11

Dependent Claim 40 words

The neural network of claim 8 wherein each neuron provides a summation node between (a) outputs of signal path connected together by respective synapses in said first layer and (b) an input of respective signal path in said second layer.

Claim 12

Dependent Claim 25 words

The neural network of claim 8 wherein others of said neurons have their outputs connected to inputs of respective signal paths in said first layer.

Claim 13

Dependent Claim 21 words

The neural network of claim 5 wherein said activation function is a sigmoid function having a slope which determines said gain.

Claim 14

Dependent Claim 26 words

The neural network of claim 11 wherein said function of said error signals comprise the negative derivative of said error signal with respect to said gain.

Claim 15

Dependent Claim 66 words

The neural network of claim 5 wherein said iterative training process comprises applying a set of known signals to inputs of selected ones of said signal paths and producing error signals corresponding to differences between predetermined values and signals at outputs of certain ones of said signal paths, and wherein the gain of each of said neurons is adjusted as a function of said error signals.

Claim 16

Independent Claim 89 words

A method for training a neural network having a plurality of signal paths, a plurality of synapses each coupling a respective pair of said signal paths together, each of said synapses having a signal weighting factor and a plurality of neurons each providing a respective gain in a respective one of said signal paths, said method comprising performing an iterative learning process while: adjusting the weighting factors of said synapses; and individually adjusting the respective gains of each of said neurons independently of the other neurons in the network.

Claim 17

Independent Claim 89 words

A method for training a neural network having a plurality of signal paths, a plurality of synapses each coupling a respective pair of said signal paths together, each of said synapses having a signal weighting factor and a plurality of neurons each providing a respective gain in a respective one of said signal paths, said method comprising performing an iterative learning process while: adjusting the weighting factors of said synapses; and individually adjusting the respective gains of each of said neurons independently of the other neurons in the network.

Claim 18

Dependent Claim 15 words

The method of claim 14 wherein each iterative learning process comprises a gradient descent method.

Claim 19

Dependent Claim 54 words

The method of claim 14 wherein the learning process of said synapses and the learning process of said neurons are the same learning process whereby the weighting factors of each of said synapses and the gains of each of said neurons are adjusted simultaneously in said same iterative learning process of said neural network.

Claim 20

Dependent Claim 36 words

The method of claim 14 wherein the learning process of said synapses and the learning process of said neurons are independent learning processes, wherein changes to said gains and changes to said weights are not simultaneous.

Claim 21

Dependent Claim 72 words

The method of claim 14 wherein each iterative training process comprises applying a set of known signals to inputs of selected ones of said signal paths and producing error signals corresponding to differences between predetermined values and signals at outputs of certain ones of said signal paths, and wherein said iterative learning process for said neurons comprises adjusting the gain of each of said neurons as a function of said error signals.

Claim 22

Dependent Claim 24 words

The method of claim 18 wherein said function of said error signals comprises a gradient of said error signals with respect to said gain.

Annuity Stage	Window Opens	Payment Deadline	Surcharge Period Until	Estimated Fee	Status
3.5 Years (Year 4)	1997-03-28	1997-09-27	1998-03-29	$2000.00	Window Closed / Lapsed
7.5 Years (Year 8)	2001-03-28	2001-09-27	2002-03-29	$3760.00	Window Closed / Lapsed
11.5 Years (Year 12)	2005-03-28	2005-09-27	2006-03-29	$7700.00	Window Closed / Lapsed

Neural network with dynamically adaptable neurons

Abstract

Inventors

Bibliographic Details

Claims Analysis

References Cited (Backward Citations)

USPTO Annuity & Maintenance Windows

Assigned Categories & Tags

Assign Category Tag