Figure 916

Two simple pattern associators representing different information. The example assumes that the patterns of activation in the vision units, encoding the sight of (a) a rose or (b) a steak, can be associated with the patterns of activation in olfaction units, encoding the smell of (a) a rose or (b) a steak. The synaptic connections allow the outputs of the vision units to influence the activations of the olfaction units. The synaptic weights shown in the two networks are selected to allow the pattern of activation in the olfaction units without the need for any olfactory input. Adapted from David E.Rumelhart and James L.McClelland, Parallel distributed processing: Explorations in the microstructure of cognition, Volume 1. The MIT Press. Copyright © 1986 by The Massachusetts Institute of Technology, reproduced with permission.

excitation is shown on the olfaction units (from top to bottom -1, -1, +1, +1). The effect of a single vision unit on an olfaction unit is determined by multiplying the activation of the vision unit times the strength of its link to the olfaction unit. So, all the vision units produce the output of the first olfaction unit in the following fashion:

1st Vision unit +1 x-.25 (1st link) = -.25 2nd Vision unit -1 x +.25 (2nd link) = -.25 3rd Vision unit -1 x +.25 (3rd link) = -.25 4th Vision unit +1 x -.25 (4th link) = -.25

In cases where the pattern associator does not learn the association, the links between the vision and olfaction units can be set so that given the vision input of +1, -1, -1, +1 the olfaction output -1, -1, +1, +1 is produced and vice versa (according to the method of combining activation just described). In this way, the pattern associator has represented the association between the sight and scent of the rose in a distributed fashion. We could also represent the sight and smell of another object by a different pattern of activation in the same network. For example, the sight and smell of a steak could be characterised by the vision pattern (-1, +1, -1, +1) and the olfactory pattern (-1, +1, +1, -1); the different pattern of activation for this is shown in Figure 9.16b. Note the differences in the weights of the links in the network.

Distributed versus local representations

Not all connectionist models use distributed representations. They also use representations similar to those used in the symbolic approach, even though the models still use networks of units. Connectionists call the latter local representations. The crucial difference between distributed and local representations is sometimes subtle. A distributed representation is one in which "the units represent small feature-like entities [and where] the pattern as a whole is the meaningful unit of analysis" (Rumelhart, Hinton, & McClelland, 1986b, p. 47). The essential tenet of the distributed scheme is that different items correspond to alternative patterns of activity in the same set of units, whereas a local representation has a one-unit-one-concept representation in which single units represent entire concepts or other large meaningful units.

To be clear about this distinction, consider two networks that deal with the same task domain; one of which uses a local representation and the other a distributed representation. These networks represent the mappings between the visual form of a word (i.e., c-a-t) and its meaning (i.e., small, furry, four-legged; see Figure 9.17a and 9.17b). The network in this case has three layers. A layer for identifying letters of the word (consisting of grapheme units, that indicate the letter and its position in the word), a middle layer, and layer that encodes the semantic units that constitute the meaning of the word (see Chapter 10 for further details on such semantic primitives; here we call them sememe units).

In the localist version of the model, the middle layer of the network has units that represent one word. So, a particular grapheme string activates this word unit and this activates whatever meaning is associated with it. In short, there is a one-unit-one-concept representation in the middle layer (see Figure 9.17a). In the distributed version of the network, the grapheme units feed into word-set units that in turn feed into the semantic units. A word-set unit is activated whenever the pattern of the grapheme units activate an item in that set. A set could be something like all the three-letter words beginning with CA or all the words ending in AT. So, in this distributed representation, activation goes from the grapheme units to many different word-set units and these in turn send activation to the sememe layer, to indicate uniquely which set of

Stop Anxiety Attacks

Stop Anxiety Attacks

Here's How You Could End Anxiety and Panic Attacks For Good Prevent Anxiety in Your Golden Years Without Harmful Prescription Drugs. If You Give Me 15 minutes, I Will Show You a Breakthrough That Will Change The Way You Think About Anxiety and Panic Attacks Forever! If you are still suffering because your doctor can't help you, here's some great news...!

Get My Free Ebook


Post a comment