Figure 917

Two examples of a three-layered connectionist network. The bottom layer contains units that represent particular graphemes in particular positions within a word. The middle layer contains units that recognise complete words, and the top layer contains units that represent semantic features of the meaning of the word. Network (a) uses local representations of words in the middle layer, whereas network (b) has a middle layer that uses a more distributed representation. Each unit in the middle layer of network (b) can be activated by the graphemic representation of any one of a whole set of words. The unit then provides input to every semantic feature that occurs in the meaning of any of the words that activate it. Only those word sets containing the word "cat" are shown in network (b). Notice that the only semantic features that receive input from all these word sets are the semantic features of "cat". Adapted from David E.Rumelhart and James L.McClelland, Parallel distributed processing: Explorations in the microstructure of cognition, Volume 1. The MIT Press. Copyright © 1986 by The Massachusetts Institute of Technology, reproduced with permission.

semantic features is associated with this particular configuration of graphemes. This representation is distributed because each word-set unit participates in the representation of many words. Stated another way, different items correspond to alternative patterns of activity in the same set of units (see Figure 9.17b).

Without wishing to be confusing, it should be noted that the local-distributed representation distinction can often be equivocal. For example, Hinton et al. (1986) admit that semantic networks that use spreading activation (see Chapters 1 and 10) are not very distinguishable from other distributed representations, even though they have units that correspond to single concepts. Similarly, it must be admitted that the word-sets in the distributed representation just described are not very feature-like entities but could be categorised as meaningful wholes. However, until more is known about the characteristics of these networks the distinction is heuristically useful.

Distributed representations and propositions/images

The sixty-four million dollar question, which we have been ignoring until now, is "What is the relationship between distributed representations and symbolic representations?" Hinton et al. (1986) argue that these views do not contradict one another, but rather are complementary. By this they mean that the high-level representations, like propositions, may be represented by lower-level distributed representations. However, this complementarity depends on the properties of the lower-level distributed representation being recognised as fundamental aspects of the higher-level representations.

Distributed representations have several properties that make them very attractive relative to symbolic representations. First, distributed representations are content-addressable. This property is an important general characteristic of human memory and refers to the fact that apparently any part of a past occurrence or scene can lead to its later retrieval from memory. For instance, you may remember your holiday on the Cote d'Azur on hearing a certain song, on smelling the aroma of ratatouille, or seeing the sun reflected in a certain way on a woman's hair. It seems that any part of the memory can reinstate all of the original memory. Similarly, in distributed representations, a partial representation of an entity is sufficient to reinstate the whole entity. For example, if we present a slight variant of the original scent of the rose (say, -1, -1, +1, 0 instead of -1, -1, +1, +1) to the network in Figure 9.16a, it will still excite the vision units in roughly the same way. Second, distributed representations allow automatic generalisation. That is, in a manner related to the content-addressibility property, patterns that are similar will produce similar responses.

In conclusion, one can view the symbolic framework as characterising the macro-structure of cognitive representation (i.e., the broad outlines of symbols and their organisation) whereas the distributed representations characterise the micro-structure of cognitive representation (see McClelland et al., 1986; Rumelhart et al., 1986c). However, the full ramifications of the relationship between the two levels requires substantial elaboration.

Business Correspondence

Business Correspondence

24 chapters on preparing to write the letter and finding the proper viewpoint how to open the letter, present the proposition convincingly, make an effective close how to acquire a forceful style and inject originality how to adapt selling appeal to different prospects and get orders by letter proved principles and practical schemes illustrated by extracts from 217 actual letter.

Get My Free Ebook


Post a comment