Higher Cognition

Abstract

Our brains play a role in almost everything we do, but in this book the focus is on activities under the general label of higher cognition, including memory and learning, speech, emotions, intellectual disciplines, planning, consciousness and self awareness.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Our brains play a role in almost everything we do, but in this book the focus is on activities under the general label of higher cognition, including memory and learning, speech, emotions, intellectual disciplines, planning, consciousness and self awareness.

Our brains are general purpose learning systems: they can learn to perform any of a very wide range of different types of behaviour. With very limited exceptions current computer systems do not learn. In contrast with such electronic systems brains are very good at defining and finding patterns and creating and recalling relevant associations between different objects and situations. Sometimes these pattern finding and association creating capabilities are misleading, such as when we see faces in clouds or in groups of rocks on the surface of Mars, or associate numbers like 13 with unpleasant situations.

Unlike our brains, electronic systems can have permanent, unchanging, reliable memories: the picture stored in the computer will look exactly the same in all details several years in the future. Brains on the other hand can extract meanings from memories, and select the more important information to be remembered.

Electronic systems are very good at precisely following tightly defined rules, such as the rules of arithmetic. Brains, on the other hand, find arithmetic much harder, making all sorts of errors. It has been tellingly pointed out that it takes from birth to about 5 years old for a human infant to learn a language, and at the end of that period speech is accurate and sophisticated. It takes another 5 years to learn arithmetic, and at the end of that period there are still lots of errors [4].

One oddity about brains is that they go off line for a significant proportion of each day into a mode called sleep that superficially looks passive. Unlike computers, which can actually be put in a passive mode (or “put to sleep”), sleep for the brain is actually a very active combination of many different processes. During sleep our brains go through several types of processing (stages), as indicated by attaching electrodes to the scalp and measuring electrical activity. This electrical activity occurs at predominant frequencies, with different frequencies in each of three or four different sleep stages, only one of which has frequencies resembling activity in an alert brain. Electronic systems perform perfectly well if rarely turned off or put to sleep, but brain sleep appears to be important for effective brain performance [5].

We do not always realize how much of our awareness is an artifact constructed by our brains. Our eyes have lenses that focus light on the retina, much like the lens in a camera focuses an image on the CCD detector. If we look at the image detected by a camera as we move the camera around we see change, the positions of all the objects move rapidly. If we move our eyes or tilt our head, the image projected on the retina is changing in the same way. But what we are aware of seeing is a stable environment, with all the stationary objects fixed in position and moving objects with the correct relative motion. This stable environment is not the image on the retina, it is constructed by our brain using inputs from the eyes and inputs from muscles in the eyes and body that indicate movement.

At a higher level, it is possible for eyes to be working perfectly, and the brain to be unable to understand what is being “seen”. For example, certain types of brain damage can result in the patient being able to recognize visual features but being unable to put them together to recognize an object. A rose might be described as “about 6 in. in length; a convoluted red form with a linear green attachment” [6]. However, the patient does not perceive that he has any problem when describing things this way. If given a photograph and asked what he sees, he may describe nonexistent scenes in detail, believing that the picture is being described, a process known as confabulating. A river, a guest house and people dining on a terrace may be confabulated from a picture of sand dunes in the Sahara desert [6]. But again, he does not see any fault in his description.

At a yet higher level, there are patients who lose the ability to create new memories, and yet appear superficially quite normal. This type of amnesia, called anterograde amnesia, occurs in patients with damage to one or more of a number of brain structures. One example is Korsakoff’s syndrome [6], where damage to a brain structure called the mammillary bodies (see next chapter) sometimes results from long term excessive alcohol consumption. You could visit someone with Korsakoff’s syndrome, have a normal conversation for a few minutes, but if you leave and come back 5 min later they have no memory of ever having met you. Such a patient has no memory of how they arrived at the place where they are currently located. A patient in hospital meeting a doctor may speak initially as if he was in his grocery store before the amnesia occurred, and address the doctor as a customer. When this story breaks down the patient will immediately jump to seeing the doctor as a pre-amnesia friend, then a different pre-amnesia friend, then a doctor he has not met before, then back to a customer in his store again. The patient remembers nothing for more than a few seconds, each confabulation is an instinctive creation a current “story” in which to locate himself, and he has no awareness that his story is changing abruptly from moment to moment. This frantic jumping from story to story reveals the importance of such “stories” to our sense of identity, and the degree to which this sense of identity is an artifact constructed by our brains from our memories. In some ways these amnesic patients seem to an outside observer to have lost their identity or self.

Our personality is a large part of who we are, but patients with brain damage often show personality changes. Such changes include uncontrolled anger, inability to take responsibility, socially inappropriate behaviour, and impaired self awareness. An investigation of subjects with a history of recurrent attacks of uncontrolled rage with little or no provocation and no known reason (i.e. with psychotic, psychopathic, profit motivated, drug related, mental retarded cases excluded) found that in almost all cases there was evidence of brain damage [7].

The most famous example of personality change following brain damage is Phineas Gage, who in the middle of the nineteenth century sustained major damage from a spike passing through the front part of his brain. As the doctor who treated him over a long period of time reported the situation:

Previous to his injury, although untrained in the schools, he possessed a well-balanced mind, and was looked upon by those who knew him as a shrewd, smart businessman, very energetic and persistent in executing all his plans of operation. In this regard his mind was radically changed, so decidedly that his friends and acquaintances said he was ‘no longer Gage’. [8]

We will now consider in more detail some different aspects of the cognitive capabilities exhibited by our brains. We will start with two fundamental aspects of all cognitive activities. The first is attention, or the ability (and necessity) of focusing on a limited range of information selected from what is currently available. The second is memory, or the ability to recall information derived from past experiences and/or to use such information to influence current behaviour.

2.1 Attention

The environment in which we operate is very complex and responses to many different objects or combinations of objects in that environment could be appropriate. Generation of a response requires processing of the information derived from an object, and it is therefore necessary to select objects within the environment in order of priority for such processing. Selection of and processing of information from objects is known as attention.

In the case of visual attention, there are three stages to the attention process. For example, presented with a visual scene, the brain drives a guided scanning process in which the eye and head are moved to centre the sensitive central part of the retina (the fovea) on different objects. This scanning process is made up of rapid jumps from one object to another (called saccades), sometimes mini-saccades to correctly centre on a target object, and pauses of one or two hundred milliseconds at each object. Subjects cannot pay attention to one object and make a saccade to a different object [9]. Once the gaze has been focused on an object, simple behaviours can be generated with respect to that object, for example naming or avoiding the object. A further attention step is called conscious awareness [10]. This step includes a much richer subjective awareness, during which it becomes possible, for example, to talk about the object. Not all object attention steps are followed by conscious awareness.

2.2 Memory

In our daily lives, we remember some things for only a few seconds, other things for hours and days, yet other things for all our lives. Some of our memories are purely factual, like the name of the capital city of the country in which we are living. Other memories are for what happened at a particular time and place, and can include recall of the emotions felt at the time, such as an embarrassing, annoying, or highly successful personal event. Yet other memories are how to perform various skills like riding a bicycle, driving a car, or playing a piano. Another memory capability is holding a telephone number in our mind just long enough to dial it. Yet another capability is remembering to do something at a particular time and place. The ability to do this is sometimes called prospective memory, and failure to perform this task correctly is one of the most common reasons for complaining about our memory.

In order to study memory scientifically it is necessary to select relatively simple tasks that can be measured consistently across different subjects. An important part of such studies is imaging the brain during the tasks, and determining which physiological structures are active during different memory activities. Another source of information is observing how patients with damage to different brain regions perform on the various memory tasks. Yet another source is study of the brains of animals on tasks which have a degree of similarity to human memory tasks.

Such laboratory studies have led to the conclusion that there are five independent systems in the brain supporting different types of memory. These five systems are semantic memory, episodic memory, working memory, procedural memory and priming [11]. Each of these systems involves a set of memory tasks that are different from the tasks performed by the other systems. The reasons for believing the systems are separate is that if the circumstances in which memory tasks are performed is changed, some changes affect one type of memory task and not another type.

One particularly important set of examples of changed circumstances are observations that local damage to one part of the brain affect memory type A with no effect on memory type B, while damage to a different part of the brain has the opposite result. These type of combined observations are called double dissociations. We will be describing examples of double dissociations later in the chapter.

However, in real life an actual cognitive task will typically be a process involving many of the different memory systems. Learning to drive a car involves learning motor skills, but will involve a fair amount of memory for facts and events, especially in the early stages. Working memory will also be required. Remembering the sequence in which memory and other cognitive processes must be performed can itself be viewed as another type of memory.

In the following sections we will consider the different memory systems separately, but much later in the book we will discuss real life cognitive tasks and how they are achieved by sequences of different memory processes.

2.2.1 Semantic Memory

Semantic memory is the memory for facts, with no memory for the circumstances in which the facts were learned. We generally know that there are planets in the solar system called Mercury, Venus, Earth, Mars etc. but do not remember when or where we learned that there are such planets and what they are called. The meanings of words are important examples of semantic memory, and we generally have no memory of when we learned the words. It is possible to make mistakes, where we retrieve a fact or word meaning which is incorrect but which we believe to be true. Such mistakes appear to be very rare in normal people. It is not always possible to immediately recall a fact or word which we have learned, and sometimes we have a “tip of the tongue” experience in which we feel as though the memory is in some sense very close to being retrieved but not quite there.

In the laboratory, a frequently used test of semantic memory is category verification. A subject is given the name or picture of an object, and the name of a category, and asked to respond yes/no that the object is/is not a member of the category. An interesting observation is that the time to respond for an object that is clearly a member of the category (e.g. hawk – bird) is about the same as for a clear non-member (e.g. walnut – bird), but the time for an atypical member of a category (e.g. ostrich – bird) is somewhat longer. The difference is slight but significant. Over lots of trials, it takes the average subject about 1.3 s to respond for a typical category member or a non-member, and about 1.5 s for an atypical category member [12]. Other tests measure the speed with which semantic memories can be accessed. For example, subjects may be asked to think of five letter words beginning with the letter M, and the number that they can retrieve in 1 min is measured. This measure can vary considerably between different subjects, from under 8 to over 12 in one study [13], although almost all the retrieved words are known to all subjects. Yet other tests investigate memory for well known facts in the subjects culture, such as the names of famous people. Many of these tests are used to compare subjects with memory deficits with normal people, or older subjects with younger [14].

The “tip of the tongue” experience has also been investigated, and it is found that if we have such an experience when, for example, trying to think of the correct word corresponding with a definition, we can often identify the first letter, the number of syllables or a rhyming word even while the actual word is still inaccessible [15].

2.2.2 Episodic Memory

Episodic memory is the memory for events, often events in which we were personally involved, in which case it is called autobiographic memory. Episodic memory can reconstruct the sequence in which things happened and the emotions felt at the time. Recalling an episodic memory is almost like reliving the event, except that the raw sensory inputs are not there, in other words the recall is not an auditory and visual hallucination.

However, our memories of an event are selective. For example, if we are feeling strong emotion (such as fear) during an experience, our memory for the general situation during the experience (the gist of the event) is enhanced, although memory for more sensory aspects of the event is unaffected [16]. If an event is novel, we are more likely to be able to recall the event later, especially if there was strong emotion at the time. An extreme example of very enhanced memory is the phenomenon of flashbulb memory. When a person recalls a particularly dramatic event (such as the assassination of President Kennedy, the moon landing, or the 9/11 attack on the World Trade towers), it is often possible to also recall many details of where the person was and what they were doing and feeling when they first heard the news [17].

At the other extreme, if our attention is focused on one aspect of a situation, we may have no memory for another aspect. A striking example of this phenomenon is the “gorilla movie” experiment. In this experiment, subjects are shown a movie of a group of people passing a ball between them. Some people have white shirts, some black shirts, and the viewer is asked to concentrate on counting how many times the ball is passed between black shirted participants. When people see the movie for the first time under these conditions, about half fail to notice or remember the person in the gorilla suit who walks through the players, pausing at one point to face the camera and thump its chest [18].

Even if we can recall a memory, it is not always a reliable reconstruction of the original experience. For example, if someone is shown a picture of an often encountered scene (such as a beach or a classroom), and later asked to recall the picture and indicate if various objects were present, objects characteristic of such scenes but not actually present in the picture are often “remembered” [19]. Talking about an event can change our memory of the event. If someone witnesses a crime scene, and afterwards is asked to verbally describe the criminal, they are less likely to accurately select the criminal from a photo lineup than someone who was not asked for a verbal description [20].

Although remembering an actual past event and imagining some future event may seem to be very different activities, the imagining process uses many of the same parts of the brain as remembering an actual past event [21]. It is even possible to create false memories for events which never happened. Subjects were asked to read paragraphs describing various events from their past experience, some of which actually happened, some of which definitely did not happen. These descriptions were created in consultation with family members. A common false event was becoming lost in a shopping centre as a child. After reading each paragraph, the subjects were asked what they could remember of the events. Several times in the next few weeks they were shown the paragraphs and asked again what they could remember. At the end of this process, a number of subjects were convinced they could remember the non-existent event [22]. It is believed that false memories of childhood abuse may have sometimes been inadvertently introduced by a process of this type [23].

Another aspect of episodic memory is the ability to fit events into a timeframe. We can judge whether one event happened before or after another event. We have a subjective sense of how long an event lasted even if we do not have a memory of the actual start and finish times. However, this subjective sense is not precise: for example, most of us feel that time seems to have passed more slowly when we were younger. This subjective sense also tells us how long a current event has been under way, but again the feeling is not fully reliable relative to the clock: time seems to pass more slowly during a boring event than during a novel, stimulating event.

In the laboratory, both long term and short term episodic memory is measured. In one type of test of relatively short term recall, subjects read a story, and later are asked to recall what they can of that story. In another type of test a subject is given pairs of objects (which may be pairs of words or pairs of pictures), and later shown one object and asked to recall the other. Longer term testing includes being given a group of words, and asked what autobiographical memory is generated. As for semantic memory, these tests are often used to compare subjects with and without memory problems.

2.2.3 Declarative Memory and the Distinction Between Semantic and Episodic Types

The content of semantic and episodic memories can be described verbally, and both these memory types are therefore known as declarative memories. An interesting aspect of declarative memory is that it is much easier to recognize that we have seen something before than it is to recall a mental image. Our higher ability to recognize familiarity implies that we are constantly recording large volumes of information, much of which is not available for easy later recall. In one study, subjects were shown several thousand photographs, each for a few seconds. Several days later they were shown pairs of photographs, one new and one previously seen. They could pick out the familiar photograph more than 90 % of the time [24]. It is much easier to recognize that we have visited a place when we return than it is to imagine the place when we are not there. However, an indication of limits to recognition of familiarity is the déjà vu experience, in which a new experience feels as if it is familiar, along with an awareness that it is in fact novel.

In healthy people, there are declines in declarative memory capabilities with age, but these declines appear to be limited to the acquisition and retrieval of new information, not in the retention of older memories [25]. With a disease like Alzheimer’s, all declarative memories can be degraded [14].

Brain damage provides some important evidence that semantic and episodic memories are supported by different brain systems. One example is the extensively studied patient Henry Molaison, known in the many papers written about him as HM until his death late in 2008. In 1953 at age 27, he had experimental surgery to treat severe epilepsy [26]. This surgery removed significant sections of his hippocampal system (see next chapter). Like the patients mentioned earlier in this chapter, he lost all ability to create new declarative memories, but retained normal intelligence and conversation skills. The separation between semantic and episodic memory is indicated by the observation that although he also lost his autobiographical memories for a period of 11 years prior to his surgery [27], he retained semantic memories for the same period, such as words which first came into use in those years [28]. HM also exhibited normal working [26], procedural [29] and priming memory [30]. In contrast with HM, other patients with damage to different brain areas can show a general deterioration in semantic memory but no effect on episodic memory [31].

2.2.4 Working Memory

Working memory refers to the number of different things we can be actively thinking about at the same time. When we look up an unknown telephone number in order to dial it, the task is more difficult if the number is longer. However, if part of the number is known from another context (such as the year we were born), the task is easier. If we are interrupted while dialling, the number may disappear from our mind and we have to look it up again. Imagine the visual appearance of cat. Now try to retain the cat image, and also imagine the visual appearance of a pineapple. Now try to retain the cat and pineapple image and also imagine a book. Then try to retain the three images and also imagine a chair. The task becomes more and more difficult. However, it is easier to imagine a scene in which a cat is sitting on a book located on a chair and scratching a pineapple than to imagine four unrelated objects.

Forming objects into a group in which they are associated in some way (four digits making up a familiar year; four visual objects relating to each other) is known as chunking, and in general chunking increases the number of objects available for immediate mental processing. The familiar process of reading depends on a hierarchy of chunking, we put words into phrases, phrases into sentences, sentences into paragraphs.

Laboratory tests of working memory attempt to quantify these observations. Subjects are given lists of unrelated words and a short while later are asked to recall them. Early work suggested that the number was the “magic number” seven [32]. However, this earlier work did not take chunking into account, and later experiments suggest that the actual number is in the range of three to four. For example, rather than measuring the average number of words that can be recalled, experiments can measure the largest list size for which the subject never makes errors [33]. If the subject is given a different verbal task during the delay between hearing the words and attempting to recall them, the number of words recalled is reduced.

There is evidence that we have several separate working memory stores, including a verbal store, a visual store, and a spatial store. The verbal store is tested by word lists as described in the previous paragraph. The visual store is tested by showing subjects a visual array of rectangles, some filled and some not (see Fig. 2.1a) and after a short delay asking the subject to indicate the filled blocks on an empty array. The spatial store is tested by the experimenter tapping on a set of blocks in some random order, and after a short delay asking the subject to reproduce the tapping order (see Fig. 2.1b). The typical subject can correctly fill in about nine rectangles, and reproduce five or six taps.

Fig. 2.1

Measuring visual and spatial working memories. (a) In visual working memory experiments, a subject views an array of rectangles for a few seconds. Arrays contain 4–30 rectangles in which about half are shaded. After a short delay, the subject is given a copy of the array with all rectangles unfilled, and asked to fill in the shading as on the previous array. Typical subjects can correctly recall up to about nine shaded rectangles. (b) In spatial working memory, a subject watches an investigator tap his finger on a sequence of two to eight wooden blocks in a random arrangement of nine such blocks. After a short delay, the subject is asked to reproduce the tapping sequence. Typical subjects can correctly reproduce sequences of up to five or six taps

The reason for believing there are separate stores is that if the delay between verbal presentation and recall is filled with a visual or spatial task, the verbal memory is less affected than if the distracting task is verbal. If the delay between visual presentation and recall is filled with a verbal or spatial task, the visual memory is less affected than if the distracting task is visual, and similarly for spatial memory [34].

Evidence that working memory is a different brain system from semantic and episodic memory comes from observations that one type of brain damage or deterioration strongly affects declarative memory with limited effect on working memory, and another type affects working memory with little effect on declarative memory. Important examples of declarative memory deficits with no effect on working memory are HM [26] and patients with Korsakoff’s syndrome as discussed earlier. Patients with Huntington’s disease show the opposite symptoms of relatively intact declarative memory combined with working memory deficits [35]. Patients with the frontal variant of frontotemporal dementia show working memory deficits but unaffected semantic memory, while patients with the temporal variant show severe semantic memory deficits but unaffected working memory [36].

2.2.5 Procedural Memory

Procedural memory refers to learning and performing skilled behaviours. Skills learned early in life include walking, running, descending stairs, and drinking from a cup. Some very complex skills are the generation of sequences of muscle movements to produce speech and writing or to play a musical instrument. A relatively simple skill often used in experiments on procedural memory is tracing the outline of a shape when the shape and trace are only seen in a mirror.

It is generally very difficult to verbally describe the practice of a skill: try to describe how you ride a bicycle. Once learned, a skill is fairly stable in time, although complex skills may deteriorate if not practiced. Only the current state of a skill is accessible, you cannot decide to go back to your skill level, say, 3 years ago.

Declarative memory is often very important in learning a skill. Early in the learning process you may use episodic memory to recall the correct sequence of actions. However, declarative memory access becomes less important when the skill has reached a high level. In fact, requiring an expert to verbally describe their skill can result in their performance deteriorating [37].

Furthermore, in some cases a skill can be learned without any apparent declarative memory, or even awareness that the skill is being learned. For example, subjects can be given a task that requires rapidly identifying the location of an object in a complex picture presented on a computer screen [38]. A sequence of pictures is presented with the object in different locations, but with a complex rule defining how the object shifts from picture to picture. Subjects learn to become faster in finding the object, but have no awareness of the rule or of having learned it.

Different deficits observed following different types of brain damage or deterioration supports the idea that declarative and procedural memory are supported by different brain systems. In the case of HM, declarative memory was strongly affected but past skills remained intact, and there was a retained capability to learn new simple skills such as mirror tracing that did not require initial assistance from declarative learning. In such simple tasks, skill would improve over a number of sessions, but with no memory of attempting the task earlier. Furthermore, patients with Parkinson’s disease show degradation of procedural skills such as walking and loss of some abilities to learn new skills [39] but relatively little loss of semantic or episodic memories.

2.2.6 Priming Memory

If a message “drink Lipton Ice” is flashed on a screen during performance of another task, and the message is visible so briefly that the subjects have no awareness of having seen it, the subjects nevertheless have a later preference for the Lipton Ice drink, provided that they are already thirsty [40]. This phenomenon is much more limited than the exaggerated claims for effective subliminal advertising, but does demonstrate that there can be memory (i.e. an effect on later behaviour) when there is no conscious awareness.

In another laboratory test for priming, subjects are shown a series of line drawings of different objects and asked to identify the object [41]. Each drawing is visible for less than 50 ms, and is followed by a meaningless pattern with strong contrast to prevent retinal afterimages. The first time a drawing is seen under these conditions, the object is identified about 13 % of the time. However, if the same drawing is seen again under the same conditions a little later, the object is identified about 35 % of the time. In other words, somehow the exposure to the drawing, even though it could not be identified at the time, increases the chance of its being identified later.

Priming memory appears to be affected by damage to the visual areas of the cortex [42], but not by damage or deterioration that affects any of the other types of memory [43].

2.2.7 Prospective Memory

Prospective memory refers to our ability to remember to do something, when the intention to do it was formed earlier. Typical examples are remembering to go to an appointment with the dentist, remembering to pay a bill on time, remembering to get milk at the supermarket on the way home and so on. Prospective memory failures make up the bulk of everyday forgetting and are very common. Such failures become much more frequent with, for example, patients with Alzheimer’s disease. Prospective memory failures are generally much more disruptive to daily life than failures to remember words or facts or past events [44]. Constant failures to remember appointments with friends or professional colleagues can be very disruptive to social and working life.

Prospective memory tasks are separated into time based or event based types. A time based task requires a behaviour at a specific time (e.g. put the meat in the oven at 5 p.m.) while an event based task requires a behaviour in response to a cue (e.g. mention something to a friend when you see them). Tasks are also classified as regular or irregular. A regular task is an habitual activity (e.g. take medicine every day with breakfast). At irregular task is occasional (e.g. water the garden tomorrow when you get home). A distinction can also be made between short term prospective memory (e.g. I forget what I was going to say) and longer term.

Prospective memory is typically regarded as a sequence of activities which can draw as required on other memory systems. The first step is a planning to develop an intention, the second is maintaining the intention while being distracted by other tasks, the third is returning to the intention at the appropriate point, the fourth is carrying out the original intention. Some parts of prospective memory could be working or episodic memory tasks. Study of brain activation during prospective memory tasks indicate overlaps with areas active during episodic and working memory [45].

2.2.8 Information Models for Different Types of Memory

The differences between memory types suggest different information models. Thus as illustrated in Table 2.1 semantic memory is learned by the repeated presence of two elements at the same time, and accessed by the occurrence of one element. This suggests an information model in which information is accessed on the basis of frequent past simultaneous activity. Episodic memory is sensitive to the degree of novelty in an experience, and novelty will tend to require more information recording. This suggests access on the basis of simultaneous past information recording. The information model for working memory is simply the number of objects which can be independently maintained in the same resources at the same time. The information model for priming memory is activation on the basis of recent simultaneous activity. The procedural memory model is reward modulated access to behaviours from sensory conditions.

Table 2.1 Examples and information models for different types of memory

As discussed in later chapters, activation on the basis of recent activity, frequent past simultaneous activity and simultaneous past information recording represent the three primary ways in which information recorded in the past about the environment can be accessed, and procedural memory represent the way in which current and past information can be linked to appropriate behaviours. Working memory represents a limit on the simultaneous information accessing processes that can occur in the same resources.

2.3 Speech

The ability to understand speech and to generate appropriate speech are fundamental to human cognition. Reading and writing are important extensions of these capabilities. Speech allows the thought processes of one person to be reproduced in the mind of another person, at least approximately. The second person can in turn modify the thought processes, and return them to the first person again. Speech and writing make it possible to experience situations that we have never directly experienced, including situations that do not exist in the real world. Frequently, even our own internal thought processes are experienced as sequences of words.

A newborn baby has the capability to learn any human language. The baby goes through a series of stages. The first stage, usually starting around 6 months, is babbling [46]. Use of individual words, usually nouns, follows by around 12 months. Meaningful use of word pairs is generally achieved by 2 years. Speech is completely intelligible by about 5 years. However, if a baby is not exposed to a particular language in the first 5 years, accent, grammar and style will not be fully mastered by later learning. There will be permanent major deficits in language use in general if no language is acquired by age 12 years [47]. Interestingly, many species of songbirds show a similar learning pattern, there are stages of babbling and subsong, with auditory feedback needed [48].

An infant hears a large amount of speech. However, in terms of the complexity of the language system which is learned, the total heard is in fact remarkably small. This suggests that the human brain must have some source of knowledge about the structure of language in general, which can be used to guide the learning process [49].

Understanding speech begins with a sequence of sounds. Within a language, there are sound units called phonemes. Each phoneme is a group of sounds, such that changing one sound to another within the group will make no difference to the meaning. For example, in kit and skill as usually spoken, the k sound is slightly different: in kit the sound is aspirated, or ends with a tiny puff of air. In English, one sound could be exchanged for the other without changing the meaning, and the two sounds are therefore both in the same phoneme. In, for example, Icelandic, changing from one of these sounds to the other would change the meaning of the word containing them, and they are therefore parts of different phonemes. There are about 40 phonemes in English. A 6 months old baby can detect the difference between any two sounds that can be in different phonemes in any human language, but by 12 months can only discriminate between phonemes in the languages the baby has been hearing regularly [50].

Out of these sound units our brains extract words, phrases, sentences and meanings on larger scales. Individual words can communicate meaning. To generate meaning on a larger scale, multiple words must be presented within a structure known as syntax. In English this structure defines word order, longer pauses at certain points, relative stress on different words, the way the tone changes within a word, and even the detailed form of the word. If the structure is violated, the meaning is changed, difficult to grasp, or absent. For examples see Table 2.2.

Table 2.2 Some ways in which different types of structure determine the meaning of a group of words

The meaning of an individual word can change depending on the context in which it is used. One word may have multiple unrelated meanings, such as “light” in the three sentences: I will light the fire; This backpack is fairly light; and The moon provides enough light. A word may have different but related meanings, such as “window” in the two sentences: I cleaned the window (an object) and The bat flew in through the window (an opening). A word may have different related and unrelated meanings, such as “bank” in the three sentences: The bank offered a no fee credit card (financial institution); The bank was destroyed in the fire (building); and There was a rabbit on the bank of the river.

Understanding speech is so natural that it is hard for us to listen to the physical sounds of our own language. For example, spoken English contains a high proportion of “ss” sounds, but it is hard for a native speaker to observe that by listening, although it is obvious from inspection of written English.

Generating speech is equally natural, the required sequences of muscle movements are produced without any attention, and we are often not conscious of the words we are going to say until we say them, only (sometimes) of a general internal image that drives speech production. Nevertheless we are able to produce accurate, grammatical speech with very few errors.

The kind of speech deficits that result from damage to different brain areas can provide significant information about how the processing of speech is organized in the brain. However, it is important not to exaggerate the conclusions that can be drawn from such associations between damage and deficit. For example, if there is a type of local damage that affects the production of speech but not the understanding of speech, this does not necessarily mean that production and comprehension are supported by completely different brain regions. However, it does imply that there is a brain area specialized in an information process that is needed by speech production but not comprehension.

There are four major types of speech deficit (or aphasia) observed in patients with brain damage to certain brain regions. One is a deficit in the production of speech. A second is in the ability to understand speech and produce meaningful utterances. A third is if the ability to repeat sentences spoken by someone else. The fourth is difficult or unclear articulation of speech that is linguistically normal [51].

In a speech production deficit (one example is Broca’s aphasia [52]), spontaneous speech is slow, laboured and choppy. On average, only two words are generated at a time, sometimes only one word, even though an intended meaning is present. Utterances include nouns, verbs and adjectives, but function words like articles, conjunctions and prepositions are not generated. An example might be an intent to take pride in a son who is attending university coming out as “son … university … smart … boy … good … studies … good”.

In a speech comprehension deficit (one example is Wernicke’s aphasia [52]), incomprehension leaves the patient unable to generate meaningful utterances, although the articulation and syntax remains normal. For example, a verbal discourse by a patient might be made up of sentences like “I told my cow with the radio but did not understand the fence”. All spoken sentences are grammatically correct but devoid of meaning to the hearer.

A deficit in the ability to accurately repeat sentences is also present in both Broca’s and Wernicke’s aphasias, but in Transcortical Sensory aphasia [53] there are Wernicke-like production and comprehension deficits but no repetition deficit.

There is a specific brain region known as Broca’s area, which often results in Broca’s aphasia if it is damaged. Brain imaging results show that Broca’s area is active during speech production [54]. However, a limited degree of speech comprehension deficit can sometimes be present with damage to this area, and brain imaging indicates that it is also active during language comprehension [55]. This region is also active during prospective memory [56]. Furthermore, a brain region very similar to Broca’s area is also present in the brains of the great apes [57]. The implication is that although this brain area performs an information processing function that is very important for human speech generation, its function is also important for other cognitive capabilities including in the great apes with no (or extremely limited) language capabilities.

2.4 Arithmetic

As mentioned earlier, many people have difficulty with arithmetic, even though the need for some skill in this area is ubiquitous in society. Errors are often made in the mental performance of the basic manipulations (addition, subtraction, multiplication and division). These errors are common in children learning arithmetic, but still occur surprisingly frequently in adults. Given the effort put into the formal teaching of arithmetic, the high level of errors suggests that there is something about the information processes needed for arithmetic that do not match well with the capabilities of our brains. As Marr commented “I have no doubt that when we do mental arithmetic we are doing something well, but it is not arithmetic” [4].

Many different strategies may be used to perform the same manipulation including counting and memorisation of a fact portfolio. For example, to find the result of 4 + 3, a child may start at 4 and count off the next three numbers in the standard number sequence. Alternatively, the answer may be known from memory. For more complex calculations we make use of various memorized rules.

Another aspect of number knowledge is the ability to look at a group of objects, and immediately know how many objects are in the group without counting. Human beings are able to do this for groups of up to five or six objects, but not for larger groups.

More fundamental is the capability called number sense, meaning an intuitive understanding of numbers, their relative sizes, and how they are affected by different arithmetic processes. Number sense is revealed in the ability to rapidly assess the relative sizes of 3/7 and ½, to immediately see that 100/0.97 is larger than 100, or if asked to multiply 7 × 99 to immediately recognize that the answer is 700 – 7, a much easier problem. Number sense includes the ability to detect when the answer to a calculation does not “make sense”, in other words when the result is wildly different from an intuitive estimate. Mistakes are more likely to be noticed and corrected if there is a good number sense. Interestingly, brain damage can sometimes make the performance of even simple arithmetic problems slower and less accurate, but leave number sense (such as judging which is the larger of two numbers) intact. Such a patient might be able to estimate answers but unable to calculate the exact answer [58].

2.5 Face Recognition

Most of us are able to remember and recognize the faces of a thousand or more different individuals, including family, friends, coworkers, and casual acquaintances. Furthermore, it is very rare to see a stranger among the many thousands we encounter and falsely identify their face as someone known to us. Compared with most of the other visual objects we need to be able to discriminate between, faces are very similar to each other. If we are brought up exposed only to black faces, all white faces look alike, and vice versa [59]. The detailed discrimination between faces is thus something we learn. In fact, it has been shown that if 6 months old babies are exposed to different monkey faces they learn to discriminate between monkey individuals [60].

There is a particular area of the brain called the fusiform face area (FFA) which is always active during face recognition tasks [61]. Damage to this area results in prosopagnosia, or the loss of ability to recognize faces, but the ability to recognize other types of object is unaffected [61]. However, the FFA seems to be also associated with different types of strong visual expertise. For the average person, the area is not active when recognizing birds, or dogs, or cars. Some people have a strong expertise in one of those areas, able to recognize many different species of birds, to distinguish between many different types of dog, or to identify the make and year of many different cars. In such people, the FFA is also active when performing tasks in their area of expertise [62].

Human beings also have a capability to recognize the emotion of a person from their facial expression. There is evidence that facial expressions indicating anger, disgust, happiness, sadness (or distress), fear, and surprise are recognized across widely different human cultures, from Japanese and American to preliterate Papua New Guinea [63]. Brain damage to specific structures can affect recognition of one or more types of emotion from facial expression [64]. An ability to recognize the emotional implications of facial expression independent of culture would imply that such expressions are to some degree programmed genetically, although it is also clear that cultural experience can modify any such genetic programming, in particular the interpretation of the degree to which an emotion is indicated by a given expression [65].

2.6 Emotions

Human beings can experience a range of emotions that are both reactions to circumstances and general influences on behaviour. An emotion can also label a personality trait (“He is an angry person”). There appear to be large numbers of emotions labelled by different English words, perhaps well over 100, and it has been argued that they can be organized to some degree on the basis of similarity into major categories, subcategories and sub-sub-categories [66]. If has also been argued that more complex emotions are combinations of simpler emotions [67]. For example, remorse could be a combination of sadness and disgust, and contempt a combination of anger and disgust.

Table 2.3 provides a list of the names of some widely accepted emotions. For the basic emotions, there is evidence that the emotion is defined under significant genetic control. Thus as discussed in the previous paragraph, the physical human response to the presence of anger, disgust, happiness, sadness, fear, or surprise is partially independent of human culture. Pride also seems to be expressed in culture independent ways [68]. For other emotions like contempt, embarrassment and shame there is weaker evidence for cross cultural commonality [69].

Table 2.3 Some different emotions

2.7 Hobbies and Intellectual Disciplines

Human beings are able to develop sophisticated skills in a wide range of intellectual areas from astronomy through history and physics to zoology. Sometimes proficiency involves a high degree of specific motor skills, such as sports of different kinds. Sometimes the skill is in complex cognition within a defined rule framework, such as the games of chess or go. An intellectual discipline may require the ability to create associations within a very large body of factual knowledge, such as history or zoology. Alternatively, a discipline may require the ability to define a small set relatively simple starting points and develop an extensive range of logical consequences, such as mathematics and physics.

Anecdotally, individuals appear to be better at some types of skill than others. However, some types of skill often go together in the same individual, such as mathematics, physics, music and chess. For some skill types, the most creative contributions tend to be made earlier in adulthood, while for others creativity can sometimes steadily increase with age. Physical scientists do their best work at age 40 ± 10 [70], but historians generally do their most significant work when late in adulthood [71]. Poets and novelists often peak in their 30s with some differences depending on the type of writing [72].

2.8 Tool Making

Human beings can create tools to assist in achieving behavioural and cognitive objectives. A simple tool could be a stone that happens to be available locally, used as a hammer or as a weapon thrown at a target. At the other extreme, an airplane is a tool for travelling long distances, and a spacecraft is a tool for acquiring information about distant objects which cannot even be perceived by unaided human senses. Human tools can thus be extremely complex. The task aided by the tool may occur far away from where the tool is made, it is not necessary for the tool maker to be where the tool is needed. Tools can even be made for tasks in locations that have never been directly experienced such as the surface of another planet. Imagining the task and location is adequate to guide tool making. The ability to make a tool can be passed to another individual, and the tool improved by that individual.

It was once thought that toolmaking was unique to human beings, but it has been found that some forms of this capability are present in a number of animal species, from crows to chimpanzees.

Some crows are able to modify objects available on the spot to use as tools. One example is bending a straight piece of wire to extract food that is out of direct reach in a deep container [73]. Another example is the snipping of leaves followed by tearing to make tools for probing crevices containing insects. Study of the variation in these probing tools suggest that crows are able to pass the ability to make the tools to other crows and build up a technology across generations [74].

Chimpanzee tool making also seems to be limited to close to the time when the tool is used. However, observation of chimpanzees making stick tools to probe underground termite nests showed that although the tools were made (stripped of leaves and shortened) at the location where they were used, raw branches were collected at some distance from that location [75]. Chimpanzees also seem able to learn from other chimpanzees, even from videos of other chimpanzees making the tool [76].

2.9 Consciousness and Self Awareness

The topic of consciousness is controversial, and even the definition of consciousness is far from being universally agreed. At a very basic level, distinctions can be observed between normal wakefulness, being stunned (by a physical blow), being asleep, and being in a coma with the extreme of a persistent vegetative state. These different states of consciousness can also be observed in animals.

The more interesting question for higher cognition is whether there is something qualitatively different about the internal experience of human beings, also labelled “consciousness” or perhaps “human consciousness”. In the nineteenth century, the psychologist William James described his concept of “stream of consciousness” as the unique and characteristic property of human consciousness [77]. This stream of consciousness concept refers to the experience of streams of mental images that have no apparent connection to current sensory inputs, and may be of situations never actually experienced or even impossible in reality. However, these mental images have some similarities to direct sensory experiences, although generally without the sensory detail that would make them hallucinations. The experience is marked by points at which the mental images are sharp, separated by periods of much more vagueness [77].

More recently, emphasis has been placed on the observations that some experiences can be reported verbally, while others can influence behaviour but are not accessible for verbal description. The ability to report verbally is called access consciousness [78]. One characteristic example of unconscious influence is the phenomenon of dichotic listening [79]. In dichotic listening experiments, a subject wearing headphones hears one meaningful text being read in one ear and a different meaningful text in the other ear, having been told to pay attention to the right ear. Such subjects can afterwards describe the material read to the right ear, but not the other ear. The material read to the left ear is ignored and is not remembered later. However, if part way through reading, the texts are switched between ears, the subject follows the continuation of the text that started in the left ear, without realizing that a switch has occurred. The content arriving at the unattended ear is thus available to influence behaviour despite being consciously unavailable. This laboratory observation is similar to the cocktail party phenomenon: one conversation can be followed in a room full of people, and generally a memory for the one conversation and no others is retained. Nevertheless, words indicating strong personal relevance in an unattended conversation can be detected and attention shifted at least briefly to that conversation.

Another possible aspect of human consciousness has been labelled phenomenal consciousness [78]. The concept of phenomenal consciousness has its origins in the observations that although our experience of something (such as the colour red) may be very vivid, it is hard to describe the full vividness of that subjective experience in words. It is therefore difficult to assess whether one person’s experience of the colour red is similar to or completely different from the experience of someone else, or even if individual experiences of red are the same on different occasions. The term ‘qualia’ is used in philosophy to refer to these vivid, subjective experiences [80], and it has even been argued that scientific understanding of the subjective experience of qualia is not possible.

A final important aspect of human consciousness is self awareness. The stream of consciousness includes images of self, behaving and experiencing. These self images can be memories of past personal sensory experiences, actions and emotions. They can also be imaginings of future experiences by self, including plans for possible experiences and impossible fantasies. There are “I” and “me” type self images [81]. If you are asked to imagine the last time you swam in a lake, the image that comes to mind could be a view of yourself, standing at the edge of a lake, and perhaps diving in. This self viewed from the outside is a perspective never seen in real life, and has been labelled the “I” self image. Alternatively, you could imagine the water splashing on your face, where the perspective is self looking out. This self view is labelled the “me” perspective. Both perspectives are useful, for example in developing plans for the future.

Are these conscious capabilities unique to human beings, or are they shared by other animals to some degree? It can be argued that human consciousness and self awareness are dependent on speech capabilities, and on the basis of evidence from human literature developed only within historical times [81].

2.10 Individual Differences and Complex Society

There is relatively little difference between different individual flies. Differences between individual lizards are difficult to find. There are some detectable differences between individual birds, and even more differences between individual chimpanzees. The differences between individual human beings are very large compared with any other species, and a significant proportion of these differences are the result of different learning histories.

These differences are valuable to a complex society, and make it possible to undertake tasks requiring the participation of large numbers of individuals, each making a unique individual contribution. One major value of large cities is that they bring into close proximity the vast range of different human skills needed, for example, to create a new industry [82]. However, the combination of individual differences and the need to coordinate the activities of large numbers of individuals presents a problem. How can it be ensured that the efforts of each individual fits with the efforts of all the other individuals as required to achieve the common purpose?

For primates other than man, the maximum size of group is about 40 [83]. This limit is set by the need for enough interaction between members of the group (such as grooming) to maintain social coherence. If a group exceeds this size, there are consequences such as increased risk from predators. The group size for human Pleistocene hunters has been estimated at 25 [84]. However, around 12,000 years ago, archaeological evidence suggests settlements of about 200 people. As pointed out by Julian Jaynes [81], such a settlement size means that, for example, a leader cannot have a face to face encounter with each person every day. The coordination problem must be solved some other way. Groups of over 200 people need a hierarchical structure to enforce rules of cooperation. Still larger groups need rule enforcement involving the threat of violence.

In modern societies, the need to coordinate the activities of millions of unique individuals is addressed by moral standards, culturally defined behavioural norms, religious beliefs, legal and governmental systems, etc. All these systems are creations of higher cognition. Julian Jaynes has argued that the major driving force that led to human consciousness and self awareness is the need to make coordination of very large groups of human beings possible [81].

2.11 Art, Music and Literature

Animals can create remarkably complex structures, from spider webs to beaver dams. A particularly unusual example is the ornately decorated construction of the New Guinea bower bird. Male bowerbirds construct large structures of wood or straw, and decorate them with small stones, glass fragments, and bones. The purpose is to attract female bowerbirds. Bower quality and number of decorations have a strong effect on male success [85].

While human art may have “practical” purposes, such as attracting members of the opposite sex, propitiating natural forces supposed to be under the control of a god, or earning a living, the fundamental difference from animal examples is that a primary objective is to generate imaginative experiences for both the artist and the audience [86].

There is evidence for the existence of hominid art several hundred thousand years ago, such as the rough stone “Venus” carvings of Berekhat Ram and Tan-Tan, and the petroglyphs at Auditorium Cave, Bhimbetka. Examples of art become much more abundant from around 20,000 years ago with hominids that are anatomically Homo sapiens.

Similarly with music, animals like birds generate songs that appear to be assertions of territorial rights or attractors for females [87]. Human music may well include these uses, but another result is the generation of an imaginative, emotional experience.

There is nothing in animals that is analogous with fictional literature, and again a major phenomenon in human beings is the generation of an extremely complex imaginative experience.

2.12 Higher Cognition and the Brain

In this chapter we have briefly outlines some of the major components of what is viewed as higher cognition in human beings. The brain appears to be the anatomical organ that provides most of the biological support to higher cognition. The question to be addressed in later chapters is how activity in the observed complex assembly of interconnected neurons making up the brain corresponds with all the higher cognitive processes we have discussed.