How About a Theory of Direct Memory?

There’s nothing so practical as a good theory.

Peter Hancock (2018) makes an interesting case that designers should explicitly address time. I agree that the consideration of time is important, and I’m going to use his article as an opportunity to discuss something related – the relationship between the way we think about memory and the way we conceptualize time. I believe that thinking about time in the wrong way has led to a scientific approach to memory that produces little or nothing that’s actually useful, at least for us designers and human factors professionals.

For decades, maybe from the beginning of psychology, psychologists have conceptualized memory as a storage system resting somehow “in the head,” containing some sort of “traces” or “stored representations” that are dredged up and used to recall the past. There are thought to be two interrelated systems: short-term, which we use to, say, remember a phone number for a few seconds, and long-term, which we use to remember, say, what our grandfather looked like when we were a child. In this way of thinking, studying memory involves the study of how we do it – how we store the traces in the first place, how they get from short-term to long-term storage, how the traces are retrieved, and so on. Memory research thus involves experiments designed to provide evidence for or against various “box-and-arrow” models of how memory works.

My problem with this is the same problem that J. J. Gibson identified with traditional models of visual perception: They don’t lead to predictive consequences and are, in fact, an artifact of looking at the “input” to the system (light in the case of visual perception) in the wrong way. The simplest way to describe the problem is this: If we need to know what goes on “in the head” in order to predict what someone sees, then we can, by definition, never predict anything because the very definition of in the head is that we don’t have access to it under real-world circumstances. So we have to know something that, in principle, we can’t know. It doesn’t help that we may find brain measurements in the lab that help us predict, because we don’t have access to these brain measurements under normal circumstances, when we would want to find predictive patterns.

Anyway, Gibson’s ecological approach – focusing on finding patterns in the “stimulus” that have predictive consequences – has made a lot of progress and led to such things as flight simulation. And his affordance concept has found widespread use in design because it provides a very practical way of thinking about how we can directly perceive how we can use things; that is, what they afford for us.

Another way to think about the problem is that we human factors people are trying to successfully predict from environmental variables (design characteristics of buildings or products or workplaces) to human results (proper behavior and/or good experiences), so we need theories that find relationships between these classes of variables, the ones we are able to manipulate. “Hidden” variables in the head don’t help because we aren’t designing minds or brains; we’re designing products, environments, services, and so on.

From yet another point of view, Gibson showed the value of getting rid of the crutch of an information-processing approach to visual perception in favor of direct perception. The former allows you to postulate hidden variables that are, in my opinion, excuses for the failure to achieve accurate prediction. The latter – the view that you “see what’s there” – forces you to find patterns in the environment that lead predictably to certain visual experiences.

So what about memory? My question is, “What happens when you get rid of the thing-in-the-head/storage-bin model of memory in favor of what we might call direct memory?” The question becomes not “How do we remember?” but, rather, “What do we remember?” . . . just as Gibson asked not “How do we see?” but “What do we see?” Such an approach should enable us to find predictive patterns that we human factors people can actually use – that allow us to predict from patterns of past experiences to present remembering. This new discipline of direct memory would ask, “What can we do to an object or event to make it more memorable?” or “What can we do over time in how we present information to make something easier to remember?”

One barrier to direct memory, I think, is that we take it for granted that we have direct access to the present but that things in the past have a fundamentally different “epistemological status” – they aren’t exactly real in the same way as the present is real. Thus, the purpose of our storage-bin models of memory is to provide us with something in the present that gives us (indirect) access to the past.

But what is the basis for treating the present and the past in fundamentally different ways? We don’t necessarily remember things perfectly, but neither do we see with perfect accuracy. And what is the duration of this present that is real? A second? A millisecond? At any rate, I propose that, regardless of how we think about the fundamental status of the past versus the present, it’s more productive to think of the object of memory as something in the past rather than as some hypothetical trace in the present. Of course, we need a nervous system to have memory, but I’m proposing that we think of the brain as giving us access to the past rather than as providing a present object that mirrors the past.

I know that I’m proposing something quite radical, something that violates what most of us have accepted as unassailable − that the present is “real” but the past is not. However, as I see it, this unexamined belief locks us into the need for a theory of memory that never seems to get us anywhere; we search for “stored mental representations,” retrieval mechanisms for the representations, etc., etc., but never seem to achieve useful prediction.

Thus, I propose that we replace the storage-bin metaphor with other metaphors that could prove more useful. One potentially useful metaphor is that of photographic exposure. With a short exposure, we get one photograph; with a longer exposure, we get something different. For example, if we leave the shutter open for a week on a crowded urban scene, the moving objects disappear in favor of those things that remain unchanged for the week. In one case, we “remember” a particular collection of pedestrians and automobiles, with the buildings and streets obscured; in the other, maybe only the buildings and streets. It’s not that the scenes have been “processed” or “stored” differently; it’s that there are two fundamentally different “pasts,” with different durations, so what is remembered is different.

From an evolutionary point of view, the purpose of memory is surely to help us act appropriately. We wouldn’t have survived very long if we couldn’t remember what we could eat and not eat, or where we could go safely versus where danger lies. It follows that the important function of memory is to inform action, not to recall specific objects and events. Somewhere in our evolutionary history, we developed the technique of memorizing things, of conjuring up iconic images, and so on. Where I think we’ve gone astray in our theorizing, though, is to think of this memorization as the core of memory – to think of the essence of memory as recall rather than recognition. In other words, the essence of memory is being able to act more effectively as a function of our past experience, no different, really, from learning. Iconic recall is a sometimes-accurate trick that we developed fairly recently in evolutionary history, presumably as a function of language, but it makes no sense to use such memorization as the core of memory, the focus of our theories of memory.

In sum, I propose that those of us who are designing products and systems should think of memory as about the relationship between patterns in the past and present experiences rather than as about the internal observation of stored representations of the past. We want the users of our products and systems to remember how to use them, and we probably want them to be memorable. At present, I don’t see that the psychologists purportedly studying memory are giving us any significant help. It’s time, I think, to think about memory in a fundamentally different way, which, in turn, requires thinking about time in a fundamentally different way.