GeoAI,counter-AI,and human geography: A conversation

Thanks.

I’m also joined by Krzysztof Janowicz (KJ), aka Jano, from UC–Santa Barbara and University of Vienna who also appeared on the panel.

Thanks for having me.

I should say that also appearing on the AAG panel were Song Gao (University of Wisconsin–Madison) and Clancy Willmott from UC–Berkeley, and I was also a co-organizer and speaker on the panel. Today what I’d like to talk about with our two guests is where they might fall on that spectrum, if we can put it that way, from proponents to critics, and what our guests today think of or see as the potentials of GeoAI. What makes it special, and what would be some of the ways we might interrogate it from a position of ethics in what's sometimes called Responsible AI. So it's quite a set of meaty topics to launch off with. And the structure is that I will occasionally interject with some questions, just to get things going, but hopefully act as a good listener. So maybe we could begin. I’ll start by asking you Jano not just what, in your view artificial intelligence is and perhaps its relation to machine learning, but also what it is not – that is, in your view, what are some of the misconceptions?

That's a very deep question for the beginning and you just blew through all the time we had! For instance, what is AGI (artificial general intelligence) is already a very difficult question. I would say it's a set of approaches and paradigms trying to emulate parts of human-like reasoning and cognition. This could include mimicking how humans conceptualize the world and act in the world. Of course, this is a very broad definition, one mirroring thinking from the 1960–80s. We distinguish between AGI, one that really tries to mimic human level intelligence, or maybe surpass human level intelligence, and then the narrower approaches that you just mentioned. Today this would include areas such as knowledge representation and reasoning as well as machine learning. I think that most of the progress we are currently seeing and what brings us together today is this narrow definition of AI. In my personal observation, the AGI direction has not only received less interest or recognition recently, both in academia and industry but also made less progress. I don’t think this is because it's less exciting or less applicable; it is just substantially more difficult. AGI research is not like a leaderboard competition.

I’m a science fiction fan as well as an academic geographer and when you talk about artificial intelligence in the science fiction community, it is AGI, which is even more specific than Jano's description; it is sentience. Whereas artificial narrow intelligence (ANI) is everything else. That everything else is usually deep learning because a lot of people just assume machine learning, which is more traditional heuristic, algorithms like Bayesian statistics and logistic regression. Some people will say that machine learning is our algorithm (or ANI) deployed in code. But in science fiction it's sentience. One of the interesting points is that sentience is not necessarily passing the Turing test, which is to fool people into thinking that they’re conversing with a human being, or a human being is, on the other end, making the decisions. One of the interesting observations that academics are making is that sentience does not have to look like human beings. There are many kinds of AGI that may very well be essentially autonomous decision-making machines that do not think like humans, but do not require humans. One distinction I would make is that, converse to popular thinking, AI is not automated. Currently, there are just so many insertion points where humans need to make interpretations, whether that's interpretations in terms of what's called feature engineering, or what you must do to manipulate the data before it goes into the machine, or a decision point about natural language processing in which you’re having to make decisions about what labels to assign to a cluster of topics. There are many places where humans must be a part of the loop, even though I don’t like the term human-in-the-loop, because often we assume that humans in the loop are decision-makers. In reality, they’re humans as workers supplying content.

When it comes to questions such as why do humans experience something, why do we feel in certain ways, why are we stimulated by colours or music, why is this beneficial from an evolutionary perspective, these are largely open questions. The subpart of AI that I’m working in doesn’t really offer any answers to these questions. Even cognitive scientists and evolutionary biologists do not fully understand these issues. Most strikingly, we are realizing that intelligence does require consciousness. This would be unthinkable in a conversation like this just 20 years ago. And even more groundbreaking are the implications of what we are seeing in some narrow AI systems. Namely that artificial intelligence is capable of creative works, not just parroting. I’m not sure whether we are doing ourselves a favour by constantly moving the goalposts on what intelligence means for machines.

Renée, could I pick up on a point you were making about automation. You used the term human-in-the-loop, and in your talk at the AAG you also mentioned society-in-the-loop. I wonder if you could say a little bit more about what that means, because that might be unfamiliar to some of the people listening to this.

Human-in-the-loop as it's used in human computer interfaces and also in the AI realm speaks broadly to a human in the algorithmic process ostensibly as decision-maker. However, human-in-the-loop really is humans as workers for most applications. A good example is crowdsourcing platforms like Mechanical Turk, where humans are necessary to either label data for the training data sets or to tweak different model outcomes. There are a variety of ways that humans are part of the algorithmic process. My concern is when we adopt that model to provide a more expansive notion of what constitutes decision-making oversight and, dare I say, justice. When we see concepts emerge like community-in-the-loop and society-in-the-loop, we do not relinquish the baggage associated with the original definition of human-in-the-loop so we’re still including all those people doing what others have called ‘ghost work’. This is an externality of ensuring our algorithms provide us with all this convenience. It's something that we pushed to the background, but it is still there. People can claim, well, we’re being participatory or inclusive or including humans, because they’re making determinations about in which block of an image is there a stop sign.

And that's very familiar to us when we try to navigate those reCAPTCHA questions, which I’ve heard are actually us working to train a machine learning system. Basically, we’re acting as unpaid labour to label imagery! But this comment about externality, I think, is a key one and certainly has come up in some related computational areas such as the blockchain and cryptocurrency, and that the only way these things can operate is by ignoring the externalities, and that's one reason why they can’t really scale. I do want to put a pin in that and come back to externalities. Somebody listening to this might say, well, all these things are all very well but they’re very far from human geography. So, I wonder if we can kind of circle back a little bit and talk about this thing which is getting some traction called GeoAI and perhaps talk a little bit about what in your mind it is or what you hope it can be. Because sometimes we hear that GeoAI is sweeping the discipline of geography, but we might bear mind that the discipline has previously been swept by other trendy notions, perhaps most prominently GIScience, and Song Gao made the connection very strongly between GIScience and GeoAI. So how do you see GeoAI?

I think there are two ways of looking at GeoAI. One is applying AI to a geo-specific problem. This puts us into the position of being tool builders. Or it could be viewed as what we call spatial explicit models, namely, how can we bake into AI or machine learning models an understanding of geographic space and time? We have lots of discussions about data versus theory and this one points directly to the role of theory.

Yes, that's quite an interesting comment because I think you’ve said previously that spatially explicit models perform better than non-spatially explicit models in this domain. If I’m hearing you right, ‘spatial is special’ because it gives us more powerful machine learning models.

Yes, of course, all the things that we could say about the ‘spatial turn’, like that the location of events is not only interesting because everything is located, but that understanding where things happen is key to understanding why they happen in the first place. The point is not whether we are special and deserve to be a scientific domain. Spatially explicit models often make a big delta and this matters. We can show that a model that is spatially or temporally explicit outperforms more general models even given less training data or more noisy training data. Let me give you one example. The way machine learning architectures are designed comes with many implicit assumptions and restrictions of what they can or cannot learn. For instance, some popular graph embedding techniques cannot handle transitivity or reflexivity. However, for instance, if you look at some of the founding pillars of our domain, like region connection calculus (RCC8), you cannot reason about topology without transitivity. So, no matter how much data you would give such a more general model, it would not be very useful. One would also need to learn formal properties of relationships from data first. The question is not whether this can be done or not, a human has to take this decision. Theory has and will always have a role in designing these models.

I’ll make two points that you may wish to pick up on. First, AI is not data. The same critiques of data cannot be universally applied to AI as in ‘we’re just repurposing our critiques of big data and we’re done’. Second, I think that AI is not representation, and I think that's a big problem for human geography and that's why we need to think more about cognitive science. The way we think of representation is predominantly visually in geography. It is the map. We may focus on images of convolutions or other visual representations of AI, but the critiques we have applied to geographic representations will not be able to be universally applied to necessary critiques of AI. We need to glassbox the blackbox of AI but our current critiques fall short. Human geography has a lot to say to the outside world, but particularly to AI scientists who want to think about location as discretized X, Y coordinates that have no geometry or topological relationship to each other. Software architecture approaches like MapReduce ¹ slice and dice geographic content in a way that does not retain basic spatial structures. Human geographers can talk about space and place relations that get lost in the computation. Human geographers can talk about how paths – not just where you are, but the path you travelled – can be an effective personal identifier. They can discuss the way that the modifiable aerial unit problem, a classic in geography, can be used for you or against you, in terms of surveillance or in terms of customization. That's human geographers talking to the outside world. A separate question is what can human geographers say to each other? Are human geographers willing to hear that they need to know more about AI than a superficial understanding of what they read in someone else's critique? That is perhaps harsh and we’ve had these conversations before about how much GIS you need to know to be able to critique GIS. I argue that human geographers need to have a conversation about how much AI human geographers need to know, even if it's numeracy, to be able to critique this novel thing. And that has all sorts of implications, for how many geography courses do we devote to numeracy or coding or algorithmic understandings? Not only in geography as a discipline but also in our human geography courses. So the question is what content is replaced? Human geographers also can offer the outside world a more nuanced understanding of the ethical responsibilities of the field of geography to these locational algorithms. I hesitate to even use the word ethics, because ethics is often sidetracked to morality or to virtue ethics (aka best practices or codes of conduct). Those will not make GeoAI sensitive to the needs of marginalized people as well as the needs of us as we’re continuously individuated and being enveloped into the big machine.

There's so many interesting thoughts here, I’m having trouble picking where to start! But one thing that maybe I should have said at the beginning: AI or GeoAI are growing so quickly and I’m only knowledgeable about a very small part. That said, one thing that makes me nervous about these critiques of GeoAI is that they’re not really critiques of GeoAI, they’re critiques of the people who use or develop GeoAI. These methods can be used for good or for bad. And it's always the humans in the driver's seat that make the decisions about what to model, how to train, how many classes to distinguish, and so on. Let me give you an example related to human geography and it's going to be a controversial example. Recently, I looked into the facial recognition from imagery literature, given we had these very substantial discussions about gender and gender identity in the United States. Several papers even up to 2019 state something like ‘gender identification via facial recognition is a “solved” problem’, which meant the accuracy was so high that machines were as good as humans or maybe even better as humans at assigning a gender given a face. Would a paper make such a statement in 2022? What I think is even more interesting is that, from a feature space perspective, nothing has changed in those few years. We as humans are the ones who decide which categories count and how many categories there should be [or that there are discrete categories]. It's us in the driver's seat. This leads to an interesting observation about the ways we do AI research – and I would love to hear your response to this – namely, we constantly aim at higher accuracy to outcompete some baseline, but obviously this is substantially easier if we only have few categories to distinguish. As a society, we have to decide on potential tradeoffs between predictive accuracy and possibly lower granularity in our categorization schemes, or do we want to have the full expressiveness of human thought and humans’ right to define which categories matter? But we may end up in a situation where the space of extractable features does not map onto our categories anymore. My own research, for instance, is about supporting diverse schemata.

First, you’re actually arguing that facial recognition technology cannot accurately predict gender. Second, I’m not convinced accuracy and number of categories are inversely related. I’m not convinced they’re related at all. Finally, accuracy, prediction; these words are part of loaded questions. ‘How can you be against accuracy?’ We can be against accuracy if it's inaccurate and, as importantly, reduces its utility. I’d argue that, to be useful, an algorithm needs to express nuance. Who cares about mappability to output space if the algorithm isn’t delivering what we need? More importantly, I am not willing to give the algorithm a pass. Sure, there are human drivers of the algorithms, including the opportunity cost of choosing to develop one algorithm instead of another. There are humans in the loop all throughout the algorithm development and decision-making space, but I’m not willing to give the algorithms themselves a pass because arguing there are dual-purpose technologies (as in technologies can be used for good or evil) allows us to say the algorithm itself is neutral, it is not situated. Algorithms are not neutral; algorithms are not objective. Consider your example above. The decision to embed certain performance metrics like speed, like binary determinations or discretized categories, or not focusing on fuzziness; these are issues within the algorithms. An algorithm, at least when we’re talking about deep learning, is not an equation, it is the tuning and only the tuning. So it's designed, it has kind of an agency. Humans absolutely were a part of determining that agency, but we should never say that algorithms themselves are neutral, nor that they are not making their own decisions and it's only how we apply them that's the problem. If we’re applying a shortest path algorithm, we have built algorithms that decide the shortest path. Why is the shortest the best? I think that we go down a dangerous road if we say (and this was a major critique of GIS), algorithms do not have a kind of agency or at minimum momentum. We are in peril if we dismiss that momentum and say it's just us.

On the issue of geographic information systems, I believe that these technologies have been used for good and evil, as all other technologies. GIS has been used for a lot of good, for instance in the disaster relief sector. But when it comes to the ‘algorithm’, I’m happy to provide a counter-perspective. I do think algorithms are neutral, because it's the humans who decide to develop and deploy them. For instance, if we talk about machine learning, this is matrix multiplication and calculus and there's nothing more to it. Keep in mind that to some degree the plasticity of neural networks is what makes them so flexible, before training they are domain and task agnostic. We humans decide on the loss function, the appropriate pooling operation, or softmax as activation function. And the very same is true for the shortest path, it always returns the shortest path. It's mathematically very clearly defined. But whether the shortest path is the fairest path, or the path that leads to the most economic imbalance, that is where humans matter. An algorithm is just an algorithm, it has no agency. Most of them are deterministic sets of instructions with well-understood termination criteria.

Let me address one point. I would argue that geographers do not have the agency that they think they have in terms of driving these algorithms. They are hostage to algorithms that have been originated by others, by computer scientists. You can go to the ACM or the IEEE all you want; we will still be viewed as people who apply stuff. That's a problem that I brought up in the AAG panel. It's hard for me to see sometimes where the true innovations in GeoAI are, where we are not merely just applying somebody else's tool, the master's tool as Audrey Lorde would say, although I am interested in counter-AI in applying the master's tool against the master. To that point, I think we’re in a liminal space now where we have with the sophistication of software libraries, we have tools and, hopefully, some people will build UIs on top of those to make AI even easier to use. We have the tools that we can reshape how the AI is deployed in a way that's much more sensitive, both to spatial awareness and also societal awareness.

Essentially, computer science is about two principles: scalability and abstraction. The interesting outcome of computer science is the realization that whether you’re standing in line at the bank, the fuel station, or your grocery store, if you arrive first in the queue, you will be served first, which is not the case, for instance, for stacks of things. The need for such abstraction is what makes computer science such a prominent discipline. From a data structures perspective, everything else is an application question. That doesn’t mean computer science would believe that geographers have nothing original to say or that our contributions have less importance. I do believe that we are making quite substantial contributions to the general body of knowledge about foundational aspects of AI. We shouldn’t suffer from computer science envy and, by the way, I believe, computer scientists are suffering from physics envy, and physicists from pure math envy, and, of course, mathematicians suffer from philosophy envy. What I am trying to say is that researchers in chemistry, for instance, have a lot to say about human behaviour, but that does not imply human geographers are just applying chemistry.

Renée, you mentioned using some of these tools as a counter-AI. And it's hard not to draw a comparison with similar moves that we’ve seen in geography in the past to GIScience. Starting in the early 1990s, we got the emergence of something like a critical GIS that tried to situate GIS not just as a set of tools – and this is where I’m drawing a comparison to GeoAI – but putting it into a broader techno-capitalist context. Understanding, therefore, that a lot of these algorithms exist for the purpose of turning a profit. On the other hand, you can look back at this debate between GIS and its critics and say, and it was a very productive debate, because it gave us some Open Source alternatives to proprietary GIS. It gave us, for example, QGIS. What might be some of the similar lines of development today? How might we learn from the case history of counter-GIS, for counter-AI?

Those are big questions! I fear that the arguments about counter-AI will be identical to counter-GIS; the arguments of critics will not recognize the unique properties in the sheer opacity in emergent properties of AI. One of the ways that counter-GIS was easier in geography was because GIS produced a map, it produced an image, and it was easier to critique the image and then what followed from that was critiquing the underlying data that created the image. It took a while for people like Robert Rundstrom (1995) to look at the underlying software architecture, the relational data model, and how that eliminated the ability to have, say, fuzzy categories. Here's one of the few examples in our literature in which the architecture itself prevented a way to understand the world. It reduced the abstraction in a way that was not sensitive to the people, in this case Indigenous people. AI is different, as I said, AI is not data, although we tend to focus currently on classificatory algorithms and we tend to focus far less on reinforcement learning, which is rules-based and does not require a ton of data. That being said, there is an opacity of big data that did not show up in many models of GIS because GIS was still relatively small data. Another difference is the sheer opacity of the modelling, which unfortunately is not improved by XAI (explainable AI) methods to glass box the AI. XAI can further obscure how something works. It was far easier in GIS than it is with many of the deep learning algorithms we’re contemplating like GPT-3, Open AI's text generator. We actually don’t know how that works; we can infer, but we should know how it's making the decisions it's making internally to see if there are any biases or harms. In many ways, counter-GIS is going to show up when we talk about counter-AI. I think that's fair, but I think we need to talk about the unique properties of counter-AI and counter-GeoAI because, as Jano is implying, the amount of algorithms that rely almost completely on location as the determinant of the outcome is kind of staggering and underreported. Whether or not developers are applying/building the algorithms correctly, appropriately is an open question. We should be questioning that.

I was thinking of a talk you gave a couple of years ago, where you developed your take on the ladder of political participation where counter-AI was a culmination of increasing ways of engaging with GeoAI.

It is difficult to insert lived experience into GeoAI, although one could argue that that could be one of the parameters of the input data. But it's very difficult for non-experts to participate in the design and deployment of GeoAI or AI systems generally. I’m trying to figure out where those insertion points are and Arnstein's (1969) ladder is instructive because it talks about how so much participation is performative. We talked earlier about human-in-the-loop, and I offered my critique that it is a kind of performative participation, where it looks like you’re meaningfully including humans, but you’re really not. I would add a rung to that ladder that AI can actually be used destructively as counter-participation; you can actually destroy public participation. You can destroy people's lives with a false arrest from predictive policing. You can use bots and surveillance to destroy, harm, or inhibit participation. There are also forms of token participation, including responsible AI, which is often technocrats deciding what's best for society but never actually talking to people impacted by AI. There are ways in which the public can take to the top of the ladder and be a little bit more subversive; they develop their own AI applications to counter the state, to counter technocapitalism. There's always the argument that you’re using the techno and you’re part of the capitalist system so you’re part of technocapitalism, but actually you can do sousveillance or to at least throw a sandal into the machine to do some sabotage.

I like that Jeremy at the beginning asked what is AI anyways, and the reason I answered the way I did was to be able to counter these arguments. First, I think critical GIS and counter-AI are very important investigations because they give us a platform for discourse that we need to know where society should go in the future. For instance, my own critique is that, currently, the biggest decisions are made by a couple of companies without any democratic process and without us having anything to say about major aspects of how society will function in 2030. However, I disagree with statements such as that we don’t understand the decisions that AI is making. This is why earlier on I insisted that (narrow) AI does not take decisions. This is not what's happening at all, and this is why I made the provocative example that neural networks are just about massive multiplication. In terms of the design phase of deep neural networks, it is very clear to the developers what will happen at every single stage. Of course, when you’re in the training or application phase and you’re trying to understand an individual result of, say, a classification task that's a totally different beast. At no stage is an agent-like AI involved that takes any form of decisions. Saying otherwise allows us to get away without asking the really hard questions. We are the agents who decide to deploy a certain algorithm, to parameterize a model; we picked the training data, allowing for it to be biased. For instance, a major problem with the application of ML is that it is often trained on data with a long temporal drag, so to speak. Hence, the wealth of data from the past influences what a model will predict about the future. But often, during phases of rapid change, the past is nothing like the future. That's all us. The beauty of machine learning, and this is why it has so much success, is that it's really domain-agnostic, you can take almost the same neural architecture and apply it to recognizing ore deposits or cats. To a substantial degree, training is what makes the difference. Consequently, ethics for GeoAI is an ethics of technology that recognized the social and environmental responsibilities involved in designing and using location-enabled AI systems. To give you an example, in my lab's research we are asking questions such as whether the idea of developing ever bigger single models is truly superior to having several regional models to account for regional variability. This is, to a certain degree, counter-AI, because you’re not only growing the space of potential decisions to be made or the granularity of data, but we also care about understanding and giving space to more diverse schema knowledge (ontologies).

I’m still not willing to give up that algorithms are not neutral! This may be an irreconcilable difference between us. The car is an example of a device that humans have huge input into the design and origination, and yet it has a crude agency, where it determines how we live as much as we determine how we live. We’re literally designing urban spaces to fit the car. We have pulled ourselves out of the loop in terms of that decision-making process. Latour speaks of the de-politicizing or the de-labelling of artefacts and how they become black boxes. It's no longer so-and-so developed blank; it's not Euler's shortest path, it is shortest path, it's not Douglas-Poiker, it's just shortest path. We remove the signifiers and we turn them into objects and artefacts that have their own agency. Perhaps we’re in this liminal space where we can tinker a lot more with the algorithms but they’re increasingly black-boxed, there are more hidden layers than ever before. They’re more opaque and more embedded and combined with other algorithms so you don’t even know where one algorithm begins and ends and another one begins and ends. It's liminal because AI's open for discovery and more and more algorithms are just getting baked into the mix, including all the discrimination, harm, and bias. Amazon's HR fiasco is a classic example of bias where Amazon used an AI to triage resumes and is instructive because this is a case in which Amazon wanted greater diversity, especially for women. One of the big fails was it used a classification algorithm and the classification was based on successful resumes which came from men. The algorithm was picking up phrases like women's colleges or women's basketball and noting in the training data that the word women didn’t appear, so therefore that couldn’t be a successful applicant. Even after explicit references to women were removed the algorithm was still selecting white or Asian men. The algorithm was picking up the subtle differences and acculturated ways that women speak or have been trained to speak, as opposed to men. These are very subtle and hard to find even if you computationally de-bias the model or you try to humanly de-bias the model.

Our positions seem similar here. Your argument is that we invented the car, but now the car determines how we get around; if somebody does not have a car, because they can’t afford one, there are certain jobs that they will never be able to do. I agree and I understand how you relate this to algorithms and the ownership of data. We need to be careful with the words here. The algorithm, for instance, Dijkstra's shortest path, is not the black box. The same is true for the design of artificial neural network. The black box, so to speak, appears when trying to inspect any one outcome of a deployed ML system. I agree that this can be troublesome and, in policymaking, this may not be the most comfortable situation. For instance, we can ask whether it is good practice to publish substantially more complex and difficult to apply models if their delta over simpler, less data and energy-hungry models is small. Also, how practitioners decide which choices to make and methods to select is not well understood.

I think you’ve brought up a good point that needs to be emphasized – that is, we’re often not talking about algorithms, we’re talking about neural architectures. The moment you talk about an algorithm is the moment that you’ve already trained it. An architecture is a series of ‘neurons’, a mesh. When we talk about algorithms, we usually mean things that already have been partially trained or tuned, so we should actually be talking about the neural architectures and not the algorithms themselves. One of the challenges when I’m working with NLP (natural language processing) is what is now called transfer learning. The days when people started from the raw architecture is largely gone. Now they always start from a model that has been trained on a human language and then they layer on domain information. Virtually no one in the NLP space anymore is dealing with naked architectures. They’re building on top of this, so that speaks to the need for a deeper understanding of the architecture. One of the most influential papers that was presented at the FAccT (fairness, accountability, and transparency) conference, the same conference at which Bender et al. (2021) presented the famous Stochastic Parrots article, there was another paper on Chinese transfer learning and language models. The argument was made that in this Chinese case, the Chinese Government has such control over the baseline language that's fed into this NLP architecture, they can literally erase words from showing up. To do that initial training on which all other applications are built, they can fundamentally alter all subsequent applications. And that's something that the data scientists and even Timnit Gebru didn’t fully comprehend because the core idea is that increasing amounts of data will refine the original algorithm.

I agree with the problem of stacking and naively reusing massive pre-trained models. I also agree that we are responsible for the algorithms we design and the data we select. Interestingly, some of the novel technologies you mentioned have this characteristic that they can be combined and composed seamlessly. However, the same broader ideas around statistical models such as the older latent Dirichlet allocation (LDA) have opened up new ways to understand very large bodies of text, for instance, to structure them and cluster them. I can hardly think of a field that is more suited or would benefit more from current trends in machine learning than geography.

Keep in mind that many of these techniques like LDA have their philosophical basis in instrumentalist ethics, the greatest good for the greatest number and so I’m still not giving the algorithms a pass!

I like that you’re not willing to! This brings us back to the principles of scalability and abstraction. GeoAI, AI, or our examples of classical algorithms like the shortest path are popular across so many different scientific disciplines because they can be adapted easily. Shortest path works on the road and for networks of friends. This is their strength and can only be their strength because they are domain-agnostic. This is not to take away from the responsibility of the algorithm designer, but to say that these very foundational and learning-based methods are difficult to grasp from within a do-no-harm framework.

But why should everything be scalable?

That's a good question. I’m a big believer in the value of friction. That's why I am a film photographer. Time, cost, intention, limitations in general, they give meaning to the making. But this decision not to scale the creation of artifacts is a human decision. Deep learning is all about scale, namely scaling billions of tiny computations.

The thing I did put a pin in was externalities, and what this current conversation makes me think about is that one of the responses that's been developed to that is responsible AI, or responsible research and innovation (RRI). Is there a way we can think about responsible GeoAI and what would that even mean? Is it technology that we’re concerned with here, or are we talking more about changing more structural societal values? I’m thinking of Ruha Benjamin's work where she has said that technology is not a producer of bias, rather societal inequalities produce certain kinds of technologies. The technologies are a symptom of an underlying structure, not the cause of inequalities.

You mentioned blockchain earlier and it's a fantastic example of a technology that was created to overcome certain power structures and the friction they introduce and that ended up creating new power structures. This is Renée's original critique. For instance, going back to GPT-3, only very few players worldwide can still compete in this space, but we are all using their results.

It's important to say what is specific to AI and what is specific to GeoAI. To a large extent, these issues are generic to AI and not GeoAI. I don’t have a good answer about the special properties of GeoAI that should require us to embed externalities – with the possible exception that it is possible for some algorithms that location is so co-linear with race, for example, or income, that you can’t use location. That would up-end a lot of algorithms if you had to eliminate location or make it so coarse that it's unusable. You do your XAI and you see that location is a major determinant and you have to get rid of it. Then what happens? That's a major challenge to our field. When we talk about accounting for externalities, we need to explore the term accountability. It is thrown around a lot in responsible AI, it is thrown around a lot in the AI ethics community, but when you pierce the veil of this important-sounding word, it may have little to do with government regulations, for example, enforcement, culpability, and compensation, and have only anything to do with explainability/ justification. Why were you denied a mortgage? Why were you denied a visa to enter a country? Why were you falsely arrested? Justify yourself! Oh well, the algorithm told me blah blah blah. It's like the old 1970s arguments about, well, the computer told me to do X. Accountability solved. Sometimes the computer does tell you to do things and you do them, unfortunately, which is part of my point of not absolving the algorithm of responsibility. That being said, it's important for us to pierce the veil of accountability if we’re to incorporate externalities like environment, the implications on ghost work. It means regulations; it means fiscal accountability. Of course, any accounting for externalities is going to run into a wall, and that wall is going to say, ‘you’re slowing innovation and you’re going to cause us to lose the AI arms race. One thing that human geographers can do in this political economy of AI is look at the national security implications of making these arguments for accountability. And looking at accountability as an important element of regulation, or slowing things down, is who gets to determine what constitutes accountability. In responsible AI, often the accountability is superficial; it's performative; it's a checkbox. Did you include the public, aka did you include impacted individuals in your design decisions? AI developers will say ‘sure check that off’ and then you get a high mark for public consultation or input and that's a win. Responsible AI – this is a part of the problem of the whole responsible AI industry – is that it's both regulating and being funded by the AI developers; these big tech firms. You can’t serve these two masters equally well because they will get mad at you when you propose regulation or propose embedding these externalities in a meaningful way. And when I say meaningful I also do not refer to personal accountability. Researchers should definitely do more when they’re proposing a new language model to consider the environmental impacts, but to me that's a lot like the recycling argument. We all have a responsibility to the climate, so you know, recycle stuff. We need to move from individual responsibility to collective, structural changes. Switching back to AI, it's big tech firms that are having the largest impacts and they should be the most responsible. We should slow them down and we should force them through regulations to incorporate more of these externalities.

Going back to Renée mentioning science fiction at the beginning of our discussion. If there is a ‘Great Filter’, it is something like a misuse of cognitive engineering, the manipulation and creation of an attention economy. For me personally, responsibility means to stop this trend. Without proper accountability, Silicon Valley is currently ripping apart the fabric of society. AI plays a part in this, I agree. Still, this is not a problem of algorithms as such, it's a problem that we are not made for the times we’re living in. Humankind is starting to push against boundaries of how biological intelligence and genetic encoding work. This causes friction that will only be resolved by changing the system and, in this case, it will probably end up by us getting ejected from that system. This brings us back to the discussion of what intelligence is. Given how we are manipulated, are we intelligent?

Let me add that in this call we’re all white and we’re looking at ourselves as canaries in the coal mine, but one thing that human geography adds is that we can look to marginalized communities; they are the canaries in the coal mine, not us. The impact of algorithms are already impacting who gets an organ transplant; who gets treatment in the veterans hospital for adequate pain medication; who gets a visa or not. Which may suggest why there may be fewer false arrests of white people than there are false arrests of Black people. There are people who are already impacted and human geographers who have already studied this in enormous detail that can point us to the implications of these technologies, now and in the future.

Yes, we need to understand these issues and how AI and big data contribute to them or even cause them. It may be worthwhile to think about where these problems are located, which of those problems are foundational AI problems in terms of how those systems work, whether they are procedural problems, for instance, in how ML is trained, or whether they are application problems like using a technology that clearly isn’t ready or well understood yet. Where on this spectrum a problem sits – I don’t believe this is very well understood, and I believe that we probably need to look a lot more into this. My intuition tells me that the foundational AI part is the furthest from being understood as far as social consequences are concerned but also unlikely to contribute to the problems we discussed today. Most of the problems seem related to the free and open availability and applicability of ML/AI technologies and pre-trained models without proper ethical considerations or legal frameworks. One could say that we took some of the most positive and empowering ideas and technologies, we then created a new wild west out of them. It's Pandora's box.

This is one way, unfortunately, that a counter-GIS is likely going to map on the counter-AI or critiques of AI, because human geographers may not understand how little reception there is for some of the critiques that they/we make. Part of the challenge of the ubiquitousness of AI is that we can critique and, like in science fiction, the critique will bounce off the atmosphere. There isn’t necessarily a willing audience as there may have been in the past to accept these critiques because my phone is really useful to me and knowing where I am in space and time is unbelievably convenient for me. So, I don’t care about your goddamn critique; don’t take my phone away! I think that we need to be cognizant of this reluctance to hear us. And you know part of that is the convenience also is racialized. People don’t want to hear problems until they affect us personally. If it's happening to people but they’re not the same income or racial or gender class as us then we won’t listen to them. So I think human geographers have a lot of work to do to get a larger public to care.

I love what you’re saying, but I think, unfortunately, this is where things get so complicated. Take me, for instance. I don’t have social media, I’m not on Facebook. I’ve never been on Facebook, because of the privacy implications, I am very, very picky about sharing my location, I do not install apps that offers coupons but steals my data, and so on. I am paying a substantial price for this as industry, government agencies, and even our own universities constantly push us into these data economies. I do not have access to a part of the 21st century. But then I realize that all this is actually an indication of privilege. For instance, I have the luxury of deciding to turn off my navigation system and figure out the path from A to B for myself. People who are already on the receiving end of life are not in a situation where they can say ‘I take the freedom of making decisions more slowly’ or ‘I’m not going to install the app that offers a good deal but steals my data’, because they really need to care about every dollar and they really need to get quickly from A to B, for instance, because they have to work two jobs. This freedom of me choosing to say no is, at the same time, a freedom that I only have because I’m incredibly lucky. Put differently, a lot of what we called ‘convenient’ before is very empowering to many and that's why it's so difficult to get rid of the related data economies and related technologies.

This has been a fascinating conversation and I wanted to thank you both for being such good sports and going well beyond the advertised time! There's still, on the other hand, much more that we could say, and no doubt the readers of Dialogues in Human Geography may want to pick up on some of them – we invite the conversation to continue.