New APPS

Regular readers of this space will know that I think large language models are deeply fascinating, in addition to being a little scary (depending on their use).  I also think that we can get some traction on both of those things by way of post-structuralist language theory, or at least, by way of Derrida.  I was thus very happy to finally read Leif Weatherby’s Language Machines: Cultural AIO and the End of Remainder Humanism, which came out earlier this year.  Weatherby’s thesis is, in brief, that the structuralists were right about language, and that we need to see this to have any hope of understanding language models and directing them to good use.  I’ll hopefully have more to say about various parts of the book later, but for now I want to offer a high level outline.

Weatherby begins by arguing that “nothing less than the problem of meaning, in a holistic sense, surfaces when language is algorithmically reproducible,” such that “this problem can be addressed only if linguistics is extended to include poetics … reversing the assumption that reference is the primary function of language, grasping it rather as an internally structured web of signs” (2).  This is because “the new AI is constituted in an conditioned by language, but not as a grammar or a set of rules.  Taking in vast swaths of real language in use, these algorithms rely on language in extenso: culture, as a machine” (5).

His immediate target is an assumption about language that he calls the “ladder of reference,” which he thinks “LLMs prove … cannot possibly be correct:”

“The ladder of reference is a hierarchical vie of language in which the ability to refer is primary. It is a notion of language as first and foremost world-representation. It need not be atomistic, in the since that a single word refers to a single object: apples to apples. But whatever is not referential is taken as secondary to reference” (13).

(Aside: I think a case can be made that the ladder of reference is parallel to the logic that Derrida doesn’t like in Searle’s invocation of the “normal” in speech act theory: the idea that the “normal” use of a term is the reference point for meaning/use.  The point in both cases is to have a non-arbitrary anchor and origin point for language.  It is of course precisely this that structuralism rejects)

Where does this ladder of reference come from? 

“I assign a majority share of responsibility for the ladder of reference – especially in computer and data science, but also in general – to the passage of logical positivism from an endangered Vienna to an ascendant United States. That group, which quickly became entangled with American behaviorism and what was called ‘descriptive’ linguistics, divided strictly between meaningful propositions and ‘poetry’ or ‘metaphysics’” (14).

Evidence that Carnap’s shadow is still with us, Weatherby argues, is in the symbol grounding debate.  He finds equally non-productive what he calls “remainder humanism,” the various debates about where to draw the line between machine language and something authentically human. Here the targets are Chomsky and the Bender/Gebru “Stochastic Parrots” paper.  Weatherby’s argument is that you can’t meaningfully draw that boundary because ‘human language’ “now exists in an extensive nonhuman linguistic environment that cannot be easily separated from it” (31).  The theoretical problem with remainder humanism is thus that “attempts to draw a dividing line between ‘real’ and ‘artificial’ language all run aground on the fact that language itself it the demarcator of the artificial, in the notion of the symbolic order” (35).  The practical consequence – as becomes evident in the last chapter – is that we lose the ability to respond intelligently to the interweaving of LLMs and global capitalism and the harms generated by that specific conjuncture.

As he argues in the conclusion, “language, as both poetic and ideological generation, is [now] on offer ‘as a service’” on the model of platform capitalism (195).  This means that “AI is not a solution [to the problems AI causes], but it is a reality, and we cannot afford to remain in a resistance stance to it as it becomes a general-purpose technology (the other ‘GPT’) for global capitalism today” (197).  I take it that he means that we can’t resist AI as such; the point of all of this is to resist the AI capital conjuncture.  Weatherby’s achievement is to show that to resist LLMs as such is generally to get involved in a theoretically incoherent account of language, one that makes it harder to resist the extractive capitalism behind AI.  What we need is “luddism,” perhaps, but with the recognition that the luddites were resisting the specific uses of technology by the capitalist class for its harm to workers, as Jathan Sadowski has recently emphasized.

At the point that AI offers what Weatherby calls “ideology” (the preceding chapter defines this term quite specifically – it is not what we believe vs. the truth; ideology is constitutive and comes much closer to a “they say,” the culturally aggregated sense of the way things are: think Ted Chiang’s characterization of ChatGPT as offering a “blurry JPEG of the web;” Weatherby says this “mixes metaphors in a way that muddles medium specificity but has some appeal nevertheless”(228n5)) as a service, we need to learn to distinguish the labor from the management of writing,  The need to do this tracks the history of computing, when early machines precisely separated the labor of computation from its management.  There is a reference to Matteo Pasquinelli’s Eye of the Master, and its thesis that AI is in essence the automation of social cognition; Weatherby endorses the direction of the thesis though he thinks one doesn’t need to take it as far as Pasquinelli does, suggesting that “the transformation of intelligent activities into forms of labot seems like a straightforward and continuous process of base-to-superstructure ‘last account determination’ to me, and so part of what Marx called the ‘real subsumption of labor’ (230 n7). For both Pasquinelli and Weatherby, Charles Babbage is a key figure, as he worked to make computation amendable to the factory division of labor.

Weatherby casts the resulting effects in Marxian terms: LLMs will extend and augment our power to generate language; that productivity bump will inevitably lead to the requirement to produce more of it, such that “we might say that intellectual labor is here really subsumed under capital, where it was only formally subsumed previously” (202).

What makes the present moment specific and important is that this circuitry is visible and hasn’t been fully smoothed over in the capitalist machine.  As Weatherby puts it:

“When language is a service, capital must run on data-semiotic pathways. Those pathways will always be partly visible, but the momentary visibility of the poetry-to-ideology spectrum that is being seeded in this crucial moment in our economic-cultural process will surely disappear. This book has been an attempt to fight the coming obscurity of the synthesis of language and computation” (206).

As the preceding should indicate, there’s a lot in the book.  There’s detailed discussions of how to think about Kant (and the argument that Chomsky is a Kantian, not a Cartesian) in the context of all of this; of the strengths and limitations of the distributional hypothesis about meaning (with reference to Lydia Liu’s work on Wittgenstein); as well as extensive discussion of structuralism and post-structuralism (he’s not a fan of Derrida’s interventions).  It significantly elevates the conversation about language models.