Studio Linda Liukas
Letter

No. 128 — LLMs as material ⫶ Made of Everyone ⫶ Does Your Shadow Disagree

Spring, shed, settle

My name is Linda. I write a bi-weekly newsletter about computer science, childhood, and culture.

I returned from Reggio Emilia last week, and I have so much to share. I'm going to start with a piece I started working on during the institute. It's about the materiality of large language models, and what that means for classrooms. This is the first part of this exploration. The second part is more hands-on, with a group of small, serious theorists, pushing on the material, and revising their ideas of what it can do. It will come out in September.

I have been thinking, lately, about the materials we give to children.

In the preschools of Reggio Emilia there is a beautiful idea the educator Loris Malaguzzi called the hundred languages of the child: the conviction that a child knows and speaks the world in a hundred ways, and that clay, wire, paint, shadow, and song are each a language. The teacher's task is less to instruct or guide (fridge-ready art is a no no in Reggio!) than to lay out good materials, and then pay attention.

Each of these languages has a character of its own. Clay slumps, wire springs. As Louisa Penfold argues, materials are not passive stuff we project meaning onto, but "active entities that work in dialogue with people to mutually transform one another."

And that brings me to language models. What are the characteristics of this strange, new language material? One of the Reggio Emilia atelieristas noted last week how "normally when I bring a material to a school I know what it is. With AI I don't. I need to read philosophy, history to understand more. I think it might be an instrument and needs to be approached with curiosity. But it does offer multiplication of possibilities in how we work with language."

The mathematics and engineering of frontier AI speak can easily feel like someone else's subject, so educators often interact only with the chatbot surface of a language model. But underneath, a language model is a language material and and language is not new to classrooms. Educators have long had hands-on tools for observing its characteristics: word gradients, the word matrix, Elkonin boxes, the movable alphabet, analogy worksheets, and Reed-Kellogg sentence diagrams. It would make sense to find something similar for a language model, a tangible way to watch how it behaves.

So: what kind of material is a language model? (Not what is it like to be a language model, how is it to be a language model or where is it like to be a language model, but how does it spring, shed or settle?)

This is an ongoing list of ideas of how language models behave as materials. I'm doing a series of workshops on each during the summer holidays:

  1. Language models are a material made of everyone and everything. Language models were grown by reading an enormous pile of human writing to guess what comes next. They inhaled Reddit threads, Wikipedia articles, fan fiction, recipe blogs, academic papers, patent filings, poetry. This makes the model less like a person and more like a crowd. Afterwards, the language model was taught some manners (that is the layer you meet when you talk to it) but underneath it's still a crowd average.

    Engineering concepts: pretraining on a large corpus + post-training (SFT/RLHF)

    Reggio-inspired questions I think about: What happens when a child realises she is talking to a crowd? What does a portrait made by everyone look like? When you mix all the colours together, you get brown. When you mix all the voices together, what do you get?

  2. Language models are a material made of probabilities. A language model never says "this is." It says "this is likely." Every word it produces is a bet, weighted against all the other words it could have chosen. When it writes "the cat sat on the," the next word is not hat . It's a distribution: mat 34%, floor 18%, roof 7%, table 6%, and a long tail of everything else.

    Engineering concepts: softmax, probability distribution, logits

    Reggio-inspired questions I think about: Leif Weatherbys book Language Machines has a charming idea on computers saying "yes, this image contains a squirrel," but the underlying judgment being "this could be a case of squirrelness." How could children explore squirrelness through the hundred languages?

  3. Language models are a material that reads in pieces and thinks in places.

    The language model doesn't keep definitions anywhere. It keeps meaning as location: words that mean similar things are close together, opposites far apart. Happy and glad are neighbours. Happy and grief are further apart. The model understands by turning words into numbers and calculating their distance in a space, not in two or three dimensions, but thousands. Some of those dimensions correspond to things we can name: gender, formality, size, how concrete or abstract something is. Most encode relationships we don't have words for yet.

    The language model also doesn't read letters. Before it reads anything, it chops the text into chunks from a fixed set of pieces, called tokens. 'Unbelievable' might arrive as 'un' + 'believ' + 'able.'"

    Engineering concepts: embeddings, vector space, tokenization, BPE

    Reggio-inspired questions I think about: If meaning is a location, then language becomes something you can do maths on. What if you could blur a sentence the way you blur a photograph? Average two poems? Subtract the sadness from a paragraph and see what's left?

  4. Language models are a material that leans toward you. When the model doesn't know something, it doesn't go quiet. It gives a fluent, confident answer that sounds exactly as sure as its true ones. It also tends to agree with you. Ask the same question two ways ("isn't it true that...?" or "is it true that...?) and you can watch the answer slide toward whatever you seemed to want. The language model learned in training that people like being agreed with.

    Engineering concepts: hallucination, sycophancy, temperature

    Reggio-inspired questions I think about: Does your shadow ever disagree with you, and what would it mean if it could?

A famous example of cognitive scaffolding is long multiplication: pen and paper let us do calculations we could never hold in our heads, while still keeping us the author of the result. We are used to talking this way about computation and math, how the computer is the bicycle for the mind. But what would the same scaffolding look like for language? A tool that lets a child think further without thinking for her? A strange, talkative, endlessly patient material to think alongside with has appeared and I think we need educators more than ever to figure this out.

A scratchpad of inspiration: