The Machine Reads Shona Now

Late one night, mostly out of mischief, I typed a Shona proverb into a chatbot and asked it what the proverb meant.

It answered in about a second. The answer was fluent, well organised, and grammatically immaculate. It was also, in a way I found hard to articulate at first, wrong — not factually wrong, but wrong in the way a translation of a joke is wrong. Everything was accounted for and nothing was funny.

What it had produced was the meaning of the proverb. What it had missed was the fact that in my family that proverb was never used to mean what it means. It was used, almost exclusively, to end an argument about money, and it was used with a particular flatness of tone that indicated the speaker had decided the argument was beneath her.

Fluency is not understanding

This is the trick these systems play, and I say this as someone broadly enthusiastic about them. They have solved fluency ahead of understanding, which is the reverse of how it works in people, and it scrambles our instincts. We are calibrated to read fluency as a proxy for depth. A person who speaks beautifully about a subject has usually thought about it. A machine that speaks beautifully about a subject has usually read about it.

For most tasks this does not matter. I do not need a machine to understand a shipping manifest; I need it to process one. But language is not a manifest. A language carries a set of arguments its speakers have already settled — about what is polite, what is owed, what may be said directly and what must be approached sideways.

Every proverb is a compressed argument that a community won a long time ago. Decompressing it correctly requires having been there.

The part that should worry us

Here is the thing that actually kept me up, and it is not the usual worry.

Shona is now, for the first time, being interpreted at scale by something that has never been argued with in Shona. The corpus these systems learn from is thin, largely written, largely formal, largely produced by institutions — bibles, newspapers, government notices, academic papers. The living register, the one my grandmother used to close a discussion about money, is almost entirely oral and almost entirely absent.

So the machine will learn a version of my language. It will then be used, at scale, to translate, to teach, and eventually to generate. And the version it learned will slowly become the version that is available.

This is not a conspiracy. It is a sampling problem with a very long tail of consequences, and it will happen to a great many African languages more or less simultaneously, mostly by accident, mostly while everyone involved has good intentions.

What to actually do

I am wary of essays that diagnose brilliantly and then recommend that we raise awareness. So, concretely, three things I think are worth doing, none of which require permission from anyone:

• Record the old people. Not for a project. For a hard drive. The oral register is held by a generation that is leaving, and a phone in a pocket is now a competent recording studio.
• Write in the language. Badly is fine. The corpus is thin because we all chose English for anything we considered serious, including, I note with some embarrassment, this essay.
• Be suspicious of fluent answers about your own culture. They will get more fluent. That is precisely the problem.

None of this is a rejection of the technology. I use it daily and it has made me more productive than any tool I have owned. But I would like the machine to read Shona the way my grandmother spoke it: with the argument still inside.

I asked it the proverb again a few months later. Better answer. Still not funny.