For this year’s Christmas card, we’ve made an AI that composes riddle-poems. Here’s an example one:
My first is in box but not in ballot,
My second is in music but not in studio,
My third is in shares but not in prices,
My fourth is in spirit but not in pride.
That was written by the computer. How does it work?
The AI has an advantage over people — it can, quite reasonably, run through the dictionary (squared) looking for pairs of words to use. With a pared down dictionary, that takes about a minute. However that alone does not make satisfying riddle-poems, because the words have no flow. Here’s an example:
My first is in expose but not in upside,
My second is in gummy but not in alkalis,
My third is in shaggy but not in beefy,
My fourth is in salon but not in grimed.
So we added in some smarter reasoning: the word pairs should be related.
Using this, the system produced output with a more satisfying feel. However we weren’t quite there…
Out came lines like “My first is in grief but not in anger” — poetical, but not very festive. Also the more grisly “in murder but not in torture“, and the suggestive “in bra but not in robe“. So since this is a Christmas card, we put in a filter to avoid negativity, sexual imagery, and some emotionally loaded topics.
Picking related word-pairs is done using an approach called “word-vector semantics”. The key insight is a simple idea: words that are used in similar ways have similar meanings. Let’s fill that out with some technical details – What kind of christmas card would it be without technical details? You take a corpus (a set of documents) such as Wikipedia, and build a big table of which words are used together, called a co-occurrence table. This table is too sparse – the noise is relatively high and these vectors don’t give good results. So you look for a smaller set of dimensions, say 300, and a projection that condenses the big table into that smaller space whilst preserving patterns in the data as much as possible. This can be done by finding eigenvectors. One effect of this reduction is to remove a lot of the noise. Now for each word, we have a 300-dimension vector — and comparing those vectors is a good way to judge how closely related the word meanings are.
You can try it here: Riddle Writer.
Picture: Xmas Robot (cc) Iconoclast