EvoLang Preview: Using Minecraft to explore Language Evolution

Replicated Typo is doing a series of previews for this year’s EvoLang conference.  If you’d like to add a preview of your own presentation, get in touch with Sean Roberts.

At this year’s EvoLang Liz Irvine and I will be talking about how pointing can inhibit the emergence of symbolic communication.

Usually, pointing is thought to help the process of bootstrapping a symbolic system.  You can point to stuff to help people agree on what certain symbols refer to.  This process has been formalised in the ‘naming game’ (see Matt Spike’s talk):

1. I request an object by naming it (with an arbitrary symbol)
2. You guess what I mean and give me an object
3. I point to the object that I meant you to give me (feedback)
4. We remember the name that referred to this object

This game is the basis for many models of the emergence of shared symbolic systems, including iterated learning experiments (e.g Feher et al., and Macuch Silva & Roberts).  Here’s some robots playing the naming game in Luc Steels’ lab:

However, the setup of these experiments assumes one crucial thing: that the individuals can’t use pointing to make the request in the first place.  Most experiments are set up so that participants must communicate symbolically before they can use pointing.  If you allowed pointing to be used in a naming game, then it would probably go something like this:

1. I point at the object I want.
2. I request an object by naming it (with an arbitrary symbol)
3. You guess what I mean and give me an object
4. I point to the object that I meant you to give me (feedback)
5. We remember the name that referred to this object

That is, if we’re good enough at pointing then we don’t need a symbolic language for this task.

Of course, there must have been some task in our evolutionary history that provided a pressure for us to develop language.  We set out to explore what kind of task this might have been by running an experiment in Minecraft.

Minecraft as a tool for language evolution research

Participants were given the task of building a complex structure together in Minecraft – a computer game which simulates a 3D world that you can interact with, a bit like life-size Lego.  Here’s that they had to build:

However, they were only given half of the plan each:

This simulates something like people having ‘specialisations’, e.g. one person knows about building wooden things and another knows about working stone.

This meant that they needed to communicate about locations and colours of blocks, but they were not allowed to talk.  Instead, they could only use their virtual avatar to ‘point’ at stuff by directing their gaze at something, then jumping up and down (we realise that this isn’t normal pointing, but it’s as close as we could get in Minecraft).  Below is a video of two people working together to place some blocks (the perspective from both players’ viewpoints are shown, but in the actual experiment they could only see their own perspective.  Halfway through, player B realises a mistake has been made and removes some blocks before continuing).

They’re able to cope pretty well without a symbolic language, just by using pointing and trial-and-error.  In a second condition, we allowed another group of people an extra communication channel – knocking on the table.  However, instead of developing a symbolic system, they used this mainly for attention-getting and signalling simple pragmatic things like ‘yes’ or ‘let’s move on’.  Below is an example of this (the scrabbling sounds in the background are just the players’ mouse moving quite close to the microphone).

Note that one player uses some blocks as ‘referents’ to indicate what colour should be placed – a clever tactic that avoids the use of a symbolic system.  We even saw the emergence of quite complicated interactions.  In this one, there’s some confusion about which block should be placed where.  They players have set up a series of ‘reference’ blocks to one side, and one player uses them to ask “you mean a white block?”, and the other player confirms by also tapping on the reference block.  This is essentially a naming game sequence, with a request, a guess and some feedback, but it has emerged in a meaningful cooperative task.

No participants came up with a symbolic way of referring to blocks.  The pair who completed the task fastest didn’t even use the extra ‘knocking’ communication channel.

From talking with the participants, we found that some had thought about setting up some kind of system, but didn’t want to spend the time doing it, since it would have taken too long and they could do the task just fine.

Clearly, symbolic language does confer some advantages – I’m communicating with you right now in a way that would be much more difficult with pointing only.  However, setting up a symbolic system takes time and effort.  If the task demands are not enough to justify that effort, then symbolic signals are unlikely to emerge.  In fact – we’ve found is that it’s actually pretty hard to think of an experimental scenario that’s somewhat ecologically valid where people would need to do this.

We have some ideas about how this task could be tweaked so that symbols do emerge, and you can hear more about this at our EvoLang talk.

Irvine E. and Roberts S. (2016). Deictic Tools Can Limit The Emergence Of Referential Symbol Systems. In S.G. Roberts, C. Cuskley, L. McCrohon, L. Barceló-Coblijn, O. Feher & T. Verhoef (eds.) The Evolution of Language: Proceedings of the 11th International Conference (EVOLANG11). Available online: http://evolang.org/neworleans/papers/99.html