Compositionality and Bilingualism

Last week I put up a link to an online experiment.  Here’s the results! You can still do the experiment first, if you like, here.  Source code and raw results at the bottom.

Languages evolve over time under a pressure to be learned by a new generation.  Does learning two languages at once effect this pressure? My experiment says … maybe.

These pressures include ones for learnability (compression) and expressivity (able to express a large variety of meanings, Kirby, Cornish & Smith, 2008).  Bilingualism seems like an unlikely ability since learning an extra language leaves the speaker potentially no more expressive at a cost of an increase in the amount of effort required to learn it.  There is no pressure for one language structure (e.g. English) to adapt to another language (e.g. Mandarin) so that they can become optimally learnable and expressive as a single medium.  That is, there’s no reason to assume that expressivity and learnability pressures apply across languages (which are not being used by the same people).

Nevertheless, children display an aptitude and a willingness to learn and use multiple languages simultaneously, and at a similar rate to monolingual children.  Therefore, languages do seem to have adapted to be learnable simultaneously.  Does the compatibility of languages point to a strong innate property of language?  In contrast, it might point to underlying similarity in the structure of languages, brought about by universal principles of communication.

I suggest that the fact that learning two languages is not twice as difficult as learning one is partially to do with the compositional nature of langauge (the meaning of sentences is made up of sub-units of the sentence).  The alternative is a holistic langauge with a separate word for each meaning.  Compositionality allows language to be compressible and generalisable, and so easier to learn.  Imagine a 3 x 3 x 3 meaning space, like the one from Kirby, Cornish & Smith (2008), below (each word represents a single meaning, e.g ‘renana’ is a red square moving in a straight line):

The meaning space has 27 meanings, but in theory the whole space can be described with 9 words:  3 words denoting shape, 3 denoting colour and 3 denoting movement.  Additionally, compositionality allows ‘codeswitching’:  You can use sub-parts from either language interchangeably and still get the message through (unless the languages use similar sub-units for different meanings).  Learning two compositional languages simultaneously, therefore, may be easier than learning two holistic languages.

Experiment

I decided to test this by getting people to learn either two compositional or two holistic languages simultaneously.  I used a 2 x 2 meaning space which meant that the number of holistic words needed to describe all meanings and the number of compositional units were equal (4).  Here’s the four meanings:

Next, I created four ‘languages’ to create two conditions where each meaning had two labels.  The compositional condition had two compositional ‘languages’ (below left) and the holistic condition had two holistic languages (below right):

          

Each word was a CVCV nonsense word.  I tried to balance the distributions of letters to be equal across the conditions, but I see now that it could have been better.

I recruited 16 people through Replicated Typo to take part in the experiment online.   Participants were told that they would be doing a langauge learning experiment where each object had two words.  Participants were exposed to each label-meaning pair in their condition (8 words).  They were then shown each meaning and asked to recall both words associated with it.  Each participant went through 5 rounds of training and testing.

Results

I used two measures of how well each participant learned the words at each stage:  Total correct responses and total (normalised) Levenshtein distance.  The Levenshtein distance was calculated as the minimum number of edits that would be required to turn the participant’s response into the correct responses (word by word, expressed as a number between 0 = all perfect and 8 = all exactly wrong).

However, the compositional languages have an extra ‘codeswitching’ edge.  Participants may have been learning the sub-units correctly, but forgetting which sub-unit belonged to which ‘language’ (e.g. responding with ‘lupu’ for a red triangle).  Therefore, I also took the maximum correct responses and minimum Levenshtein distances for all responses where responses for the same object were spliced.  For example, if the participant responded with ‘lupu’ and ‘haka’ for the red triangle (instead of the correct ‘luka’ and ‘hapu’), then this would count as correct under the ‘compositional adjustment’.

The results for correct responses (without compositional adjustment) showed a main effect of test block (participants got better over time, see table below generated automatically in R with xtable), and condition, with the holistic group doing better on average in each test block than the compositional group (mean for compositional group = 3.6 words, mean for holistic group= 4.6).  There was no significant interaction between test block and condition.  Using the compositional adjustment score did not yield qualitatively different results.

ANOVA for number of correct responses

	Df	Sum Sq	Mean Sq	F value	Pr(> F)
Test Block	1	121.60	121.60	28.79	< 0.0001 *
Condition	1	22.85	22.85	5.41	0.0223 *
Block x Condition	1	2.11	2.11	0.50	0.4814
Residuals	91	384.39	4.22

 ANOVA for number of correct responses with compositional adjustment

	Df	Sum Sq	Mean Sq	F value	Pr(> F)
Test Block	1	121.60	121.60	29.08	< 0.0001 *
Condition	1	18.43	18.43	4.41	0.0385 *
Block x Condition	1	2.11	2.11	0.50	0.4792
Residuals	91	380.49	4.18

>
For the Levenshtein distance, there was a main effect of test block, but not condition.  With this measure, the groups are a lot closer.

ANOVA for Levenshtein distance

	Df	Sum Sq	Mean Sq	F value	Pr(> F)
Test Block	1	83.23	83.23	49.74	< 0.0001 *
Condition	1	2.29	2.29	1.37	0.2452
Block x Condition	1	0.34	0.34	0.20	0.6555
Residuals	91	152.25	1.67

ANOVA for Levenshtein distance with compositional adjustment

	Df	Sum Sq	Mean Sq	F value	Pr(> F)
Test Block	1	78.98	78.98	51.48	< 0.0001 *
Condition	1	1.50	1.50	0.98	0.3247
Block x Condition	1	0.23	0.23	0.15	0.7019
Residuals	91	139.61	1.53

Discussion

Contrary to the expectation that compositional languages are always equally easy or easier to learn than holistic languages, there’s a tantalising possibility that holistic languages (at least very small ones) are easier to learn simultaneously.  This might mean that the pressure for a langauge to become compositional might be less in a bilingual community.  At the least, the differences between the groups might suggest that they took different approaches to learning the two types of language.  They did not differ on the Levenshtein score suggesting that, in a real communicative situation, there may not have been such a big difference between the groups.

Doing this study online limits some options.  I would have liked more participants and to have more training and test rounds.  But there’s only so much you can ask people to do.  Perhaps feedback during testing would be a good idea, too.  I’d like to do this experiment properly with the traditional people-in-a-lab setup, and run people on a single langauge for a control, too.

Download

You can view the html and javascript source for running the experiment here and download the results here : scores , words.

References

Kirby, S., Cornish, H., & Smith, K. (2008). Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language Proceedings of the National Academy of Sciences, 105 (31), 10681-10686 DOI: 10.1073/pnas.0707835105