DeCaBERT is a RoBERTa-architecture MMT for languages in the widely-discredited Dené–Caucasian language macrofamily. This includes languages as diverse as Basque, Chechen, Tibetan, Mandarin, Tlingit, and even Navajo! Though not very linguistically sound, I find the theory really interesting, and I want to run some experiments that might shed some light on the validity of the theory.

The dataset spans 24 languages, mostly spanning common crawl data, Leipzig corpora, and bible translations. There aren't really many real-world cases I can think of for this model. Perhaps machine translation from Basque to Western Apache?