22 November 2016
Investing in Lilt: Translation for All
We’ve longed for universal translation — a babel fish in every ear — for as long as we’ve interacted with each other.
One could make a case for a uniglot society but it’s hard to believe that we will see — or want — that anytime soon. So, we either rely on an army of human translators or one-size-fits-all machine translation provided by companies like Google and Microsoft. These translation options are expensive, prone to error or inaccurate. The old Italian saying, “traduttore, tradittore,” (translator, traitor) rings true. Lilt is changing that by creating cheap, reliable and customized machine translation models on-the-fly.
We’re pleased to see Lilt announce their new partnership with us and Redpoint. Below is some background on why we think Lilt will be a company of consequence. We start with some background on language translation, apply our virtuous loop mental model and talk about the new markets created by Lilt.
The Great Debate
Early approaches to language translation involved building formal models of syntax. The idea was that if we could represent language with a common syntactic model, then we could simply transfer meaning from one set of symbols to another. However, the meaning of a sentence is often more than the sum of its constituent words. For example, the sentence, “I like Ike,” somehow doesn’t have the same import in French.
Recent approaches to scaling language translation involve statistical and neural methods to induce models from data. The idea is that we don’t need a formal description of language if statistical tendencies in the data can generally tell us how to translate a word or phrase.
“the vast majority of people who study interpretation tasks, such as speech recognition, quickly see that interpretation is an inherently probabilistic problem: given a stream of noisy input to my ears, what did the speaker most likely mean?” —Peter Norvig
There is an ongoing debate about the optimal balance between linguistic and statistical methods. As much as we’d like to indulge our interpretation of that debate, you’re probably best directly reading about it, starting here: http://norvig.com/chomsky.html.
There is another debate about the optimal balance between statistical machine translation and neural machine translation. Suffice to say that, despite some great work by Google and others, neural methods are too expensive, unreliable and slow for production use. The best system uses a bit of each.
Lilt most sensibly resolved this debate by creating the world’s first adaptive machine translation product. The product uses statistical and neural machine translation methods to provide an initial translation of source text, lets a human correct that initial translation, then recomputes the translation model according to what that human thinks is correct. Lilt puts a human in the loop of machine translation for the first time in history. This is the best of both worlds, taking advantage of the cheap computing power, huge cross-language corpora and neural methods while not being naive to what humans know.
The Virtuous Loop
We’re completely focused on investing in intelligent enterprise companies. That is, companies that derive their competitive advantage from their unique data, and compound that competitive advantage with an intelligent system that learns from their data. One mental model we use to figure out if a company has this category of competitive advantage is the virtuous loop. That is, an architecture feeds public data, the anonymous customer data and data exhaust from the use of a product into machine learning algorithms to generate both cleaner data and insights. This leads to more use of the product, more data exhaust, and so on.
Applying this to Lilt:
- Lilt uses the United Nations and other data sets to train its models — public data;
- The customer uploads their Translation Memories (i.e. custom dictionaries) — anonymous customer data;
- Lilt generates a translation for the customer;
- The customer makes corrections to this translation through Lilt’s intuitive interface, and Lilt stores these corrections — data exhaust;
- All of this data is fed into both neural and statistical machine translation models;
- Lilt generates a new, better translation for the customer;
- The customer translates more text and makes more corrections;
- Lilt generates a new, better translation for the customer; and so on.
Lilt’s technology advantage is at every level of the ‘stack’. That is, the company built every element of this adaptive machine translation system so that it has full control of data collection, storage, model features and such.
Entirely New Markets
One of the biggest, remaining barriers to trade is language. Imagine if that barrier didn’t exist and we could increase trade to improve our quality of life. Scalable machine translation is thus one of the biggest opportunities for technology to improve our quality of life.
However, the language translation industry doesn’t currently look like it’s going to change the world, being largely made up of low margin labor marketplaces and low quality software vendors. Lilt has a big challenge ahead of it to bring this industry forward but it’s already being recognized as a leader.
We believe that Lilt will go beyond what translation companies do today — mostly localizing content — to enabling whole new categories of commerce and collaboration. With cheap, reliable and customized translation one can imagine a world where workers chat in realtime, customers get support and people sell their wares across borders. We’ve sized up each of these three markets and believe Lilt has a $22B revenue opportunity in just enterprise collaboration, customer support and e-commerce.
Frontier Dwellers
The quality of Lilt’s team is such that we wanted to partner with them after our first meeting. Aside from their intellectual and personal integrity, the Lilt team are true domain experts in language translation. The founders, John and Spence, worked on Google Translate. Their main advisor, Franz Och, started Google Translate and ran it for ten years. The founders completed their Ph.D.s in this area, carefully studying linguistics, statistical machine translation and the human-computer interface challenges of translation tools. They know what’s worked and what hasn’t and are on the frontier of what’s possible. We’ve found that partnering with people dwelling on this frontier have the best intuition about what to build for customers today.
