Localization, Translation, and Machines

TL;DR: Is there research bringing together Software Analysis and Machine Translation to yield Machine Localization of Software?

I’m Telling You, There Is No Word For ‘Yes’ Or ‘No’ In Irish

from Brendan Caldwell

The art of localizing a piece of software with a Yes button is to know what that button will do. This is an example of software UI that makes assumptions on language that hold for English, but might not for other languages. A more frequent example in both UI and languages that are affecting is piecing together text and UI controls:

image0

In the localization tool, you'll find each of those entries as individual strings. The localizer will recognize that they're part of one flow, and will move fragments from the shared string to the drop-down as they need. Merely translating the individual segments is not going to be a proper localization of that piece of UI.

If we were to build a rule-based machine localization system, we'd find rules like

  • gaelic-yes: If the title of your dialog contains a verb, localize Yes by translating the found verb.

  • pieced-ui:
    For each variant,

    • Piece together the fragments of English to a single sentence
    • Translate the sentences into the target language
    • Find shared content in matching positions to the original layout
    • Split each translated fragment, and adjust the casing and spacing
    • Map the subfragments to the localization of the English individual fragments

    Map the shared fragment to the localization of the English shared fragment

Now that's rule-based, and it'd be tedious to maintain these rules. Neural Machine Translation (NMT) has all the buzz now, and Machine Learning in general. There is plenty of research that improves how NMT systems learn about the context of the sentence they're translating. But that's all text.

It'd be awesome if we could bring Software Analysis into the mix, and train NMT to localize software instead of translating fragments.

For Firefox, could one train on English and localized DOM? For Android's XML layout, a similar approach could work? For projects with automated screenshots, could one train on those? Is there enough software out there to successfully train a neural network?

Do you know of existing research in this direction?