Managing Localization Files

Starting with TYPO3 4.6, the new localization format is XLIFF. But what does it mean? Should you care as extension developer? How to deal with different localization format? This article aims at describing the different formats and how to manage your labels efficiently.

Ease ll-XML Management

First of all, I would like to start with something that proves to be not known very well by extension developers.

You certainly have some extensions (possibly of yours) that comes with huge locallang*.xml files, containing lots of labels in… many languages, don’t you?

Here is such an example with only a few labels for each language:

<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<T3locallang>
   <meta type="array">
      <type>module</type>
      <description>Language labels for extension 'your-ext'</description>
   </meta>
   <data type="array">
      <languageKey index="default" type="array">
         <label index="pi1_title">My extension</label>
         <label index="pi1_plus_wiz_description">Adds a list of stuff to the page.</label>
      </languageKey>
      <languageKey index="fr" type="array">
         <label index="pi1_title">Mon extension</label>
         <label index="pi1_plus_wiz_description">Ajoute une liste de trucs à la page.</label>
      </languageKey>
      <languageKey index="de" type="array">
         <label index="pi1_title">Meine Erweiterung</label>
         <label index="pi1_plus_wiz_description">Fügt eine Liste von Dinge.</label>
      </languageKey>
   </data>
</T3locallang>

There is a high chance you did not know that you could split this file into separate files for each and every language.

English, referencing the other languages:

<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<T3locallang>
   <meta type="array">
      <type>module</type>
      <description>Language labels for extension 'your-ext'</description>
   </meta>
   <data type="array">
      <languageKey index="default" type="array">
         <label index="pi1_title">My extension</label>
         <label index="pi1_plus_wiz_description">Adds a list of stuff to the page.</label>
      </languageKey>
<languageKey index="fr">EXT:your-ext/path/to/fr.locallang.xml</languageKey>
<languageKey index="de">EXT:your-ext/path/to/de.locallang.xml</languageKey>
</data> </T3locallang>

French (same applies for German):

<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<T3locallangExt>
<data type="array"> <languageKey index="fr" type="array"> <label index="pi1_title">Mon extension</label> <label index="pi1_plus_wiz_description">Ajoute une liste de trucs à la page.</label> </languageKey> </data>
</T3locallangExt>

I chose to save my split localization files within the same directory as the main locallang*.xml file, using a language-key prefix just as when you retrieve translations from TER (which get stored within typo3conf/l10n).

But this decision is up to you when using ll-XML localization files. You may as well store them in fileadmin if you start your reference with "fileadmin/" instead of "EXT:your-ext/".

Please note that when using XLIFF, you will have to store them next to the main language and will have to prefix them with the language key, thus why not do the same with ll-XML?

Switching to XLIFF

You may wonder how an XLIFF localization file looks like? Here is the German localization file corresponding to the labels we had above:

<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<xliff version="1.0">
   <file source-language="en" target-language="de" datatype="plaintext" original="messages"
                                     date="2011-08-25T17:54:43Z" product-name="your-ext">
      <header/>
      <body>
         <trans-unit id="pi1_title" approved="yes" xml:space="preserve">
            <source>My extension</source>
            <target>Meine Erweiterung</target>
         </trans-unit>
         <trans-unit id="pi1_plus_wiz_description" approved="yes" xml:space="preserve">
            <source>Adds a list of stuff to the page.</source>
            <target>Fügt eine Liste von Dinge.</target>
         </trans-unit>
      </body>
   </file>
</xliff>

In fact, the major difference between ll-XML and XLIFF — apart from the XML structure of course — is that XLIFF allows you to store a single language. You cannot store all your translations in a single XLIFF localization file.

The easiest way to switch from ll-XML to XLIFF is to install the extension  extdeveval from TER and choose the corresponding function in the drop-down list:

Convert locallang.xml files to XLIFF

Managing XLIFF files

Once you converted your locallang*.xml files to XLIFF, you may maintain them with a text editor or with a desktop application such as  Virtaal.

Virtaal
French Context Sensitive Help (CSH) localization file in Virtaal

XLIFF in TYPO3 4.5 LTS and older

If you don’t want to maintain both ll-XML and XLIFF versions of your localization files, you may opt for completely switching to XLIFF and do not care anymore of ll-XML. But what if you want to support TYPO3 4.5 for instance? Just add a dependency to extension  xliff and you’re done! It will take of creating ll-XML localization files out of your XLIFF version, thus easing your job.

Advanced Usage of XLIFF

You may add internal notes for your translators. They will be rendered by Pootle, on  http://translation.typo3.org but at the moment unfortunately not when using Virtaal.

Notes rendered in Pootle
Internal notes as rendered on TYPO3 translation server
<trans-unit id="_.image" approved="yes" xml:space="preserve">
   <source>EXT:cms/cshimages/pagemodule_13.png,
EXT:cms/cshimages/pagemodule_14.png,
EXT:cms/cshimages/pagemodule_15.png</source>
   <note from="developer">This string contains an internal text, which must not be changed.
         Just copy the original text into the translation field. For more information have
         a look at the Tutorial.</note>
</trans-unit>

Overriding Labels with Custom Translation Files

Perhaps you know the existence of the $GLOBALS['TYPO3_CONF_VARS']['SYS']['locallangXMLOverride'] array? It allows you to override some or all labels of a given translation file (ll-XML or XLIFF) with your own custom translation file. It simply allows you to define an additional overlay on top of the official labels.

Overriding ll-XML Localization Files

$GLOBALS['TYPO3_CONF_VARS']['SYS']['locallangXMLOverride']['path/to/originalTranslationFile.xml'][]
= ’path/to/otherTranslationFile.xml’;

where otherTranslationFile.xml may contain all or part of the labels of originalTranslationFile.xml and any number of languages. In fact, you may override XLIFF localization files with ll-XML localization files or the other way around but if you use XLIFF for otherTranslationFile.xlf, then you have to read following section.

Overriding XLIFF Localization Files

The syntax is the same when overriding the default (English) language but as XLIFF may contain only a single language, there is a way to let you define additional overriding files for the other languages you wish to override with your own labels:

$GLOBALS['TYPO3_CONF_VARS']['SYS']['locallangXMLOverride']['path/to/originalTranslationFile.xlf'][]
= ’path/to/otherTranslationFile.xlf’;
$GLOBALS['TYPO3_CONF_VARS']['SYS']['locallangXMLOverride'][fr]
[’path/to/originalTranslationFile.xlf’][] = ’other/path/to/fr.otherTranslationFile.xlf’;

Adding a custom language in Backend

Imagine you would like to provide a TYPO3 Backend in Switzerdütsch for your editors (could be seen as the locale de_CH). Is it possible at all? Starting from TYPO3 4.6, YES!

But does it mean that you will have to translate the whole German locale or start making a copy of it and adapt the texts? What about updates in the German texts you did not translate? It would be regrettable to be stuck with old labels. Fortunately, TYPO3 4.6 will come with a clever fall-back mechanism allowing you to create a de_CH locale, and use German (locale “de”) labels when a key is not found in your localization file and only fall back to English (default language) if it is still not found in the German translation.

How does it work?

First of all, you should register your new locale. Open typo3conf/localconf.php and add:

// Beware: make sure you don't "drop" previously added locales...
$TYPO3_CONF_VARS['SYS']['localization']['locales']['user'] = array(
    'de_CH' => 'Swiss German',
);

You don’t need to configure anything else as TYPO3 will automatically use German (language prefix of “de_CH”) if a translation is not found in your Swiss German localization file, before falling back to English as last resort.

If however you would like to use Austrian German (de_AT) before German and only then fall back to English, you could add this to your configuration:

// Beware: make sure you don't "drop" previously added dependencies...
$TYPO3_CONF_VARS['SYS']['localization']['locales']['dependencies'] = array(
   'de_CH' => array('de_AT', 'de'),
);

Now you have to create directory typo3conf/l10n/de_CH and start localizing your Backend. One of the must-have is to translate the name of the language “Swiss German” into … Swiss German by creating file typo3conf/l10n/de_CH/setup/mod/de_CH.locallang.xlf:

<?xml version='1.0' encoding='utf-8'?>
<xliff version="1.0">
  <file source-language="en" target-language="de_CH" datatype="plaintext" original="messages" product-name="setup">
    <header/>
    <body>
      <trans-unit id="lang_de_CH" approved="yes" xml:space="preserve">
        <source>Swiss German</source>
        <target state="translated">Swiizertütsch</target>
      </trans-unit>
    </body>
  </file>
</xliff>

Then choose your language within the User settings and enjoy:

You may download the few files I translated here:  de_CH.tar.gz (924 bytes)

Have fun!!!

FAQ

  1. [Q] Do we really need to duplicate all the strings in XLIFF files?
    [A] If you open the main localization file of some extension or the different localization files from Core, you will figure out that only <source> is defined. So yes and no, you don't have to duplicate English texts but you have to provide English as source and the corresponding translation as target for all other languages. In some applications, this allows retrieval by “source” instead of trans-unit id and allows typos in English to be fixed without breaking this retrieval by source in other languages.
  2. [Q] I don't see my new labels, why?
    [A] Localization files are cached to enhance performance. You may either manually clear the cache by removing files (or the directory itself) within typo3temp/Cache/Data/t3lib_l10n/ or you can add following line to your localconf.php:
    $TYPO3_CONF_VARS['SYS']['lang']['cache']['clear_menu'] = TRUE;
    This will add a clear entry to the context menu on the “storm icon” in the main toolbar, top left.
  3. [Q] How may I figure out in which file a label is defined?
    [A] Simply add the following line to your localconf.php, clear cache and reload the Backend:
    $TYPO3_CONF_VARS['BE']['lang']['debug'] = TRUE;
Flattr