Converting Langfiles using Bash and Translate-Shell

Introduction

The most mundane task a developer can be asked to do is creating new lang files (adding support for Spanish,German ) in your app. Unless you are using a library for dynamic translation in your app or browser capabilities,the whole process to convert the base language file (say English) to other languages is just time-consuming also if the software you are working on is huge, say 5k lines of text on the base language file. Converting the file manually will take you days.

So why not create a shell script to do the translation while you sip on some ☕

Using Translate-Shell

All the translation via web services in the shell script is done by an Open Source Library called Translate-Shell.

We run the translate shell with a bash ./trans.sh -R option to list all the set of supported languages.

Parsing the source file

Our goal is to write a shell script that will parse the source language file and output a translated language file in the chosen language.

Here is the code below of a sample lang file of a project in php.

1
2
3
4
5
6
7

<?php

$language['hello_text'] 	= "Hello World!!";
$language['story_label'] 	= "This is a story label";
$language['story_desc']     = "Testing for langage conversion";

?>

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

#!/bin/bash

bash ./trans.sh -R

echo "Enter the language code you want to test";
read lang;

echo "Enter word to test"
read word;

After we have the language and the word to test as input,we do a test run with a sample word and the list of engines (a file containing a list of engines supported by translate-shell) to see whats best service available for that language and availability of the web service at that time, recommend are google and bing but choose whatever you feel like.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

for j in $(cat ./engine_list)
do
	out=$(./trans.sh -b -t $lang -e $j \$word 2> /dev/null);

    if [ $? == 0 ]; then
     	echo "engine $j working -> $out ";
    fi

done

If the test translation executes , show the sample translated texts to judge for accuracy, the user is now finished with test translation from various engines on the list and can now select an engine based on availability and accuracy.

We set the Internal file separator as \n so files are processed line by line, i iterates over every line in the english_lang file sed 's/[ ;]*$// english_lang removes all ; (not permanently) and the output is stored in a variable.

We then split each line in the english lang file into keys and values, the key will be the same in the destination language file and only the values needs to be converted.

1
2
3

key=$(echo $i | awk -F'=' {'print $1 '});
  text=$(echo $i| awk -F'=' {'print $2'});

To separate the key and values pair from the records in english file, we use AWK.

Using AWK we specify the field separator ‘-F’ that indicates how the record should be split,

the records are split into parts using = as a separator, and are stored in variables $1,$2 respectively.

Completing the implementation

Below is a sample implementation for php based lang files, feel to use it in your own conversions.

Find the entire project on github here

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

#!/bin/bash

bash ./trans.sh -R

echo "Enter the language code you want to test";
read lang;

echo "Enter word to test"
read word;

for i in $(cat ./engine_list)
do
	out=$(./trans.sh -b -t $lang -e $j \$word 2> /dev/null);

    if [ $? == 0 ]; then
     	echo "engine $j working -> $out ";
    fi

done

echo "Enter the engine you want to use for the translation";
read engine;

IFS=\$'\n'

for i in $(sed 's/[ ;]*$//' english_lang); do

key=$(echo $i | awk -F'=' {'print $1 '});
  text=$(echo $i| awk -F'=' {'print $2'});

if [[ \$text = _[!\ ]_ ]]; then

    	value=$(./trans.sh -b  -t $lang -e $engine $text);
    	value=$(echo $value| tr -d '"');
    	echo "$key =\"$value\"";
    	echo "$key =\"$value\";" >> output.txt;

fi
done

Demo