Commit cdc8aef0 authored by Xavier Perseguers's avatar Xavier Perseguers
Browse files

[!!!][TASK] Provision Pootle using Ansible

A complete Pootle server may be provisioned using Ansible. This includes
basic scripts to migrate from Pootle v1.9 to v2.5 and new scripts to
synchronize changes in TYPO3 system extensions with Pootle.

The major new feature is the support of removal of outdated labels,
renaming (and move) of localization files within a system extension and,
last but not least, basic support for moving localization file from one
system extension to another one (by copying existing translations to the
new locations when this is detected).

Packaging scripts will be adapted in further patches.

Please refer to README.rst for installation instructions.

Note: The jump into Ansible and the provided setup was inspired by
      https://serversforhackers.com/an-ansible-tutorial

Change-Id: I386e7177c5d72dc5d361d8ab29d2237e95a7be25
Reviewed-on: http://review.typo3.org/41148

Reviewed-by: Xavier Perseguers's avatarXavier Perseguers <xavier@typo3.org>
Tested-by: Xavier Perseguers's avatarXavier Perseguers <xavier@typo3.org>
parent 7511164c
.build-language-pack-lastbuild
.build-language-pack-state
tmp/*
# Sphinx-related files (local rendering)
index.rst
Makefile
conf.py
warnings.txt
_build
_static
_templates
=========================
Pootle Translation Server
=========================
This document describes how to install a personal Pootle translation server.
Requirements
============
* `Ansible <http://docs.ansible.com/>`_. We use Ansible to easily deploy Pootle and scripts to the server. The deployment receipes and scripts are found in this Git project.
* Linux server. We will use a blank `Debian AMD64 <https://www.debian.org/CD/netinst/>`_ virtual machine to start with. Installation was done with only a SSH server running and command :command:`sudo` being available.
Installing Ansible
------------------
Ansible is needed on a so-called "Control Machine"; that is, a computer that will control the Pootle server. This is typically your personal computer.
We will install Ansible from source since this is the recommended method. Just pick your preferred user directory and:
::
$ sudo easy_install pip
$ sudo pip install paramiko PyYAML Jinja2 httplib2 six
$ git clone git://github.com/ansible/ansible.git --recursive
$ cd ./ansible
$ source ./hacking/env-setup
Configuring Ansible
-------------------
Edit or create file :file:`/etc/ansible/hosts` and put a reference to your (blank) server::
[pootle]
192.168.81.128
Provisioning Pootle server
--------------------------
Run::
$ ansible-playbook -s install.yml
Administrating Pootle
---------------------
In default Pootle installations, an admin account (the password matches the username) is created with superuser privileges which can be used to administer the whole site.
.. note::
It's highly recommended that you change the password for the default admin account on your first login, or even delete the account
and assign superuser rights to another user.
When logging onto your server, you are encouraged to work as user "pootle" and follow the guide::
$ sudo su - pootle
Migrating data from production
------------------------------
- Copy po files from production and deploy to new server
- Dump the production database
- Remove every database table from new Pootle server
- Import production database dump
- Run::
$ sudo -u pootle /opt/local/pootle/bin/pootle --config=/etc/pootle/pootle.conf setup
- Migrate XLIFF to global space with unique id::
$ sudo su - pootle
$ ~/scripts/bin/migration/migrate-typo3-xliff
- Create Pootle projects:
- TYPO3.TYPO3.core.frontend
- TYPO3.TYPO3.core.compatibility6
- Update your password to be "password"::
UPDATE auth_user SET password='sha1$b5711$bb690a5a16cff60ae3b01fa285bcb2e68cd244f7' WHERE username='admin';
#!/bin/bash
set -e
. /var/www/vhosts/pootle.typo3.org/home/scripts/etc/pootle.conf
. /var/www/vhosts/pootle.typo3.org/home/scripts/etc/functions
python /var/www/vhosts/pootle.typo3.org/pootle/manage.py $@
#!/bin/bash
set -e
GIT_DIRECTORY="/var/www/vhosts/pootle.typo3.org/home/git/Core"
PO_DIRECTORY="/var/www/vhosts/pootle.typo3.org/pootle/po"
TEMPLATE_DIRECTORY="/var/www/vhosts/pootle.typo3.org/home/templates"
# Usage: project_exist project.name
function project_exist() {
echo -n "Check if project exist: "
if [ "$1" != "" ] && [ -d $PO_DIRECTORY/$1 ]; then
echo "OK"
if [ ! -d $PO_DIRECTORY/$1/templates ]; then
mkdir -p $PO_DIRECTORY/$1/templates
echo "Check if the templates exist: CREATED"
return 1
fi
else
echo "FAILED"
return 1
fi
}
# Usage: check_available_languages project.name
function check_available_languages() {
for LANGUAGE_KEY in $(ls /var/www/vhosts/pootle.typo3.org/home/templates); do
if [ "$1" != "" ] && [ ! -d $PO_DIRECTORY/$1/$LANGUAGE_KEY ]; then
mkdir -p $PO_DIRECTORY/$1/$LANGUAGE_KEY
echo "[NOTICE] Create project language \"$LANGUAGE_KEY\""
fi
done
}
# Usage: sanitize_template_file /path/to/file.xlf
function sanitize_template_file() {
if [ "$1" != "" ] && [ -f "$1" ]; then
xmlstarlet ed -d "/xliff/file[@xml:space]/@xml:space" $1 | \
xmlstarlet ed -d "/xliff/file/body/trans-unit/target" | \
xmlstarlet ed -d "/xliff/file[@target-language]/@target-language" | \
xmlstarlet ed -d "/xliff/file/body/trans-unit[@approved]/@approved" | \
xmlstarlet ed -u "/xliff/file[@source-language]/@source-language" -v en
fi
}
# Usage: list_typo3_typo3_core_projects
function list_typo3_typo3_core_projects() {
ls $PO_DIRECTORY | grep TYPO3.TYPO3.core
}
# Usage; check_typo3_typo3_core_projects_languages
function check_typo3_typo3_core_projects_languages() {
echo
echo "Check available language ,,,"
for CORE_PROJECT in $(list_typo3_typo3_core_projects); do
check_available_languages $CORE_PROJECT
done
}
# Usage: update_typo3_typo3_core_projects
function update_typo3_typo3_core_projects() {
echo
echo "Update TYPO3.TYPO3.core projects ..."
for CORE_PROJECT in $(list_typo3_typo3_core_projects); do
pootle-manage update_translation_projects --project=$CORE_PROJECT
pootle-manage update_from_templates --project=$CORE_PROJECT
done
}
function git_typo3_typo3_core_pull() {
echo
echo "TYPO3 Core GIT Pull from $1"
cd $GIT_DIRECTORY
git checkout master
git reset --hard origin/master
git fetch
git checkout $1
git pull
}
# We successively fetch from old to newer branches to get back files which
# may have been moved around. BEWARE: labels should not get deleted (well,
# possible if file is moved as well) and "templates" language MUST be
# manually updated (in GUI) to discover new files before they are mirrored
# to other locales
#for BRANCH in 'TYPO3_4-5' 'TYPO3_6-2'; do
for BRANCH in 'TYPO3_6-2'; do
# GIT pull
git_typo3_typo3_core_pull $BRANCH
# Check project language configuration
check_typo3_typo3_core_projects_languages
echo
echo "Update templates projects ..."
for FILE in $(find $GIT_DIRECTORY/ -name *.xlf | grep sysext); do
RELATIVE_FILE_PATH=$(echo $FILE | sed s#$GIT_DIRECTORY/## | cut -d/ -f4-)
PROJECT=$(echo TYPO3.TYPO3.core.$(echo $FILE | sed s#$GIT_DIRECTORY/## | cut -d/ -f3))
echo "$PROJECT/$RELATIVE_FILE_PATH"
# Check project configuration
project_exist $PROJECT || exit 0
# Update project templates
if [ $? = 0 ]; then
# Check if directory exist
RELATIVE_FILE_DIRECTORY=$(dirname $PO_DIRECTORY/$PROJECT/templates/$RELATIVE_FILE_PATH);
if [ ! -d $RELATIVE_FILE_DIRECTORY ]; then
echo "Create directory: $RELATIVE_FILE_DIRECTORY"
mkdir -p $RELATIVE_FILE_DIRECTORY
fi
# Deploy file from GIT
sanitize_template_file $FILE > \
$PO_DIRECTORY/$PROJECT/templates/$RELATIVE_FILE_PATH
fi
done
done
# Update all core project
update_typo3_typo3_core_projects
echo "====== UPDATE FINISHED ======"
POOTLE_SCRIPT_HOME=/var/www/vhosts/pootle.typo3.org/home/scripts
POOTLE_PID=$POOTLE_SCRIPT_HOME/tmp/fcgi-t3-pootle.pid
POOTLE_FCGI_SOCK=/tmp/fcgi-t3-pootle.sock
POOTLE_LOG=/var/log/pootle/pootle.log
POOTLE_ERR=/var/log/pootle/pootle.err
POOTLE_PO=/var/www/vhosts/pootle.typo3.org/pootle/po/
CONF_LANG_COMPAT=$POOTLE_SCRIPT_HOME/etc/lang-compat.conf
LOCAL_GIT_CLONE_DIRECTORY="/var/www/vhosts/pootle.typo3.org/home/git"
XSL_LLXML2XLIFF_SOURCE=$POOTLE_SCRIPT_HOME/etc/xsl/xliff2llxml-source.xsl
XSL_LLXML2XLIFF_TARGET=$POOTLE_SCRIPT_HOME/etc/xsl/xliff2llxml-target.xsl
_LAST_BUILD_FILE="$POOTLE_SCRIPT_HOME/.build-language-pack-lastbuild"
TER_L10N_PATH=/var/www/vhosts/pootle.typo3.org/l10n_ter/
# Ansible Role: Memcached
[![Build Status](https://travis-ci.org/geerlingguy/ansible-role-memcached.svg?branch=master)](https://travis-ci.org/geerlingguy/ansible-role-memcached)
An Ansible Role that installs Memcached on RedHat/CentOS or Debian/Ubuntu Linux.
## Requirements
None.
## Role Variables
Available variables are listed below, along with default values (see `defaults/main.yml`):
memcached_user: memcache
The user under which the Memcached daemon will run.
memcached_port: 11211
memcached_listen_ip: 127.0.0.1
The port and IP address (127.0.0.1 for localhost) on which Memcached will listen for requests.
memcached_memory_limit: 64
memcached_connections: 1024
Memcached limits. The maximum amount of RAM `memcached` will consume (64MB is the default), and the maximum number of simultaneous connections memcached will handle.
memcached_log_file: /var/log/memcached.log
The location of the memcached log file.
## Dependencies
None.
## Example Playbook
- hosts: cache
roles:
- { role: geerlingguy.memcached }
## License
MIT / BSD
## Author Information
This role was created in 2014 by [Jeff Geerling](http://jeffgeerling.com/), author of [Ansible for DevOps](http://ansiblefordevops.com/).
---
memcached_user: memcache
memcached_port: 11211
memcached_listen_ip: 127.0.0.1
memcached_memory_limit: 64
memcached_connections: 1024
memcached_log_file: /var/log/memcached.log
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment