Queues

The backbone of the a10n l10n automation is queues. Queues enable us to run changes through the system, modify them, and fork them. Queues enable some processing steps to be monolithic and state heavy, while other processing steps are lightweight on status and can be parallized.

Implementation

The implementation of a10n is in python. We’re using kombu as low-level queue API, and rabbitmq as queue implementation. Redis as queue backend doesn’t seem to support requeue, so even if we use redis for other aspects, the queue itself should be rabbitmq.

Configuration

The configuration is held in a10n/settings, a python module suitable to configure django. Local configurations go into a10n/local.py, an adaptation of a10n/local.py-dist.

Entrypoints

The entrypoints to the queues are the version control systems. We’re starting off with a poller on http://hg.mozilla.org/, which is ported over from master-ball.

Hg Poller

The hg poller is based on twisted (13.0.0 at this point). It’s feeding hg-push events into the hg queue. These events have the following structure:

{
    "type": "hg-push",
    "repository_id": int,
    "pushes":
    [
        {
            "id": int,
            "date": int,
            "user": str,
            "changesets": [str]
        }
    ]
}

Workers

Workers may be singletons or parallelized tasks. Singletons are mere kombu workers with Consumers, whereas parallized tasks are implemented on top of celery.

Mercurial

The mercurial worker listens to the hg-push messages on the hg queue. It pulls local clones and updates the elmo database by forwarding the data from the message on to ‘elmo’s pushes.utils.handlePushes.

The worker implements a simple error recovery, and retries on failures inside handlePushes, for autosettings.MAX_HG_RETRIES tries. If configured, it forwards the failures to a sentry installation.