Content translation/Apertium/Deployment
This document describes deployment procedure Apertium in production, which is almost similar to cxserver now.
Testing
editNote image tag version from Gerrit patchset to be deploy. For eg: 2020-12-01-140153-production
.
Run:
docker pull docker-registry.wikimedia.org/wikimedia/mediawiki-services-apertium:2020-12-01-140153-production
to get Apertium docker image, which can be tested locally.
TODO: Add: How to test it with cxserver.
Updating packages
edit- Clone https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/services/apertium/
- Edit or update package list at:
.pipeline/blubber.yaml
Config
editApertium on Production config stays in: helmfile.d/services/apertium/values.yaml
Once list of packages is changed, we need to do deploy new image in production. Config change can be directly deployed without updating image.
Deployment
edit- Clone deployment-charts repository (for first time).
- Do needful changes in config (update image or other configuration changes as needed).
- Make a CR (Example: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/623475) and after a successful review, merge it.
- After merge, log in in a deployment server (eg: deploy1001), there is a cron (1 minute) that will update the /srv/deployment-charts directory with the contents from git.
- Go to /srv/deployment-charts/helmfile.d/services/apertium.
- Execute:
helmfile -e ${CLUSTER} diff
This will show the changes that it will be applied on the cluster. - Execute:
helmfile -e ${CLUSTER} -i apply
This will materialize the previous diff in the cluster and also will log into SAL the change.
Status
editThis is done using helmfile:
- Change directory to /srv/deployment-charts/helmfile.d/services/${CLUSTER}/apertium on a deployment server
- Unless you are mid un-applied changes the current values files should reflect the deployed values
- You can check for un-applied changes with:
helmfile -e ${CLUSTER} diff
- You can see the status with
helmfile -e ${CLUSTER} status
Logs
editLogs can be access from deploy1001 if needed:
cd /srv/deployment-charts/helmfile.d/services/${CLUSTER}/apertium
source .hfenv
kubectl logs apertium-production-8649f66884-znfgl
apertium-production-8649f66884-znfgl is pod name.
To see all logs:
kubectl logs -l app=apertium -c apertium-production
Rolling back changes
editIf you need to roll back a change because something went wrong:
- Revert the git commit to the deployment-charts repo
- Merge the revert (with review if needed)
- Wait one minute for the cron job to pull the change to the deployment server
- Change directory to /srv/deployment-charts/helmfile.d/services/apertium
- Execute
helmfile -e ${CLUSTER} diff
to see what you'll be changing. - Execute
helmfile -e ${CLUSTER} -i apply
where CLUSTER is one of (staging, eqiad, codfw).
See also
edit- https://gerrit.wikimedia.org/g/operations/deployment-charts
- `helm` can be install using methods described at: https://helm.sh/docs/intro/install/
- Deploying with helmfile
- List of apertium Production tags
- apertium Grafana dashboard
- Kubernetes at Wikitech
- Minikuke (ie Kubernetes on laptop)
- ContentTranslation MT configurations: https://www.mediawiki.org/wiki/Content_translation/Machine_Translation/Configuration