MediaWiki

Content translation/Deployments/Deployment Plan/Archive4

< Content translation | Deployments | Deployment Plan
(Redirected from Content translation/Deployment Plan/Archive4)

CX Deployment Plan for January 2015

edit
  • Project: Content Translation
  • Deployment date: Week of January 12, 2015
  • Initial Release: Released in Beta Labs environment (July 2014)
  • Past Releases: See: Archives
  • What is targeted for January 2015 release:
Content Translation 0.03 release: https://www.mediawiki.org/wiki/Content_translation/Roadmap#Ongoing_Release:_Content_Translation_0.03_release
  • Long-term project roadmap:
https://www.mediawiki.org/wiki/Content_translation/Roadmap
  • Language Pairs to be supported:
Target Wikipedia Sources (MT) Sources (without MT)
Catalan Spanish, Portuguese, English -
Spanish Catalan, Portuguese, English -
Portuguese Catalan, Spanish English
Danish - English, Swedish
Esperanto English -
Malay Indonesian English
Indonesian Malay English
Norwegian (Bokmål) Norwegian Nynorsk English
  • List of wikis to deploy Content Translation
    • cawiki
    • dawiki
    • eowiki
    • eswiki
    • idwiki
    • mswiki
    • nnwiki
    • nowiki
    • ptwiki
    • svwiki
  • Release as: Beta Feature

Overall Plan

edit

This release is targeted to Production with 0.03 feature set.

System Architecture

edit

See:

https://www.mediawiki.org/wiki/Content_translation/Technical_Architecture
https://www.mediawiki.org/wiki/Content_translation#Workflow_and_Technical_Architecture
https://www.mediawiki.org/wiki/Content_translation

Caching Architecture

edit

The following diagram includes the caching requirements for the CX framework:

https://www.mediawiki.org/wiki/Content_translation/Server_communications_workflow
https://commons.wikimedia.org/wiki/File:CX_ArchitectureV1.svg

Components to be provisioned for production

edit
cxserver
edit

Content Translation service is puppetize and available in operations/puppet repository as "cxserver" module.

Primary setup instructions are at: https://git.wikimedia.org/markdown/mediawiki%2Fservices%2Fcxserver.git/HEAD/README.md

For detailed information about component, installation and configuation and instructions: https://www.mediawiki.org/wiki/Content_translation/Setup

Apertium
edit

Apertium service is puppetize and available in operations/puppet repository as "apertium" module.

Apertium service installation and configuration: https://www.mediawiki.org/wiki/Content_translation/Apertium/Service

Node.js
edit
Dictd
edit

No additional configuration needed for the planned languages. In future, when we enable more languages, dictd based dictionaries or external dictionaries may get introduced- but not planned right now. See: https://www.mediawiki.org/wiki/Content_translation/Dictionaries for details.

Backend Services
edit
Extension dependencies
edit
  • BetaFeatures
  • CLDR
  • EventLogging
  • GuidedTour
External APIs called by CX
edit
  • Wikidata
  • Parsoid API
Configuration Scripts
edit

Provisioning Plan

edit

a. Storage and Hardware Requirements

Service cluster A.
  • cxserver service hardware is ready. For ContentTranslation hardware measurement, we monitored Beta cluster at,
https://tools.wmflabs.org/nagf/?project=deployment-prep#h_deployment-cxserver03
  • Apertium service hardware is ready. For Apertium hardware measurement, we monitored Beta cluster at,
https://tools.wmflabs.org/nagf/?project=deployment-prep#h_deployment-apertium01

b. Bandwidth Requirements

Service cluster A.
Tech Ops has indicated that cxserver/apertium instances will go into "Service cluster A" (ie http://ganglia.wikimedia.org/latest/?r=hour&cs=&ce=&c=Service+Cluster+A+eqiad&h=&tab=m&vn=&hide-hf=false&m=cpu_report&sh=1&z=small&hc=4&host_regex=&max_graphs=0&s=by+name )

d. Performance expectations

See, https://www.mediawiki.org/wiki/Content_translation/Performance

Monitoring and metrics

edit

LE Team responsibilities

edit
  • Kartik - Deployment, Engineer
  • Niklas - Engineer, Code Reviewer
  • Santhosh - Engineer, Code Reviewer
  • David - Engineer, Code Reviewer
  • Joel - Engineer, Code Reviewer
  • Runa - Team Scrum-Master / testing and communications
  • Pau - Feature UX reviewer, designer
  • Amir - Feature signoff

Timeline

edit
Date Milestone People Involved Status
December 12, 2014 Beta-Feature Sign Off Amir, Greg, JamesF   Done
December 12, 2014 Hardware Availability for CX and other services Tech Ops, Kartik (LE)   Done
Within 7 days of HW procurement Puppet config preparation for cxserver and apertium service Kartik (LE)   Done
Within 10 days of HW procurement Database deployment Kartik (LE), Tech Ops   Done
January 5-8, 2015 MediaWiki config and CX extension deployment Kartik (LE), Niklas, Ops, Greg, <any others>   Done
January 5-12, 2015 Beta Feature security review Chris S/Joel S   Done
Week of January 12, 2015 Deployment as beta-feature completed on 8 target Wikipedias Kartik (LE), <any others>   Done
January 13-16, 2015 Beta-feature testing period LE team, <any others>   Done

Current Status

edit

CX Deployment Plan for January 2015

Done

edit
  • Hardware Availability for CX and other services.   Done
    • Started: 2014-11-10
    • cxserver hardware.
      • Ready as per Ops/Alex. Planned to host on Service Cluster A.
    • Apertium hardware from Ops.
      • Apertium instance is ready to go with Service Cluster A as per Alex/Kartik's estimation of CPU/RAM monitoring for Beta.
  • Varnish setup on Beta (and Production).
    • Started: 2014-11-19
    • Ticket: https://phabricator.wikimedia.org/T76200
    • Status:
      • Assigned to Alex.
      • deployment-sca-cache01.eqiad.wmflabs instance has been created (2014-11-25)   Done
      • Alex need to setup Virtual IP/instance to finalize VCL rules for Varnish on Beta.
  • Security review of Content Translation "Beta Feature"

In Progress

edit
  • Testing!

To be started

edit
  • None

Stalled

edit
  • None
Retrieved from "https://www.mediawiki.org/w/index.php?title=Content_translation/Deployments/Deployment_Plan/Archive4&oldid=3605650"