Wikimedia Apps/Team/RESTBase services for apps/Deployment process/Setup notes
Here are the detailed notes to get ready to update the deployment repo for our mobileapps RESTBase service on a Ubuntu box setup (inside a VirtualBox VM).
Most also applies for setting up directly on a MacOS machine using Docker for Mac.
Additional Software on Ubuntu
editsudo apt-get update
sudo apt-get upgrade
sudo apt-get install curl git nodejs nodejs-legacy npm git-review
Git setup
editBasic Gerrit setup
editThis is more a reminder in case you setup a new machine, as I was doing that initially in a fresh Ubuntu install inside VirtualBox.
See Gerrit/Tutorial#Set_Up_SSH_Keys_in_Gerrit.
Setup source and deploy repos
editNote that you have to set an environment variable here for the setup only (temporarily, no need to stick that into an rc script):
GERRIT_USER=
BASE_DIR=~/code/mcs
SRC_DIR=${BASE_DIR}/mobileapps
DEPLOY_DIR=${BASE_DIR}/deploy
mkdir -p ${BASE_DIR}
cd ${BASE_DIR}
git clone ssh://${GERRIT_USER}@gerrit.wikimedia.org:29418/mediawiki/services/mobileapps
git clone --recursive ssh://${GERRIT_USER}@gerrit.wikimedia.org:29418/mediawiki/services/mobileapps/deploy
cd ${DEPLOY_DIR}
git submodule update --init
git review -s
cd ${SRC_DIR}
git review -s
git config deploy.dir ${DEPLOY_DIR}
git config deploy.name mobileapps
Git user name and email
editYou may also need to set your email and name in git if you haven't done so. See the Gerrit/Tutorial#Set_your_username_and_email.
Git tag setup
editGit tags are cryptographically signed and a GPG signing key is therefore required.
gpg --list-keys
(or get GPG if you need it)
If no keys are listed or you just installed GPG, create a new signing key with:
gpg --gen-key
Then configure the new key (or desired existing key) as the signing key for the mobileapps repo:
git config user.signingkey <keyId>
(add the --global
flag if you want to use this signing key for all repos)
For more information straight from the source: https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work
Docker
editOn MacOS
editIf running on Mac OS you can install Docker for Mac.
On Ubuntu
editThe easiest way to install Docker:
curl https://get.docker.com/ > install-docker.sh
sudo bash install-docker.sh
Then, you need to add your user to the docker
group, follow the note at the end of the docker commands to do that. In other words:
sudo gpasswd -a $USER docker
After all of that, reboot your VM. Then, you can run the tests inside Docker:
cd ~/code/mcs/mobileapps
./server.js docker-test --verbose
Optional: Nagios Setup
editChecking the service with Nagios before building the deploy repo can help to identify problems and prevent deployment reverts.
(Note: These instructions are only the minimum required for command line usage, adapted from the Nagios Ubuntu Quickstart guide.)
Install prereqs:
sudo apt-get install apache2 libapache2-mod-php5 build-essential libgd2-xpm-dev
Create a Nagios user:
sudo -s
cd /usr/sbin
useradd -m -s /bin/bash nagios
passwd nagios #any password is fine
groupadd nagios
usermod -G nagios nagios
groupadd nagcmd
usermod -a -G nagcmd nagios
usermod -a -G nagcmd www-data
exit
Download, build, and install Nagios:
wget http://prdownloads.sourceforge.net/sourceforge/nagios/nagios-3.2.3.tar.gz
tar xzf nagios-3.2.3.tar.gz
cd nagios-3.2.3
./configure --with-command-group=nagcmd
make all
sudo -s
make install
make install-init
make install-config
make install-commandmode
exit
Download, build, and install nagios-plugin:
wget http://www.nagios-plugins.org/download/nagios-plugins-2.1.1.tar.gz
tar xzf nagios-plugins-1.4.11.tar.gz
cd nagios-plugins-2.1.1
./configure --with-nagios-user=nagios --with-nagios-group=nagios
make
sudo make install
Start Nagios:
sudo -s
ln -s /etc/init.d/nagios /etc/rcS.d/S99nagios
/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
/etc/init.d/nagios start
exit
Download the WMF service checker plugin:
wget https://raw.githubusercontent.com/wikimedia/operations-puppet/production/modules/service/files/checker.py
Now, to run the endpoint checks, start the service, then do:
python ./checker.py 127.0.0.1 http://localhost:6927
Once that is all working you can prepare the deploy repo as part of the deployment process.