Meza/Importing data

Importing data from another wiki can be done in several ways.

Defining existing server to pull SQL and uploads from

secret.yml

The following lines are required in secret.yml. See Meza/Secret config for more info.

backups_server_alt_source:
  addr: example.com
  remote_user: src-meza-ansible
  uploads_dir_path: /opt/data-meza/uploads/<id>

backups_server_db_dump:
  addr: example.com
  remote_user: src-meza-ansible
  mysql_user: backupuser
  mysql_pass: "some123long456password7890"

hosts file

The following lines are required in your hosts file. See Meza/Secret config for more info.

[db-src]
example.com alt_remote_user=src-meza-ansible

[backup-src]
example.com alt_remote_user=src-meza-ansible

[exclude-all]
example.com

Setup backup server

1. Create user src-meza-ansible on backup server. Unfortunately, with current Meza, this user has to be a passwordless sudoer.
2. Create MySQL/MariaDB user backupuser with rights SELECT, INSERT, UPDATE, LOCK TABLES, SHOW DATABASES, SHOW VIEW. If the backup server is a Meza server, this can be achieved by adding the following to the mysql_users section of secret.yml:

mysql_users:
  - name: "backupuser"
    host: "%"
    password: "some123long456password7890"
    priv: "*.*:SELECT,INSERT,UPDATE,LOCK TABLES,SHOW DATABASES,SHOW VIEW"

Run meza deploy with --overwrite

Running sudo meza deploy <environment> --overwrite will overwrite all wikis with their corresponding backups. This is highly destructive!

Manually position files for import

When running sudo meza deploy <environment> --overwrite Meza will try to pull data from a backup source. The simplest backup source is the one that is generated when you do sudo meza backup <environment>. This generates the required backup directories for your existing wikis. Navigate to /opt/data-meza/backups/<environment> to see the structure. It looks something like:

/opt/data-meza/backups/<environment>
    demo
        20180414122334_wiki.sql
        20180413122334_wiki.sql
        20180412122334_wiki.sql
        uploads/
    anotherwiki
        ...
    anotherwiki2
        ...

If you run sudo meza backup <environment> then delete these SQL files, upload your own SQL file, and replace the uploads/ directory with your desired uploads, running sudo meza deploy <environment> --overwrite will import that data into the wiki. Note that all wikis will be overwritten with whatever they have as backups, so be careful how you use this!

Pushing database and uploads rather than pulling

Available since Meza 31.5.0

If you have a production server, and you want development servers to periodically synchronize with data from production, the "pull" method above works. However, if your security policy does not allow for development servers to SSH into production, then instead you can setup a push-from-production as outlined below. This (a) doesn't require any servers to log into production, (b) requires production to log into other servers with non-sudo access, and (c) this allows for pushing upload files directly to the uploads/ directory of the destination server rather than into the backups directory (removing duplicated directories which could be very large). For the purposes of this manual we'll assume you want to push from Prod to Dev, but any server could be setup to push to another server.

On Prod, configure secret config as follows:

backups_server_db_push:
  addr: example.com                                     # domain or IP address to push to
  remote_user: meza-push-user                           # username on remote server to login to
  sql_files_path: /opt/data-meza/backups/vagrant/<id>/  # where to put SQL files

backups_server_uploads_push:
  addr: example.com                             # domain or IP address to push to
  remote_user: meza-push-user                   # must be the same username as above
  uploads_dir_path: /opt/data-meza/uploads/<id> # where to put uploads

Here you define what servers to push data to, what user to push data with, and the exact location in the file system to push it to. In the paths listed, <id> will be replaced with each wiki's ID. While you're on Prod, copy the meza-ansible user's public SSH key for later user (i.e. copy the output of sudo cat /opt/conf-meza/users/meza-ansible/.ssh/id_rsa.pub). Then on Dev do the following to create meza-push-user:

# Add meza-push-user and put its home directory within meza config to avoid
# getting in the way of "real" users.
sudo useradd -m -d /opt/conf-meza/users/meza-push-user meza-push-user

# Add the meza-backup and apache groups to meza-push-user
sudo usermod -a -G meza-backups meza-push-user
sudo usermod -a -G apache meza-push-user

# become meza-push-user and setup SSH
sudo su meza-push-user
cd ~
mkdir .ssh
chmod 700 .ssh

# Edit .ssh/authorized_keys and add meza-ansible's public key from your Prod 
# server (id_rsa.pub you copied earlier)
vim .ssh/authorized_keys

# Then set permissions on authorized keys:
chmod 644 .ssh/authorized_keys

Now test that meza-ansible on Prod can ssh into Dev as meza-push-user: sudo su meza-ansible then ssh meza-push-user@your.Dev.server.address (you'll have to accept host key).

If you now run meza push-backup <your Prod environment> on Prod it will push data from Prod to Dev. In a later version of Meza a crontime field will be added allowing configuration of push-backup on regular intervals.

See also

• https://github.com/enterprisemediawiki/meza/pull/547