Wikimedia Labs/Shared home directories per project


A few possible approaches for this:

  1. create one instance that has access to shared storage (via volumes in gluster or ceph) and export home directories, restricting access to different sets of home directories by project to instances within that project in a few possible ways:
    1. Manage /etc/exports by searching LDAP for projects (&(objectclass=groupofnames)(owner=*)) and instances within that project (puppervar=instanceproject=<project>), sharing the project in the following way:
      /export/home/<project1> <project1-instance1>(rw,no_subtree_check) <project-instance2>(rw,no_subtree_check) <project_instance...>(rw,no_subtree_check)
      /export/home/<project2> <project2-instance1>(rw,no_subtree_check) <project2-instance2>(rw,no_subtree_check) <project2_instance...>(rw,no_subtree_check)
    2. Write an mcollective script that updates /etc/exports using searches which return the information for all instances in a specific project, sharing the project the same way as 1.1.
    3. Add netgroup support to OpenStackManager or to nova-api as an extension. When an instance is created/deleted the instance should be added/removed to/from the project's netgroup. When a project is created/deleted, the netgroup itself should be created/deleted. /etc/exports would need to be updated when netgroups or projects were added or removed, and would be configured like so:
      /export/home/<project1> @<project1-netgroup>(rw,no_subtree_check)
      /export/home/<project2> @<project2-netgroup>(rw,no_subtree_check)
    4. Create a user for doing nova queries, and open firewall access for the nova-api for the instance. Create /etc/exports via queries for projects and instances within projects. The format for /etc/exports would be like 1.1. Trigger via cron or mcollective through OpenStackManager.
    5. Investigate if possible: Add nisnetgroup objectclass to project objects and create a collective attribute on instance objects for the nisnetgrouptriple attribute that will get applied to objects in ou=hosts with a filter of (cn=<project>). The nisnetgrouptriple collective attribute should have the form (<instance-hostname>,,). The conflict-behavior should be merge-real-and-virtual. /etc/exports would need to be updated when netgroups or projects were added or removed, and the format would be like 1.3.
      • groupOfNames and nisNetgroup are structural objectclasses; it isn't possible to have both objectclasses on a single object
      • Collective attributes only work on subtrees. ou=hosts and ou=groups are both on the base, so it isn't possible.
    6. Add a nova-api extension to manage shares via an ssh command (add-to-project/remove-from-project) or that adds messages to a queue, which a daemon on the nfs server will pull.
  2. Add a nova-api extension to manage netapp shares. When instances are created/deleted, update the exports on the netapps for specific projects to instances in those projects. When projects are created/deleted, the extension should add/remove qtrees and shares.

In the above solutions, if a nova-api extension is made, it should work for either netapp, or an instance. The instance is likely preferable over the netapp.

A nova-api extension is likely the best solution from a compatibility point of view. Calling the nova-api from the instance is likely a good second option, for compatibility reasons. A collective attribute with netgrouptriples is likely the easiest solution overall, but depends heavily on the likely already obsolete way of handling projects in nova releases essex and greater.