Sysadmin Meeting Minutes

Participants

kallisti5, waddlesplash, nielx

Date

8 October 2017

Location

#haiku-dev on Freenode

Decisions

The following decisions were made:

• All essential services will be moved to maui
• Baron will be decommissioned when that process has finished
• The team will first focus on moving/retiring the services on vmrepo. With those lessons learned the rest of the plan can be made.

Action Points

The following tasks were created and assigned:

• nielx: Investigate the best way to share critical passwords
• kallisti5: Investigate moving git and cgit to maui, in combination with gerrit
• nielx: Investigate haikudepot and work on moving it
• nielx: investigate the svn and hg repositories on vmrepo and see if they need to move

Minutes

Current plans

• Move essential services to the new maui server
• Decommission baron and return the server to Hertzner
• The new server will primarily use Docker containers to host the services

Maui

• This is a beefy new server to replace baron.

Server Specs

• Hosted by Hetzner (like baron)
• 64 GB of RAM (was 16 GB)
• 4 TiB in a RAID 1 mirror
• More CPUs than baron
• Runs on Fedora server
• Setup with SELinux in enforcing mode

Documentation

• kallisti5 did the initial configuration

Access to maui

• Current access is for axeld, jessicah, pulkomandy, kallisti5, waddlesplash, nielx
• Though pulkomandy has indicated that after initial setup he does not want to be a permanent member of the system administration team

Server configuration: Puppet

• User administration is done using Puppet ​https://www.lennu.net/puppet-manifest-examples
• In the future it could also be used for firewall rules, config files, etc.
• Puppet serves both as configurator, and as documentation (for when a server needs to be rebuild)

Security/Sysadmin team

• Question is how do we share critical and sensitive information
• The approach was to store them on baron for root users
• Major downside: what do you do when baron is down
• Alternatives:
  • Share this data in the haiku-sysadmin team on Keybase (decentralized encrypted communication and fileshare)
  • Use the ​Dashlane service
• nielx will do some investigating and make a proposal about the best solution on haiku-sysadmin

Containers

• kallisti5 has an overview of how infrastructure should use in his opinion ​https://github.com/haiku/infrastructure/blob/master/docs/haiku-infrastructure.png
• Advantages of containers: all the benefits of VMs, none of the performance hits
• Fewer/none VMs also mean less maintenance
• Docker will be used, the configuration will be stored, shared and maintained on ​https://github.com/haiku/infrastructure
• kallisti5 and jessicah are the resident Docker experts

Challenges of using Docker for packaging infrastructure

• The largest challenges are with the whole package building infrastructure, because:
  • It has several ties with git hooks and git repositories and it relies on 'internal' communication
  • By nature of that it relies on external processes it is quite something to 'contain' in one container
• kallisti5 has taken some steps to look into it, but the difficulty is that the setup is still a moving target

Why Docker?

• Docker is a tool that helps to create, maintain and deploy containers.
• They are exactly exactly that: a contained set of software, that is connected to contained data volumes.
• In theory they are highly portable, one might just pick a container and its data up and deploy elsewhere.
• It also means that the actual software deployments are documented in the Docker scripts
• Docker-compose will be used to deploy containers to maui. Example for gerrit: ​https://github.com/haiku/infrastructure/blob/master/docker/gerrit/docker-compose.yml
  • Docker-compose runs all the docker commands on the server to deploy
• nielx informed about the relation with the Dockerfile: kallisti5 explains that this is the blueprint to build an image, and docker-compose is about deploying that image
• More on Docker: ​https://docs.docker.com/engine/docker-overview/

Migration Timeline

• It seems that vmrepo is ready to migrate within weeks
• The most important services are git and cgit
  • kallisti5 has been working on those, in combination with Gerrit
  • nielx asks if it is an option to first move over the git and cgit services as is, and later add/attach gerrit
  • kallisti5` will investigate that

Haikudepot on vmrepo

• nielx wonders about the state of Haikudepot
• waddlesplash notes that it goes out of memory often
• Can be found on github: ​https://github.com/aplgithub/haikudepotserver
• nielx will contact the maintainer Andrew Lindesay

SVN repositories on vmrepo

• There are still two repositories hosted on ​http://svn.haiku-os.org/
• Both are out of use
• nielx will have a look to see whether they need to be transferred over, or if they are somewhere permanently stored
• nielx will also look at the mercurial repositories hosted there.

Maintaining Baron

• The maui maintainers will do essential maintenance on baron
• It has been decided that there will be no major changes to baron, but instead to focus on moving services