Infrastructure-as-code migration
Posted on Oct 02, 2023 by F-DroidWe just completed migrating one more key piece of our infrastructure to a completely automated setup: the “originserver”. The originserver is the holder of all of the public files for the website and the f-droid.org repository. It is also responsible for pushing out the updates to all of the primary mirrors. So it is safe to say that the originserver is central to our operations. Ciaran Gultnieks, the founder of the F-Droid project, has been quietly and dutifully maintaining the originserver since he first set it up more than a decade ago. The F-Droid users and community owe him a debt of gratitude for so many years of faithful service, keeping things going through thick and thin.
The main reason why we undertook this migration is to improve our “bus factor”. F-Droid relies on contributors to keep it running, but no single person should be solely responsible for any key piece of infrastructure. We are still a small team, so we focused on migrating this central server to be “Infrastructure as code”. Specifically, the whole central web and mirroring setup is now maintained in Ansible projects in the F-Droid GitLab group. Maintaining servers as code has a number of key advantages:
- The servers can be easily recreated by anyone who can run Ansible.
- Automated server config makes it feasible to create highly detailed, custom security sandboxes.
- Temporary instances making auditing easy, anyone with access to the code can test.
This provides a huge “bus factor” improvement. Now anyone with access to the code can recreate a copy of our infrastructure. The F-Droid board controls the domain names, so in case of emergency, the board can direct the domain name to point to new infrastructure. Since F-Droid is all about free software, there are very few things that need to be kept secret, basically, just the cryptographic keys for signing, encryption, and access control. These keys can be regenerated and set up where they are needed (e.g. mirrors). The f-droid.org data is replicated across many mirrors around the world. And the f-droid.org website and repository comes from pulling in many public source code repositories to build the apps.
Last one: signing server
There is one last production server that is not maintained as Ansible code: the signing server. We do intend to eventually manage that machine as code as well. It is a quite simple setup, so converting it to Ansible is currently not high priority. It requires a bare bones Debian install with fdroidserver installed from the Git repository. We have pushed the rest of the required tools, like apksigner, to Debian, where they are maintained and available to all. If you are looking for a fun project to contribute, we still need an Ansible project for creating a standalone signing servers. Once we have that, we can iterate on it to build a very hardened setup.
Infrastructure as code
Here are the key servers that are now managed as code. Not all of these are public yet, we want to ensure they are carefully reviewed and audited before publishing them (let us know if you want to contribute auditing or pentesting, and we can set something up).
- buildserver: https://gitlab.com/fdroid/fdroid-bootstrap-buildserver/
- f-droid.org deployserver: https://gitlab.com/fdroid/fdroid-deployserver/
- f-droid.org: https://gitlab.com/fdroid/fdroid-http-fronters/
- fdroid.link: https://gitlab.com/fdroid/fdroid-http-fronters/
- fdroidstatus.org: https://gitlab.com/fdroid/fdroid-statuspage-deployment/
- forum.f-droid.org: https://f-droid.org/docs/Maintaining_the_Forum/
- IPFS pinning server: https://gitlab.com/fdroid/ipfs-pinning-server
- matrix.f-droid.org: https://gitlab.com/fdroid/wiki/-/wikis/Internal/Servers#matrixf-droidorg
- monitor.f-droid.org: https://gitlab.com/fdroid/fdroid-monitor-deployment/
- search.f-droid.org: https://gitlab.com/fdroid/fdroid-website-search-deployment/
The F-Droid community works to serve its users as openly and transparently as possible. This is a big step to making F-Droid ever more accountable to all of you who trust us to provide the apps you need.