Administration through subversion
I would like to come up with an administration method with the following properties:
- every change to the system's state is kept under revision control;
- changes are done primarily by editing files in the repository, and are pulled from there by the running system;
- deamons are automatically restarted to take the latest changes into account;
- dysfunctionnal modifications are automatically rolled back;
- noone should ever have to log in as root, except maybe for triggering an update.
The goal is to ease collective administration or restoring from backups, and improve traceability. This method would be based (at least initially) upon subversion and make, since those are the things I'm confortable with. I'm targeting Debian systems first for the same reason.
Subversion
The root filesystem would be a subversion working copy.
Only the modified parts of it would be checked in the repository
(mostly some configuration files).
The svnroot wrapper would be used for updating it,
so as to set file owners and permissions in a proper way,
and trigger some post-update scripts.
The administors would use svnroot
as well,
so as not to have inappropriately world-readable files
in their working copies.
Of course you would get all the goodies of revision control, including commit mails to a mailing list of administrators. Some mechanism to encrypt those would be needed since the diffs would be prone to containing sensitive information.
Post-update scripts
After an update, svnroot would run a post-update script.
This script would reload all deamons affected by
the newly checked out changes, as well as run some tests.
Most probably this would be done with a few make
rules
and a set of stamp files.
Should the post-update script fail,
svnroot
would revert the system to the previously
used revision and get it back running.
Everything as configuration files
I will need a set of tools to store the aspects of system
configuration which are not already there as configuration files.
Most notably, I'm planning to write a "retrograde package tool",
which would read a set of wanted package lists from /etc/rpt/pkgs.d
and use apt and deborphan to synchronize the list of installed packages
accordingly.
On the debconf
side, using a DirTree
database and checking
parts of it (ie. the questions for which the default answer
was modified) into the subversion repository would be a solution.
Local installations
Locally compiled and installed packages (ie., in /usr/local
)
could be installed through the repository, using fakeroot or something.
Possibly some support would be needed in svnroot
to ease
creating the needed permission table entries.
Backups
Once this method is in place, you would only need to backup your repository as
far as system administration goes. Restoring it would be a matter of
svnroot checkout
.
One could export data they want in the repository
(such as the contents of an LDAP directory, as LDIF)
to /var/backups
and commit that on a daily basis.
This would offer the extra interest of giving the administrators
an opportunity to scrutinize the changes in such data.
Problems
Most probably there would still be some aspects of system admininstration
which could not be converted to configuration files. Hopefully this might
be dealt with with post-update scripts for svnroot
,
but poeple will still need root access for some activities,
such as testing or investigating problems.
This could be worked around with sudo
logging:
one could make sudo run a shell script after its message has been logged,
and before the command is actually invoked,
which would ensure the log file to be successfully commited
and the post-commit mails have been sent out,
thereby avoiding root commands to be executed without leaving a trace.
While this still breaks the "every state change comes from the repository"
rule, the logfile could be easily studied afterwards,
for instance when a backup is to be restored,
and fellow administrators would still be informed that
an extra root command was needed.
Random notes
The mr and etckeeper tools from Joey Hess may prove useful.
isconf may also be worth looking into.