Package Management for the Upcoming PC-BSD 9
Kris has written an article that discusses the new PBI format. It appears on pp 16–17 of the April issue of BSD Mag and is reprinted here with permission.
The updated PBI format offers many new features such as digital signing, binary diff patching, and repository management, while preserving the original goals of format, by allowing packages to be installed and run in a self-contained manner. Lets first
take a look at how the PBI format differs from traditional package management, and then explore the changes in the upcoming 9.0 release.
The biggest difference between applications packaged in the PBI format vs traditional FreeBSD packages or RPMs is that each archive contains a complete set of libraries and dependent data required for for the target software to function. This means in effect, that a PBI is self-contained, and doesn’t require changing system packages to load the appropriate dependencies. A PBI file can be installed or removed at will, without effecting other applications or running the risk of causing breakage elsewhere in
the process. With the re-implementation of the format for the upcoming PC_BSD 9, this core concept has remained and been expanded upon.
One of the first major changes for the next PBI format has been the addition of library and file sharing between PBIs. In the previous implementation of the PBI format it was common that identical files existed between various applications. These duplicates, while necessary to provide self-contained functionality, still wasted both disk and runtime memory space. In the new format this waste of space has been greatly reduced through the usage of what has been dubbed the hash-dir. In this directory, libraries and common files are able to be shared between various PBIs through a system of hard-links. When an identical file is found in a PBI, the original will be removed and a hard-link created to the copy already in the hash-dir. After a PBI has been removed, any unneeded files left in the hash-dir are cleaned up by the pbid daemon, which monitors and maintains the integrity of the shared files.
Another new feature is the ability for administrators and PBI builders to create and manage their own repositories of PBIs. This repository system provides a number of tools for PBI distribution, release management and more. For the end user, tools are now provided which allow the browsing of PBIs in a repository, enabling auto-updates and mirror configuration. Tied into the new repository system is a new feature to digitally sign PBI files, and verify their contents. When a distributor creates a new repository, it includes an openssl public key file which is installed on the end users system. When the distributor runs a PBI build process, the resulting PBI file includes several signatures for the content archive and installation and removal scripts. During the PBI installation process, these signatures are checked to confirm that the archive has not been tampered with during transit. This key file is also used to associate a particular PBI with a parent repository for upgrade purposes, since it is possible that multiple repositories will have the same applications.
Since the very nature of self-contained packages tends to produce larger installation files, one thing which needed improvement was the updating process. In the previous implementation, updating an installed PBI would require the re-downloading of the entire installation archive. For larger applications this could be a rather time-consuming process, especially for users running over low bandwidth connections. In the typical application version update, only a small percentage of files had actually been changed, and a majority of these may simply be building time-stamps. In order to solve this problem the new PBI specification now supports updating via binary diff patches. Distributors running PBI builds can now enable options to generate small PBP (Push Button Patch) files, which are often a fraction of the size; in some cases less than 5% of the original PBI archive. When the end user begins the update of a PBI file from a repository, it automatically checks for for the presence of a PBP file, and attempts to use it, only falling back to the original archive should the process fail.
A more recent addition to the new PBI format is the ability for applications to be installed by user (non-root) accounts. Since each application is entirely self-contained, and doesn’t require changing around other installed packages, it became very easy to implement functionality for user-installation. By default the PBI format allows users apart of the systems operator group to perform installations and upgrades. This allows enhanced security in office or home situations, where users can now add/remove software on their desktop without being able to use the root account.
All these new features in the PBI format for PC-BSD 9.0 have already made it far superior to the existing legacy format. For traditional FreeBSD users though, perhaps the most important new feature is the implementation. The previous PBI incarnation was developed in QT/KDE C++, which made running on native FreeBSD cumbersome, especially on a system which had no need for X11 installed. The new format is implemented 100% in shell, and is able to run on traditional FreeBSD entirely from the command-line. The various functionality is broken up into 15+ command-line utilities with man-pages for each, which makes native FreeBSD usage a natural fit.
We’ve just taken a brief look at this reimplemented PBI format, and some of the new features it offers. This format will ship as the default for PC-BSD 9.0 and beyond and is currently available for beta testing in our PC-BSD 9-CURRENT snapshots. Once out of beta, it will be available to install on traditional FreeBSD via the ports system.
Trackback from your site.