Kris has written an article that discusses the new PBI format. It appears on pp 16-17 of the April issue of BSD Mag and is reprinted here with permission.
The updated PBI format offers many new features such as digital signing, binary diff patching, and repository management, while preserving the original goals of format, by allowing packages to be installed and run in a self-contained manner. Lets first
take a look at how the PBI format differs from traditional package management, and then explore the changes in the upcoming 9.0 release.
The biggest difference between applications packaged in the PBI format vs traditional FreeBSD packages or RPMs is that each archive contains a complete set of libraries and dependent data required for for the target software to function. This means in effect, that a PBI is self-contained, and doesn’t require changing system packages to load the appropriate dependencies. A PBI file can be installed or removed at will, without effecting other applications or running the risk of causing breakage elsewhere in
the process. With the re-implementation of the format for the upcoming PC_BSD 9, this core concept has remained and been expanded upon.
One of the first major changes for the next PBI format has been the addition of library and file sharing between PBIs. In the previous implementation of the PBI format it was common that identical files existed between various applications. These duplicates, while necessary to provide self-contained functionality, still wasted both disk and runtime memory space. In the new format this waste of space has been greatly reduced through the usage of what has been dubbed the hash-dir. In this directory, libraries and common files are able to be shared between various PBIs through a system of hard-links. When an identical file is found in a PBI, the original will be removed and a hard-link created to the copy already in the hash-dir. After a PBI has been removed, any unneeded files left in the hash-dir are cleaned up by the pbid daemon, which monitors and maintains the integrity of the shared files.
Another new feature is the ability for administrators and PBI builders to create and manage their own repositories of PBIs. This repository system provides a number of tools for PBI distribution, release management and more. For the end user, tools are now provided which allow the browsing of PBIs in a repository, enabling auto-updates and mirror configuration. Tied into the new repository system is a new feature to digitally sign PBI files, and verify their contents. When a distributor creates a new repository, it includes an openssl public key file which is installed on the end users system. When the distributor runs a PBI build process, the resulting PBI file includes several signatures for the content archive and installation and removal scripts. During the PBI installation process, these signatures are checked to confirm that the archive has not been tampered with during transit. This key file is also used to associate a particular PBI with a parent repository for upgrade purposes, since it is possible that multiple repositories will have the same applications.
Since the very nature of self-contained packages tends to produce larger installation files, one thing which needed improvement was the updating process. In the previous implementation, updating an installed PBI would require the re-downloading of the entire installation archive. For larger applications this could be a rather time-consuming process, especially for users running over low bandwidth connections. In the typical application version update, only a small percentage of files had actually been changed, and a majority of these may simply be building time-stamps. In order to solve this problem the new PBI specification now supports updating via binary diff patches. Distributors running PBI builds can now enable options to generate small PBP (Push Button Patch) files, which are often a fraction of the size; in some cases less than 5% of the original PBI archive. When the end user begins the update of a PBI file from a repository, it automatically checks for for the presence of a PBP file, and attempts to use it, only falling back to the original archive should the process fail.
A more recent addition to the new PBI format is the ability for applications to be installed by user (non-root) accounts. Since each application is entirely self-contained, and doesn’t require changing around other installed packages, it became very easy to implement functionality for user-installation. By default the PBI format allows users apart of the systems operator group to perform installations and upgrades. This allows enhanced security in office or home situations, where users can now add/remove software on their desktop without being able to use the root account.
All these new features in the PBI format for PC-BSD 9.0 have already made it far superior to the existing legacy format. For traditional FreeBSD users though, perhaps the most important new feature is the implementation. The previous PBI incarnation was developed in QT/KDE C++, which made running on native FreeBSD cumbersome, especially on a system which had no need for X11 installed. The new format is implemented 100% in shell, and is able to run on traditional FreeBSD entirely from the command-line. The various functionality is broken up into 15+ command-line utilities with man-pages for each, which makes native FreeBSD usage a natural fit.
We’ve just taken a brief look at this reimplemented PBI format, and some of the new features it offers. This format will ship as the default for PC-BSD 9.0 and beyond and is currently available for beta testing in our PC-BSD 9-CURRENT snapshots. Once out of beta, it will be available to install on traditional FreeBSD via the ports system.
On Saturday, April 9, Festival Latinoamericano de Instalación de Software Libre (FLISOL) will be hosting a virtual conference (in Spanish). Details for joining the conference using streaming media, Second Life, and IRC are at the FLISOL website.
Sergio Ligregni will be giving a presentation on PC-BSD and a link to his presentation video will be posted here once it is available. In the mean time, here is the abstract for the presentation and a bio for Sergio:
This presentation explains to non-Open Source users the PC-BSD operating system, with a little demo of its easy install. It is meant to be more a video to encourage people to use PC-BSD than a tutorial, talking about a little history, the advantages of using it (making special mention of students and casual computer users, and also strong computer users like system administrators) and some project info, i.e. the video tells that if you are about to start a Computer Science major, you can contact the project and get a lot of learning from it, contributing with its development. The video will be presented at FLISOL Virtual (Latin American Free and Open Source Software Installation Festival, virtual mode). This video mainly express the idea: “PC-BSD is ready for you, you just have to use it!”.
Sergio Ligregni, Computer Science Engineer, lives in Guadalajara, Jalisco, Mexico. Currently he works as C++ Developer Architect for IBM, previous experience includes two years in C++ app development and one year in tech support and testing. He participated in Google Summer of Code 2010 with the Distributed Audit project (about to be implemented). He taught during one period at Queretaro’s Technological Institute a Programming Fundamentals class where he spread the word about FOSS, particularly PC-BSD.
A new 9-CURRENT snapshot is now available from our ftp site for both 32 and 64 bit systems.
Included in this snapshot:
- KDE 4.6.1
- Fixes for installing across multiple drives with gmirror
- Improvements to the networking GUI, able to run without prompting for root passwords
- Fixes to the AppCafe, improvements for installing apps as a user account
- Bugfixes to the pbi_* commands
- There is still a bug with using disk-encryption from the installer which will be fixed soon.
Testing snapshots are ALPHA quality software, meaning that you should test them on a test system or virtual environment. Expect issues and bugs and please report any issues back to the testing mailing list.
Sam Lin, a PC-BSD user who is active on the testing and translation lists, was recently interviewed by Chris Gardner, Farming and Technology Editor at the Waikato Times newspaper in New Zealand. His interview appeared on page C7 in the Technology section on February 5. The contents of the interview are posted here with permission:
A Waikato computer club whose members promote the use of the free Linux operating system are being told to broaden their horizons or fold.
The Waikato University-based Waikato Linux Users Group (WLUG) was founded in 1998 for users of the alternative to Microsoft’s Windows operating system, flavours of which are installed on more than 90 per cent of the world’s personal computers, and Apple’s iOS. Membership peaked at about 50, but has fallen in recent years to a 10th of that as it struggles to get enough together to hold a committee meeting.
Dr Shih-Min “Sam” Lin, of Waikato University’s honey research unit, who recently joined the committee, said the group almost folded last year as the half a dozen committee members were reluctant to take leadership roles.
“There should be much more members than this in the user group,” he said. Dr Lin said WLUG had not been close enough to the mainstream user and was unorganised. WLUG’s monthly meetings on topics like macro programming for Open Office were “quite geeky or techy for the mainstream user”, he said. “It is doubtful if the community would last long without making any change.”
Dr Lin, who uses another free operating system called PC-BSD, said it was easy to find books on paid-for software, but there was little available in the free software market although there was plenty of support online. “This is where I think the user group can cut into the general public. If in the monthly meetings some useful user-firendly tutorials can be given, people would be intrigued and be willing to come. More people, more interaction, more discussion, more vitality and more fun.”
Dr Lin said the free software community was open to everyone, not just computer geeks, and modifying software to suit personal needs was sometimes as easy as adding a few words to a text file. “As a biological scientist I analyse my data of the research on honey with a Free License Open Source Software (FLOSS) called R, which is an internationally well-known statistical package of which one of the original developers is a New Zealander. I typeset my thesis and long document with a FLOSS called LaTeX which is very common among computer science/mathematical/statistical fields. I use GIMP to edit my photos, and I use Firefox to hunt on Trade Me.”
The firefox PBI has just been updated to 4.0.1. If the firefox PBI is already installed on your system, you can update to version 4 using Software Manager. If it is not installed, you can also install it through Software Manager.