May
15

Weekly Feature Digest 29 — PBING

We’ve been seeing a lot of confusion and questions about the PBI changes that were recently pushed out those of you running the Edge package sets, and Ken Moore was nice enough to break the changes down in this week’s PC-BSD weekly digest.

First, a little history about the PBI system.
It was initially created when the only/primary application distribution method for FreeBSD was the ports system — meaning that any FreeBSD user who wanted frequent updates to their applications needed to manually compile/install any application through the FreeBSD ports tree on a fairly regular schedule. The PBI system was designed as an alternative to provide simple application packages that could easily be downloaded and installed without the need for the user to compile any source code at all. As an added benefit, the PBI system installed these applications into a seperate container on the system — leaving all the “complicated” system configuration and integration to still be run through the FreeBSD ports system. This allowed PC-BSD to have a stable base system for a release (because the base system packages would almost never get touched/updated), while at the same time provide the ability to keep the main end-user applications up to date between releases.

Now fast-forward a bit to the PC-BSD 10 series.
At this time the FreeBSD ports system, while still existing for the “hardcore” users, has mainly been replaced by the pkgng distribution system for general system/application usage. This has provided quite a bit of confusion for PC-BSD users, because they now had two different ways to install applications, and each application on the system would behave differently depending on how that particular application was installed. To make the distibution model simpler for PC-BSD, the PBI files were already being created from pkgng packages (ensuring that there was a lot less compiling done on the build servers), and those packages were simply being collected into “fat files” with a few compatibility scripts and such thrown in for good measure.This meant that there was a lot of duplication between the pkg and PBI systems, resulting in a lot of effort to maintain compatibility between the two systems. The main problem however, was that the special PBI runtime container itself was causing all sorts of system stability issues. Since the release of PC-BSD 10.0 we have actually tried 3 or 4 different types of application runtime containers, each of which was designed to solve a critical flaw in the previous version, but always kept running into large limitations/problems with each new type of container.

At this point we decided to take a step back and refocus on what the PBI system was originally intended to do — provide a “Push Button Installer” to install and run applications while keeping things as simple as possible for the end user. With this definition for the PBI system, it makes perfect sense that the pkgng system should be chosen  as our default application installation method for a couple reasons:
1) Integration with the system environment for things like setting up and running default applications works a lot better (mimetype integration/use).
2) Startup/runtime speed. Applications installed to the base system simply startup and run a lot faster than the ones that are installed into the containers.
3) User Confusion. Lots of people simply did not understand that the “contained” application libraries/files were not installed to the normal location on the system, and that an application in a container could not easily see or use the system-installed applications.

The next-generation PBI system.
This re-implementation is designed so that it no longer uses the “PBI Containers” exclusively and instead returns to its original goal — to provide a simple interface for the end user to install/use applications of all types and in all ways. This means that it is now a system that uses the pkgng packages as it’s basis — but provides all sorts of other information/functionality that the pkgng system does not fully support yet (such as mimetype integration, desktop/menu entries, and graphical information like icons for applications). Additionally, it also provides a number of enhancements to how the user can utilize the different pkgng packages, mainly through how the packages get installed.

1) Standard pkgng installation to the base system.
This allows the user a simple interface to install/remove application on the base system while providing a number of additional safety checks to prevent random “foot-shooting”.

2) Jail management.
By running the AppCafe on the base system, you can now manage all the applications/packages in any of the running jails on your system. Combined with the Warden for creating/managing different kinds of jails, the user now has a simple way to manage and run applications that (for security reasons) should never be installed/used from the base system (such as web servers or network-facing services).

3) Application containers with plugins!
By using the “portjail” creation options in the Warden, you now have a method to safely contain a graphical application while also allowing for a system of installing/removing optional packages into that jail for plugin support without touching your base system packages (very similar to our previous container system, but with a few more layers of separation between the jail and the system).

4) Other installation methods.
Because the PBI system is now installation-method agnostic (almost), we can provide support for alternate types of installation methods (such as into specialized containers like our previous PBI versions have had). While we do not have any other installation methods included at the moment, we can add new methods relatively easy in the future if those installation methods do not break system stability.

So what does this mean for a PC-BSD user?
1) Access to thousands more applications and plugins by default through the AppCafe. The “PBI” applications will show up with things like screenshots, available plugins, nice looking icons, user ratings/tips, and more while you also have the ability to install and use the “raw packages” (which will always have the icon of a box/package) even if the nicer recommendations and information is not available for that raw package.

2) Less confusion about application installations. Since applications will always be installed/integrated into the local system by default, this will prevent a lot of confusion in people who are used to the standard FreeBSD/Linux/Unix installation methods and file locations for applications.

3) Greater flexibility for different installation methods to suite your specific needs. System installation, traditional jail installation, portjail installation, additional future types of installations, it give the user freedom to truly run the system as you need, rather than forcing you to use a particular system that might not be what you were looking for.

Share This Post:
  • Digg
  • Facebook
  • Twitter
  • email
  • LinkedIn
  • Slashdot

Written by Josh Smith. Posted in Uncategorized

Trackback from your site.

Comments (27)

  • sg1efc
    May 15, 2014 at 3:21 pm |

    Thanks Josh, Ken and everyone else. From the above information, it sounds like these enhancements will have lots of great benefits for everyone. :)

  • rick
    May 15, 2014 at 4:07 pm |

    This means that, as a normal user, I should deinstall my PBI and proceed with the portjail method of warden. Is that right?

    In any case, this seems the right direction for me. PBI was starting to pose more problems than the ones it solves.

    On the other hand, it seems that the PCBSD packages are a bit out of date compared with what one sees in the freebsd ports web page. Any reason for this?

    • Ken Moore
      May 15, 2014 at 6:21 pm |

      rick » There is no need to deinstall your current PBI’s. The first time you run the new AppCafe (with the new PBI system in the backend), it will open up a migration utility which will convert all of your current PBI’s over to the new system for you.
      Since you don’t have the new system yet, that would mean that you are currently on the “Production” schedule for package updates — which means that you only get bulk updates to packages once every 3 months (which is why the ones that you see are “a bit out of date”). If you switch over to the “Edge” package set instead (through the package manager), you will be getting package updates every week or two instead. You will also be able to test out the new PBI system after switching over to the Edge set and updating all the PC-BSD utilities.

  • Julian
    May 15, 2014 at 7:03 pm |

    it sounds like you are losing the single biggest feature of PBIs. The ability to install without any concern as to the correct functioning of another installed component. If htis is not so, can you explain? The PBIs will no longer be self contained is the way I hear this.

  • LGE
    May 17, 2014 at 4:49 am |

    PBIs worked fine on PCBSD 8.x and 9.x. On PCBSD 10 something major changed and it’s being an stability nightmare.

    Please please please bring back PCBSD 9 style PBIs. Let them use the common /usr/local/ hierarchy instead of using weird hacks that are impossible to get right, use on a day-to-day basis or maintain properly (by your own admission).

    Julian is right. The single most important feature of PBIs is isolation between application updates. Nobody has that on UNIX except PCBSD. Don’t throw that out of the window.

    • Ken Moore
      May 19, 2014 at 6:16 am |

      LGE » The major 9.x->10.x change was exactly what you want for the old 9.x format — that they were compiled with the standard /usr/local/ heirarchy instead of the special non-standard /usr/pbi/ heirarchy of the 9.x series. That is one of the major hangups now — because the application container has to somehow virtualize the /usr/local namespace to prevent conflicts with the standard system files and keep the application looking at its own “contained” dependencies instead of bleeding out of the container and causing conflicts with the system-installed applications.
      For now, the portjail is your best option for a completely sandboxed application container. We are still looking at alternative container options to add in the future.

      • LGE
        May 19, 2014 at 1:34 pm |

        Ken, thanks for answering and all the hard work.

        There are advantages to packing together as much as an application needs in one independent location, as you guys did with PBI, that’s really cool. Complete isolation of an application, however, is an illusion, more so in a UNIX system composed of many focused, interacting utilities.

        I think this whole animated debate is about where to draw the boundary of what gets packaged into a PBI. Runtime libraries, utilities used internally, all that is OK; but each application cannot have its own fonts, or its own PDF reader, etc.

        That’s why I preferred PBIs in 9.x and before. In any PBI needed a Java runtime, I could use the one in the Java PBI. On the other hand, if a PBI required a specific version a Java, then that version could be included in the PBI. If I decided to use xpdf in the Opera PBI, but Okular on the desktop, that was OK, because the Opera PBI had access to the xpdf binary in the system. None of that is possible now because Opera has to “scape out” of its sandbox to use something like xpdf.

        I fear that the proposed uber-isolated jail application will be a pain to use because it is impossible to consider every single bit that might be used from the application if the user decides to.

        On the other hand, I also think PBIs should be reserved for big applications, like Eclipse, Opera, or GCC, which carry a lot of stuff with them. If somebody wants to install ZSH, well, yeah, they can use pkg for that, no doubt.

        Sorry for the the long rant.

  • jd
    May 17, 2014 at 6:14 pm |

    PBI’s are sandboxed and included all dependencies. Now we are going to a system (pkgng) where dependencies can break applications.

    I guess it’s better than having outdated applications which exhibit all sorts of instabilities.

    • Julian
      May 17, 2014 at 9:56 pm |

      I don’t think so.. outdated does not mean useless.
      If it aint broke, don’t fix it.
      Most packages just get ‘outdated’ but are not security risks.. If they do what you want that’s all you care about.

      As I said.. the complete self contained nature of PBIs was the most important feature of them (I’d say about 95% of the reason I supported them).
      If they lose that then I see no reason for them.
      they are then just another package setup and add nothing.

      • Ken Moore
        May 19, 2014 at 6:09 am |

        Julian » Yes and no. The simplest way to describe the changes is that we are moving the stand-alone application format to a side option for installing applications instead of it being the only PC-BSD supported method. We will still have application containers, but they will be changing a little bit because they are no longer expected to be the primary method by which applications are installed on the system.

  • Julian
    May 17, 2014 at 10:03 pm |

    Adding more.…
    I might add that you could make a self contained package (PBI) packed entirely of ‘pkg’ packages and install them in the way that PBIs did/do instead of the way they would in a normal freeBSD system.
    I mean each PBI would install a bunch of ‘pkg’ packages as needed, but in a separate sandboxs and do the cross linking for common stuff as it does now..
    That way a PBI would really just be a metadata package with ‘pkg’ doing the heavy lifting as far as compiles is concerned and the BI installer keeping track of what is where and keeping them independent of each other.

    • LGE
      May 18, 2014 at 12:47 am |

      Julian, I think that’s how PBIs work right now. They are created from packages directly. I think that would be fine, but the whole /usr/local emulation needs to go.

      • Julian
        May 18, 2014 at 4:22 am |

        I need to go look at the blog entry again. I’m not fussed about how it’s made up, but I do care about the fact that each pbi app only sees the libraries that came with it (or were found to be identical to them). my original reading was that the whole sandboxing aspect was going to go away. A sI said.. I’ll go and read it again ;-)

        • julian
          May 18, 2014 at 4:32 am |

          Ok so I read it again but I’m still not clear on whether PBI installed packages will be sandboxed or not. as far as that is concerned it is the single best thing about PBIs.. no interdependence between packages (unless you want to). that means that updating one package doesn’t mean you have to update others. Losing that will basically lose the reason for PBIs from my perspective.. I don’t care so much about all the other minor features mentionned.

          • Ken Moore
            May 19, 2014 at 6:05 am |

            julian » By default, applications will *not* be sandboxed. Instead, the sandbox/container will be an alternate installation option that must be explicitly selected by the user. For the moment portjails are the preferred sandbox/container — but we are looking into adding another option here soon once there has been enough time for people to make the transition over to the new PBI system.

            • julian
              May 19, 2014 at 6:11 am |

              it’s a pitty because that was the single thing that set PBIs above and ahead of the other forms.

              • Ken Moore
                May 19, 2014 at 6:28 am |

                julian » I agree. We just had to face the fact that most applications are not written to be able to function properly in a self-contained format. This makes it difficult for us to require that containers be the only “supported” application option because the number of applications that function properly in a container continues to decrease every year. Self-contained apps are still my preference as well — so I will be trying to make containers as simple/easy to use as possible — no matter which container format we end up going with.. :-)

                • Gour
                  May 29, 2014 at 6:20 am |

                  > Self-contained apps are still my preference as well — so I will be trying to make containers as simple/easy to use as possible — no matter which container format we end up going with..

                  Unfortunately I had to leave PC-BSD due to no solution for my overheating problem — PC-BSD was running at almost 20C higher temperature than Debian LInux — but seeing all the discussion about new steps in PC-BSD and anticipating to try it again in the future, I would like to say that self-contained apps were the main reason to consider PBI, so I wonder if what do you think about Nix — http://​nixos​.org/​n​ix/ — since PBI was always reminding me on it when I was playing with NixOS?

  • May 19, 2014 at 3:42 am |

    I’ve been waiting for this to happen since I first heard about PKGNG. Well done guys, I think this is a step in the right direction and will open up PC-BSD to a lot more people and hopefully relieve you of some work. As a casual user I could really benefit from a GUI interface to PKGNG and I’m sure many others would too.
    Having championed the PBI system for so long it must have been hard to move away but I think this is the best move for PC-BSD. Now if only those OS display drivers perform well then PC-BSD is really going to rock the desktop even more!

    • Ken Moore
      May 19, 2014 at 5:59 am |

      Geoff » Thanks for the support… :-)
      As mentioned previously, we will still have some kind of “contained” application format, but we will simply make it an optional installation option rather than the only supported method for applications.

      • RPG
        May 19, 2014 at 1:40 pm |

        I agree that this is a great move, and will simplify PC-BSD for both users and devs. I always avoided PBIs in the past, especially after moving to 10, unless I really needed something to be in a container. A sandbox is a useful and great option, but not appropriate for the majority of packages, so I think this plan is the best of both worlds. Also, I think that jails are the appropriate place for sandboxed applications on (Free/PC-)BSD. Together, these moves will make PC-BSD more like FreeBSD underneath, while being more streamlined for users on the surface–a great idea in both respects.

  • Hari Balaraman
    May 25, 2014 at 5:37 am |

    apostrophe.org.uk)apostrophe.org.uk)“uses the pkgng packages as it’s basis”

    in

    The next-generation PBI system”.

    should be

    uses the pkgng packages as its basis”

    it’s” is the contraction for “it is” whereas ‘its’ is possessive for the pronoun ‘it’. (check out http://​www​.apostrophe​.org​.uk)

  • bsdkeith
    June 12, 2014 at 3:08 am |

    Very glad to hear you have moved away from all that duplication of PBI packages.
    That is why I would not use PC-BSD before, now with ‘10’, I can run a normal FreeBSD system that has been pre configured straight out of the box. Excellent.

    • G.O.
      June 27, 2014 at 3:57 am |

      I dont see what prevented you from not using PBIs even on 8,9. There were always “ports” and “packages”. You did not have to touch PBIs.

    • julian
      June 27, 2014 at 4:42 am |

      I think what you say shows a real lack of understanding of the situation. with 30+ years of experience in the topic I can say that that was not a problem, and in 9.x PBIs there was very little duplication.. only in your mind.. because common libraries, while being “apparently duplicated” were in fact hard links, so there was only one copy.

      having self contained packages such as PBIs is a security plus, because you can update packages that need it without updating everything due to a complicated web of dependencies.

  • bsdkeith
    June 27, 2014 at 5:57 am |

    That being the case, then your promotional literature of the time lacked clarity. I am happy to even promote ‘PC-BSD 10′ now that it clearly uses the FreeBSD package system as its basis.

  • Jason
    July 5, 2014 at 3:58 pm |

    Thank you for making the switch. In the future, perhaps you can build a way of containing large applications/dependencies if that becomes a real problem. The old AppCafe was not very functional because it had such a tiny amount of software and much of it was old… I generally installed most of my applications outside of it. The new AppCafe is a huge improvement — and really allows one to “push-button install” just about anything they want.

    Frankly — what would be a better solution to software incompatibility problems is to rely/promote “Life Preserver” to be able to go back to a snapshot before each software install or upgrade.

Leave a comment

*

Please leave these two fields as-is:

Help the Project, Donate Today!