1 Topic by aida

  • aida
  • Member
  • Offline
  • Registered: 2018-11-06
  • Posts: 10

Topic: How to upgrade packages/ports on FreeBSD 12.0-RELEASE

==== REQUIRED BASIC INFO OF YOUR IREDMAIL SERVER ====
- iRedMail version (check /etc/iredmail-release): 0.9.9
- Deployed with iRedMail Easy or the downloadable installer? downloadable installer
- Linux/BSD distribution name and version: FreeBSD 12.0-RELEASE (iocage jail)
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Web server (Apache or Nginx): Nginx
- Manage mail accounts with iRedAdmin-Pro? No
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

Hi ZhangHuangbin,

This is more or less a best practice question on how to upgrade packages/ports on FreeBSD.

During the years a sort of mixed information appeared on the forums. For example in May 2015 you confirmed that:

ZhangHuangbin wrote:

(pkg) Should be fine. But if you want to upgrade packages, why not use tools like 'portmaster'?

Then later in July 2017 you on this question:

sethkush wrote:

So 'pkg upgrade' is the recommended way to update?

maybe a bit reluctantly confirmed that

ZhangHuangbin wrote:

It's ok.

I'm not going to accept something that is just ok, I'm looking for the best way how to go about performing the upgrade.

From my experience, a 

pkg upgrade

leads to broken iRedMail and running 

portmaster -a

requires pressing enter about 400 times on course of many hours.

I understand that noting works smoothly all the time and that there is a need to troubleshoot the problems, but I would really like to know what is the best practice to perform a package/port upgrade before I start solving incompatibilities with every individual one.

Thank you so much for your advice.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,091

Re: How to upgrade packages/ports on FreeBSD 12.0-RELEASE

hmm, that's the weakness of FreeBSD ports tree, but not OpenBSD ports tree, neither NetBSD pkgsrc. Because for example  the goal of OpenBSD ports tree is binary packages, OpenBSD team builds binary packages for all ports so that you can save a lot time and just run "pkg_add foobar" to install a package, just 1-3 seconds with fast network, no surprise, no panic, kind of boring which is the best for sysadmin. This is achieved by different "flavor"s of each port. For example, for same Postfix version (same port), there're different packages:

postfix-3.3.3-ldap.tgz
postfix-3.3.3-mysql.tgz
postfix-3.3.3-pgsql.tgz
postfix-3.3.3-sasl2-ldap.tgz
postfix-3.3.3-sasl2-mysql.tgz
postfix-3.3.3-sasl2-pgsql.tgz
postfix-3.3.3-sasl2.tgz
postfix-3.3.3.tgz

The "ldap", "mysql", "pgsql", "sasl2-ldap", ... are produced by the "flavors", if you need ldap support, just install the "postfix-3.3.3-ldap.tgz". Same to the Dovecot package:

dovecot-2.3.5.1.tgz
dovecot-ldap-2.3.5.1.tgz
dovecot-mysql-2.3.5.1.tgz
dovecot-pigeonhole-0.5.5v0.tgz
dovecot-postgresql-2.3.5.1.tgz

The binary packages built by FreeBSD team can not fully match what iRedMail needs, mostly caused by its default compile options, for example, iRedMail supports storing mail accounts in different backends, OpenLDAP/MySQL/PostgreSQL, but Postfix binary package doesn't support them (either support them all, or offer the sub-packages to support each so that you can just install the one you need). But OpenBSD binary packages is perfect.

Back to your question, the best way to upgrade FreeBSD packages on an iRedMail server, is tool like "portmaster" which can upgrade packages with ports tree directly. But the problem remains: it may break something due to different versions in latest ports tree, for example, upgrade Dovecot 2.x to 3.x (some parameters were deprecated, or not supported, or new parameters are required but not used in your old config file), this is caused by the software internal change, not iRedMail, so we can not guarantee or promise anything.

3 Reply by aida (edited by aida 2019-10-01 04:51:04)

  • aida
  • Member
  • Offline
  • Registered: 2018-11-06
  • Posts: 10

Re: How to upgrade packages/ports on FreeBSD 12.0-RELEASE

Hey ZhangHuangbin,

Thank you so much for your kind reply. This is something what I pretty much expected, I was just looking for confirmation that the pain is unavoidable. :-)

If I may share here my upgrade strategy is as follows:


1) Make a snapshot. This is the most important step and I can't stress enough how important this is.

2) Make sure you made a snapshot.

3) Stop all the important services. This might be just superstition.

/usr/sbin/service php-fpm stop && /usr/sbin/service nginx stop && /usr/sbin/service postfix stop && /usr/sbin/service dovecot stop && /usr/sbin/service mysql-server stop

4) Fetch & Update Ports Tree.

5) Install and/or Upgrade Portmaster.

For steps 4 and 5 follow The FreeBSD Handbook chapter 4.5. Using the Ports Collection

6) Let portmaster upgrade everything. For that I use the following command:

portmaster -ady --no-confirm --packages-build

7) Wait for a few hours for the builds to finish and occasionally press enter (when prompted).

8) Make sure that all the builds finished without throwing an error.

9) Restart the machine while crossing your fingers.

10) Make sure that everything still works.


Troubleshooting

In the case, a build of one of the ports fails, check the list of used components and packages_freebsd.sh to get an idea about the importance of the specific port and to see if the port's configuration varies from the default.

If the port is important and individually configured - rollback to the previous snapshot and wait for a fix. Of course in the meantime feel free to Google the issue and possible workarounds, contact the port maintainer, report bugs, etc.

If the port is a dependency and uses a default configuration you can try to install it as a binary package. Once that is done run portmaster again.  

portmaster -ady --no-confirm --packages-build

In the case of something not working after a successful build of all the ports use logs (/var/logs) to find out what is not working and why it is freaking out then you can rollback, try to fix it or report the issue here.

4 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,091

Re: How to upgrade packages/ports on FreeBSD 12.0-RELEASE

- Step 3 can be removed. smile
- Also, maybe build binary packages first, then upgrade.