Topic: Not a support ticket: Sharing tests, ideas and comments
==== REQUIRED BASIC INFO OF YOUR IREDMAIL SERVER ====
- iRedMail version (check /etc/iredmail-release): 1.7.4
- Deployed with iRedMail Easy or the downloadable installer? Download
- Linux/BSD distribution name and version: FreeBSD 14.3-RELEASE-p5 GENERIC amd64
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): PGSQL
- Web server (Apache or Nginx): Nginx
- Manage mail accounts with iRedAdmin-Pro? No
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====
NOTE: Not a support ticket!
Introduction
I have been using iRedMail for a few years. I use it to run the mails in my personal domain (just me, no company, no employees, not even family or friends).
After the initial install back on the day, I just let it run.
Recently, I got some motivation to update the OS (FreeBSD) and iRedMail, so I am doing some tests in virtual machines to get reacquainted with all the details.
It was a hard reminder of how much time it takes to install, given the use of ports and the need to compile everything from scratch. (BTW: I do not disagree nor am I complaining with the approach, rather the opposite, I like it). However, compiling Rust, LLVM, and traces of X11, not to mention Python, PHP, and Perl sounds a bit excessive. It is my first attempt, and I am reaching 24 hours, as I have to investigate and fix underlying issues.
I am investigating, fixing and will report as I go. I hope this helps the iRedMail team and the community.
As iRedMail runs as a service on headless computers, there is no need for extras, such as code libraries, documentation, or manuals. Most extras are disabled. But dependencies have their own configurations, and it becomes hard to keep track of all the derivations. I have questions, comments, and, at times, suggestions, again in the hope of contributing in some way.
Thanks to iRedMail for a good product.
Environment
I am playing with iRedMail (1.7.4) on FreeBSD (14.3) with PostgreSQL (15, but trying 17 as well) and Nginx, before I upgrade from my working setup, which is the same, only with older versions of the software combo. My input is specific to the environment I am currently investigating. If anyone wants to contribute to the email track, please keep in these boudaries.
Perl
Is Perl necessary in iRedMail because of SpamAssassin?
I hope it is not only for string substitution in the installation scripts?
echo "* Comment out 'smtp inet ... smtpd' service in ${MASTER_CF}."
perl -pi -e 's/^(smtp .*inet.*smtpd)$/#${1}/g' ${MASTER_CF}I used sed years ago when fail2ban was not implemented on FreeBSD, so I amended the scripts to add it:
# fail2ban.md:
# I am old-school. I like awk, sed, etc. I am not familiar with Perl. Do not care for it.
sed -i '' -e "s#PH_FAIL2BAN_ACTION#$FAIL2BAN_ACTION#" $JAIL_D_LOCAL
sed -i '' -e "s#PH_SSHD_LOGFILE#$SSHD_LOGFILE#" $JAIL_D_LOCALSpamAssassin (vs something else)
Speaking of Perl and SpamAssassin, any interest, consideration or reasearch on alternatives?
A quick internet search points to RSPAMD. I was trying to get rid of extra scripting languages (looking at Perl here), and got Lua instead.
But at least the stats says that RSPAMD is way faster than SpamAssassin) with some modern bells and whistles. (!?)
- RSpamd on GitHub
- Migrating from SpamAssassin to Rspamd
Graphics
1. Why these dependencies? My guess is to inspect graphic and image files for viruses.
- graphics_php99-gd
- The gd extension for PHP, which enables the creation and manipulation of images, can be found in the graphics/phpXX-gd port.
- graphics_cairo
- Cairo is a vector graphics library with cross-device output support. Currently supported output targets include the X Window System and in-memory image buffers. PostScript and PDF file output is planned. Cairo is designed to produce identical output on all output media while taking advantage of display hardware acceleration when available (eg. through the X Render Extension).
- Cairo provides a stateful user-level API with capabilities similar to the PDF 1.4 imaging model. Cairo provides operations including stroking and filling Bezier cubic splines, transforming and compositing translucent images, and antialiased text rendering.
2. There are traces of the X11, such as fonts and other libraries associated with it. I know X11 is excluded from installation in the scripts, but something still falls through the cracks.
Rust
The big picture here is about all the scripting, interpreted, and compiled languages (Rust, Python, PHP, and Perl). It is possible that interpreted languages need to be compiled from scratch to add specific (security) modules that are not included in most distributions (for some reason).
I could not identify the same rationale in Rust, or why ClamAV needs it beyond using the toolchain.
With all the fuss about Rust (ClamAV being one that affects iRedMail), the installation time has skyrocketed.
I am still trying to get the dependencies, but I wonder if, specifically for Rust, it could be a pre-requisite to be already installed on the system, even via pkg. (See Note 1. below)
Back to Rust, it uses a customised version of LLVM, which makes the iRedMail installation extremely long.
After I wrote the paragraph above, I checked why Rust and LLVM get compiled and how to skip their compilation. Rust gets compiled because of a check in the make file for cargo (/usr/ports/Mk/Uses/cargo.mk). To skip compilation, either install a brand-new version or instruct cargo not to compile.
My suggestion is to document Rust as a pre-requisite for the installation, like bash-static. It can be installed as follows:
- Make install Rust a pre-requisite, before installing iRedMail
## Step 1 - Option 1 - Install the latest Rust version using the instructions from the rust-lang.org
This is probably the best option.
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh## Step 1 - Option 2 - Stay with the pkg version
# check the Rust version available
export RUST_PKG_VERSION=`pkg search rust | grep -E "^rust" | grep "Language" | head -1 | cut -d" " -f1`
echo ${RUST_PKG_VERSION}
pkg install -y ${RUST_PKG_VERSION}Note: Sorry not to offer a more "developer" format, such as diff, but it would make a longer message than it is already. There are two points that can be changed (one or the other, not both):
## Step 2 - Option 1 - Modify the installation script ./functions/packages_freebsd.sh
Add export CARGO_BUILDDEP='no' to ./functions/packages_freebsd.sh close to the the references to make
# Make it don't popup dialog while building ports.
export PACKAGE_BUILDING='yes'
export BATCH='yes'
# Skip compiling Rust and LLVM from source # ADD THIS LINE (And remove the comment)
export CARGO_BUILDDEP='no' # ADD THIS LINE (And remove the comment)## Step 2 - Option 2 - Modify the installation script ./functions/packages_freebsd.sh
Add freebsd_make_conf_add 'CARGO_BUILDDEP' "no" to ./functions/packages_freebsd.sh to the existing list.
freebsd_make_conf_add 'DEFAULT_VERSIONS' "..."
freebsd_make_conf_add 'CARGO_BUILDDEP' "no" # ADD THIS LINE (And remove the comment)NOTE: Correction: Despite all the above, at some point the depencies force the compilation of Rust and LLVM from scratch, regardless of any variables, either in the scripts or even the /etc/make.conf. The only option is to actually remove the BUILD_DEPENDENCY line from the /usr/ports/security/clamav/Makefile, which is a terrible idea.
-----
(1.) Simplified explanation: FreeBSD uses two methods to install programs: pkg, which installs binary code; and ports, which are installed by compiling the program from the source code. There is an unwritten rule that one should avoid mixing the two installation methods, as dependency libraries may have misaligned versions, leading to issues and instability. But it is possible with care and proper control.
(2.) Out of curiosity, LLVM compiles 3,795 object files.
Random Password Generator
Funny, we have been here before, and I tought this was fixed. Surprise, back to square one.
In my tests I got the message Fatal: canceling authentication due to timeout thrown by PostgreSQL.
After all sort of misdirections, this has to do with the random password generation (Please refer to 1.).
After jointly chasing this with ZhangHuangbin back in 2020, we nail it to the use of eval not working properly on FreeBSD around version 12.1-RELEASE-p10.
I then suggested to use another method to create the random password via openssl.
To my surprise I found the old offender back in iRedMail 1.7.4.
My huntch is that iRedMail on FreeBSD with PostgreSQL is not a popular combination.
- ./conf/global
elif [ X"${KERNEL_NAME}" == X'FREEBSD' ]; then
# export RANDOM_STRING='eval </dev/urandom LC_CTYPE=C tr -dc a-zA-Z0-9 | head -c 32'
export RANDOM_STRING='openssl rand -base64 24' # ADD THIS LINE (And remove the comment)Which works to create the random passwords:
- ./config
export STORAGE_BASE_DIR='/var/vmail'
export WEB_SERVER='NGINX'
export BACKEND_ORIG='PGSQL'
export BACKEND='PGSQL'
export VMAIL_DB_BIND_PASSWD='uvzO2hqbrO+s/Kk26EELgor1YFyxg8BO'
export VMAIL_DB_ADMIN_PASSWD='ha4CiG50m0EQmpN3zufImFgDo6TmJJj/'
...(1.) Message in the iRedMail Support Forum: Topic: Install Hangs while creating database on the 2020-10-09 17:24:56. Guess by whom? Me
# Log timestamps
Any chance of adding a timestamp to [ INFO ], [ * ] without breaking the instalation process?
Currently only the [ INFO ] Installing port: ... (yyyy-mm-dd hh:mm:ss) has a timestamp.
Style and preferences are boundless, but change ECHO_INFO(), ECHO_SKIP(), ECHO_QUESTION(), ECHO_ERROR(), and ECHO_DEBUG() at ./conf/core should be easy.
Maybe soomething like [ INFO ] yyyy-mm-dd hh:mm:ss - ....
This would make it easier to have a mental notion of how long things are by checking at ./runtime/install.log.
# FreeBSD install using pkg instead of ports
As I was waiting for things to compile, so I went to the iRedMail@Github and was clicking around. I bumped in the interesting colaboration disguised as an issue. #272-Use packages instead of ports ?.
Great work! and Thank you.
I will test that next.
Update: I run it twice as I document step-by-step, so I can reproduce in the separate domains I have. I wonder if one day I could/shold do this with Ansible.
It works just fine, and I guess it is official, as it is an iRedMail branch on Github about the same time of the 1.7.4 version. If it is official, I only which it would be referenced and offered somewhere, instead of me pocking around and finding this by accident.
Regards to the iRedMail team.
Thanks again for the product.
----
Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.