1 (edited by Neovana 2024-04-14 19:00:24)

Topic: openbsd fail2ban(failed)

- iRedMail version 1.6.8 PGSQL edition
- Deployed with downloadable installer
- OpenBSD 7.3 GENERIC.MP#5 amd64
- Store mail accounts in PGSQL
- Web server Nginx
- Manage mail accounts with iRedAdmin & command line
- no found logs related to fail2ban even after increasing logging to DEBUG

fail2ban is failing to load at startup, and the only error message I see is "fail2ban(failed)" in the console during startup, or if I attempt to start or check the service:

# /etc/rc.d/fail2ban check 
fail2ban(failed)

# /etc/rc.d/fail2ban start
fail2ban(failed)

# /etc/rc.d/fail2ban restart
fail2ban(failed)

There are no error messages in /var/log/messages which relate to fail2ban.

Additionally, in the mail server daily output email to root it includes

Services that should be running but aren't:
fail2ban

The logfile is empty:

# ls -al /var/log/fail2ban.log
-rwxr-xr-x  1 root  wheel  0 Mar 30 02:54 /var/log/fail2ban.log

I saw in the forums that previous version of the openbsd installer failed to install fail2ban completely, but that issue was fixed.

What more information can I post to be of assistance, or how can I safely reinstall fail2ban on openbsd so that it will play nicely with iredmail?

Thank you for your assistance.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2

Re: openbsd fail2ban(failed)

What's the output of command "fail2ban-client -x start"?

3

Re: openbsd fail2ban(failed)

ZhangHuangbin wrote:

What's the output of command "fail2ban-client -x start"?

% doas fail2ban-client -x start             
Server ready

% doas /etc/rc.d/fail2ban check
fail2ban(ok)


Unfortunately, this did not survive a restart, and at startup, upon loading daemons, it is reported in the console: fail2ban(failed)

% doas /etc/rc.d/fail2ban check             
fail2ban(failed)

% doas /etc/rc.d/fail2ban start
fail2ban(failed)

Should the "-x" flag be added to the "/etc/rc.d/fail2ban" file? Or is the startup script not loading with the appropriate permissions? Or does the /var/run/fail2ban directory have incorrect permissions?

% doas ls -al /etc/rc.d/fail2ban
-rwxr-xr-x  1 root  wheel  346 Mar 30 02:54 /etc/rc.d/fail2ban

% doas ls -al /var/run/fail2ban
drwx------   2 root       wheel        512 Apr 14 21:41 fail2ban

4

Re: openbsd fail2ban(failed)

Did fail2ban log something in log files under /var/log/* after a reboot?

5

Re: openbsd fail2ban(failed)

Unfortunately, not.

# ls -al /var/log/fail2ban.log
-rwxr-xr-x  1 root  wheel  0 Mar 30 02:54 /var/log/fail2ban.log

6

Re: openbsd fail2ban(failed)

Try this: open file /etc/rc.d/fail2ban, append " -xv" (there's a leading whitespace) to the first line, like this:

#!/bin/ksh -xv

Then restart OS and check console during system starts up.

7 (edited by Neovana 2024-04-24 13:57:27)

Re: openbsd fail2ban(failed)

Here is the relevant section from /var/log/daily.out:

#!/bin/ksh -xv
 
daemon="/usr/local/bin/fail2ban-client"
+ daemon=/usr/local/bin/fail2ban-client
 
. /etc/rc.d/rc.subr
+ . /etc/rc.d/rc.subr
#    $OpenBSD: rc.subr,v 1.160 2022/10/19 21:04:45 ajacoutot Exp $
#
# Copyright (c) 2010, 2011, 2014-2022 Antoine Jacoutot <ajacoutot@openbsd.org>
# Copyright (c) 2010, 2011 Ingo Schwarze <schwarze@openbsd.org>
# Copyright (c) 2010, 2011, 2014 Robert Nagy <robert@openbsd.org>
#
# Permission to use, copy, modify, and distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# copyright notice and this permission notice appear in all copies.
#
# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

_rc_actions="start stop restart reload check configtest"
+ _rc_actions=start stop restart reload check configtest
readonly _rc_actions
+ readonly _rc_actions

_rc_check_name() {
    [[ $1 == +([_[:alpha:]])+(|[_[:alnum:]]) ]]
}

_rc_do() {
    if [ -n "${_RC_DEBUG}" ]; then
        echo "doing $@" && "$@"
    else
        "$@" >/dev/null 2>&1
    fi
}

_rc_err() {
    [ -n "${1}" ] && echo "${1}" 1>&2
    [ -n "${2}" ] && exit "${2}" || exit 1
}

_rc_parse_conf() {
    typeset -l _key
    local _l _rcfile _val
    set -A _allowed_keys -- \
        accounting amd_master check_quotas ipsec library_aslr \
        multicast nfs_server pexp pf pkg_scripts shlib_dirs spamd_black

    [ $# -gt 0 ] || set -- /etc/rc.conf /etc/rc.conf.local
    for _rcfile; do
        [[ -f $_rcfile ]] || continue
        while IFS='     ' read -r _l; do
            [[ $_l == [!#=]*=* ]] || continue
            _key=${_l%%*([[:blank:]])=*}
            [[ $_key == *_@(execdir|flags|logger|rtable|timeout|user) ]] ||
                [[ " ${_allowed_keys[*]} " == *" $_key "* ]] ||
                continue
            [[ $_key == "" ]] && continue
            _val=${_l##*([!=])=*([[:blank:]])}
            _val=${_val%%#*}
            _val=${_val%%*([[:blank:]])}
            # remove leading and trailing quotes (backwards compat)
            [[ $_val == @(\"*\"|\'*\') ]] &&
                _val=${_val#?} _val=${_val%?}
            eval "${_key}=\${_val}"
        done < $_rcfile
    done

    # special care needed for spamlogd to avoid starting it up and failing
    # all the time
    if [ X"${spamd_flags}" = X"NO" -o X"${spamd_black}" != X"NO" ]; then
        spamlogd_flags=NO
    fi

    # special care needed for pflogd to avoid starting it up and failing
    # if pf is not enabled
    if [ X"${pf}" = X"NO" ]; then
        pflogd_flags=NO
    fi

    # special care needed if nfs_server=YES to startup nfsd and mountd with
    # sane default flags
    if [ X"${nfs_server}" = X"YES" ]; then
        [ X"${nfsd_flags}" = X"NO" ] && nfsd_flags="-tun 4"
        [ X"${mountd_flags}" = X"NO" ] && mountd_flags=
    fi
}

# return if we only want internal functions
[ -n "${FUNCS_ONLY}" ] && return
+ [ -n  ]

_rc_not_supported() {
    local _a _enotsup _what=${1}
    for _a in ${_rc_actions}; do
        [ "${_what}" == "configtest" ] &&
            ! typeset -f rc_configtest >/dev/null && _enotsup=NO &&
            break
        [ "${_what}" == "restart" ] && _what="stop"
        if [ "${_what}" == "${_a}" ]; then
            eval _enotsup=\${rc_${_what}}
            break
        fi
    done
    [ X"${_enotsup}" == X"NO" ]
}

_rc_usage() {
    local _a _allsup
    for _a in ${_rc_actions}; do
        _rc_not_supported ${_a} || _allsup="${_allsup:+$_allsup|}${_a}"
    done
    _rc_err "usage: $0 [-df] ${_allsup}"
}

_rc_write_runfile() {
    [ -d ${_RC_RUNDIR} ] || mkdir -p ${_RC_RUNDIR} &&
        cat >${_RC_RUNFILE} <<EOF
daemon_class=${daemon_class}
daemon_execdir=${daemon_execdir}
daemon_flags=${daemon_flags}
daemon_logger=${daemon_logger}
daemon_rtable=${daemon_rtable}
daemon_timeout=${daemon_timeout}
daemon_user=${daemon_user}
pexp=${pexp}
rc_reload=${rc_reload}
rc_reload_signal=${rc_reload_signal}
rc_stop_signal=${rc_stop_signal}
rc_usercheck=${rc_usercheck}
EOF
}

_rc_rm_runfile() {
    rm -f ${_RC_RUNFILE}
}

_rc_exit() {
    local _pfix
    [ -z "${INRC}" -o X"$1" != X"ok" ] && _pfix="($1)"
    echo ${INRC:+'-n'} "${_pfix}"
    [[ $1 == @(ok|killed) ]] && exit 0 || exit 1
}

_rc_alarm()
{
    trap - ALRM
    kill -ALRM ${_TIMERSUB} 2>/dev/null # timer may not be running anymore
    kill $! 2>/dev/null # kill last job if it's running
}

_rc_sendsig() {
    pkill -${1:-TERM} -T "${daemon_rtable}" -xf "${pexp}" 
}

_rc_wait_for_start() {
    trap "_rc_alarm" ALRM
    while ((SECONDS < daemon_timeout)); do
        if _rc_do rc_check; then
            [ X"${rc_bg}" = X"YES" ] || [ -z "$$" ] && break
        fi
        sleep .5
    done & wait
    pkill -ALRM -P $$
    return
}

rc_exec() {
    local _rcexec="su -fl -c ${daemon_class} -s /bin/sh ${daemon_user} -c"
    [ "${daemon_rtable}" -eq "$(id -R)" ] ||
        _rcexec="route -T ${daemon_rtable} exec ${_rcexec}"

    ${_rcexec} "${daemon_logger:+set -o pipefail; } \
        ${daemon_execdir:+cd ${daemon_execdir} && } \
        $@ \
        ${daemon_logger:+ 2>&1 |
            logger -isp ${daemon_logger} -t ${_name}}"
}

rc_start() {
    rc_exec "${daemon} ${daemon_flags}"
}

rc_check() {
    pgrep -T "${daemon_rtable}" -q -xf "${pexp}"
}

rc_reload() {
    _rc_sendsig ${rc_reload_signal}
}

rc_stop() {
    _rc_sendsig ${rc_stop_signal}
}

rc_cmd() {
    local _exit _n _ret _timer
    # optim: don't sleep(1) in the first loop
    _1stloop=true

    [ -n "${1}" ] && echo "${_rc_actions}" | grep -qw -- ${1} || _rc_usage

    [ "$(id -u)" -eq 0 ] ||
        [ X"${rc_usercheck}" != X"NO" -a X"$1" = "Xcheck" ] ||
        _rc_err "$0: need root privileges"

    if _rc_not_supported $1; then
        [ -n "${INRC}" ] && exit 1
        _rc_err "$0: $1 is not supported"
    fi

    [ -n "${_RC_DEBUG}" ] || _n="-n"

    [[ ${1} == start ]] || _rc_do _rc_parse_conf ${_RC_RUNFILE}

    case "$1" in
    check)
        echo $_n "${INRC:+ }${_name}"
        _rc_do rc_check && _rc_exit ok || _rc_exit failed
        ;;
    configtest)
        echo $_n "${INRC:+ }${_name}"
        _rc_do rc_configtest && _rc_exit ok || _rc_exit failed
        ;;
    start)
        if [ X"${daemon_flags}" = X"NO" ]; then
            _rc_err "$0: need -f to force $1 since ${_name}_flags=NO"
        fi
        [ -z "${INRC}" ] && _rc_do rc_check && exit 0
        echo $_n "${INRC:+ }${_name}"
        while true; do # no real loop, only needed to break
            # running during start is mostly useful for daemons
            # whose child will not return a config parsing error to
            # the parent during startup; e.g. bgpd, httpd...
            if typeset -f rc_configtest >/dev/null; then
                _rc_do rc_configtest || break
            fi
            if typeset -f rc_pre >/dev/null; then
                _rc_do rc_pre || break
            fi
            # prevent hanging the boot sequence
            _rc_do _rc_wait_for_start & _TIMERSUB=$!
            trap "_rc_alarm" ALRM
            _rc_do rc_start; _ret=$?
            kill -ALRM ${_TIMERSUB}
            wait ${_TIMERSUB} 2>/dev/null # don't print Alarm clock
            # XXX for unknown reason, rc_check can fail (e.g. redis)
            # while it just succeeded in _rc_wait_for_start;
            # check to cope with failing daemons returning 0
            #[[ "${_ret}" == @(0|142) ]] && _rc_do rc_check || break
            [[ "${_ret}" == @(0|142) ]] || break
            [[ "${_ret}" == 142 ]] && [ X"${rc_bg}" != X"YES" ] &&
                _exit="timeout"
            _rc_do _rc_write_runfile
            _rc_exit ${_exit:=ok}
        done
        # handle failure
        _rc_do _rc_rm_runfile
        typeset -f rc_post >/dev/null && _rc_do rc_post
        _rc_exit failed
        ;;
    stop)
        _rc_do rc_check || exit 0
        echo $_n "${INRC:+ }${_name}"
        _rc_do rc_stop & _timer=$!
        while ((SECONDS < daemon_timeout)); do
            # last chance: send a SIGTERM first in case the process
            # used another signal to stop (e.g. SIGQUIT with nginx)
            # or a non-default rc_stop() function; do it 2s before
            # timeout to re-enter the loop one last time which will
            # give 1s for SIGTERM to terminate the process
            ((SECONDS == daemon_timeout-2)) &&
                _rc_do _rc_sendsig TERM && sleep .5
            pkill -0 -P "$$" 2>/dev/null || _rc_do rc_check ||
                break
            ${_1stloop} && _1stloop=false || sleep .5
        done
        kill -ALRM ${_timer} 2>/dev/null
        wait ${_timer} # don't print Alarm clock
        [[ $? == 0 ]] || _exit=failed
        # KILL the process
        _rc_do rc_check && _rc_do _rc_sendsig KILL && _exit="killed"
        _rc_do _rc_rm_runfile
        if typeset -f rc_post >/dev/null; then
            _rc_do rc_post || _exit=failed
        fi
        _rc_exit ${_exit:=ok}
        ;;
    reload)
        echo $_n "${INRC:+ }${_name}"
        _rc_do rc_check || _rc_exit failed
        if typeset -f rc_configtest >/dev/null; then
            _rc_do rc_configtest || _rc_exit failed
        fi
        _rc_do rc_reload & _timer=$!
        while ((SECONDS < daemon_timeout)); do
            pkill -0 -P "$$" 2>/dev/null || break
            ${_1stloop} && _1stloop=false || sleep .5
        done
        kill -ALRM ${_timer} 2>/dev/null
        wait ${_timer} # don't print Alarm clock
        _ret=$?
        [[ ${_ret} == 142 ]] && _exit=timeout || [[ ${_ret} == 0 ]] ||
            _exit=failed
        _rc_exit ${_exit:=ok}
        ;;
    restart)
        if typeset -f rc_configtest >/dev/null; then
            _rc_do rc_configtest || _rc_exit failed
        fi
        $0 ${_RC_DEBUG} ${_RC_FORCE} stop &&
            $0 ${_RC_DEBUG} ${_RC_FORCE} start
        ;;
    *)
        _rc_usage
        ;;
    esac
}

_name=${0##*/}
+ _name=fail2ban
_rc_check_name "${_name}" || _rc_err "invalid rc.d script name: ${_name}"
+ _rc_check_name fail2ban

[ -n "${KSH_VERSION}" ] || _rc_err "$0: wrong shell, use /bin/ksh"
+ [ -n @(#)PD KSH v5.2.14 99/07/13.2 ]
[ -n "${daemon}" ] || _rc_err "$0: daemon is not set"
+ [ -n /usr/local/bin/fail2ban-client ]

unset _RC_DEBUG _RC_FORCE
+ unset _RC_DEBUG _RC_FORCE
while getopts "df" c; do
    case "$c" in
        d) _RC_DEBUG=-d;;
        f) _RC_FORCE=-f;;
        *) _rc_usage;;
    esac
done
+ getopts df c
shift $((OPTIND-1))
+ shift 0

_RC_RUNDIR=/var/run/rc.d
+ _RC_RUNDIR=/var/run/rc.d
_RC_RUNFILE=${_RC_RUNDIR}/${_name}
+ _RC_RUNFILE=/var/run/rc.d/fail2ban

# parse /etc/rc.conf{.local} for the daemon variables
_rc_do _rc_parse_conf
+ _rc_do _rc_parse_conf

rc_reload_signal=${rc_reload_signal:=HUP}
+ rc_reload_signal=HUP
rc_stop_signal=${rc_stop_signal:=TERM}
+ rc_stop_signal=TERM

eval _rcexecdir=\${${_name}_execdir}
+ eval _rcexecdir=${fail2ban_execdir}
_rcexecdir=${fail2ban_execdir}
+ _rcexecdir=
eval _rcflags=\${${_name}_flags}
+ eval _rcflags=${fail2ban_flags}
_rcflags=${fail2ban_flags}
+ _rcflags=
eval _rclogger=\${${_name}_logger}
+ eval _rclogger=${fail2ban_logger}
_rclogger=${fail2ban_logger}
+ _rclogger=
eval _rcrtable=\${${_name}_rtable}
+ eval _rcrtable=${fail2ban_rtable}
_rcrtable=${fail2ban_rtable}
+ _rcrtable=
eval _rctimeout=\${${_name}_timeout}
+ eval _rctimeout=${fail2ban_timeout}
_rctimeout=${fail2ban_timeout}
+ _rctimeout=
eval _rcuser=\${${_name}_user}
+ eval _rcuser=${fail2ban_user}
_rcuser=${fail2ban_user}
+ _rcuser=

# set default values; duplicated in rcctl(8)
getcap -f /etc/login.conf.d/${_name}:/etc/login.conf ${_name} 1>/dev/null 2>&1 && \
    daemon_class=${_name} || daemon_class=daemon
+ getcap -f /etc/login.conf.d/fail2ban:/etc/login.conf fail2ban
+ > /dev/null 
+ 2>&1 
+ daemon_class=daemon
[ -z "${daemon_rtable}" ] && daemon_rtable=0
+ [ -z  ]
+ daemon_rtable=0
[ -z "${daemon_timeout}" ] && daemon_timeout=30
+ [ -z  ]
+ daemon_timeout=30
[ -z "${daemon_user}" ] && daemon_user=root
+ [ -z  ]
+ daemon_user=root

# use flags from the rc.d script if daemon is not enabled
[ -n "${_RC_FORCE}" -o "$1" != "start" ] && [ X"${_rcflags}" = X"NO" ] &&
    unset _rcflags
+ [ -n  -o check != start ]
+ [ X = XNO ]

[ -n "${_rcexecdir}" ] && daemon_execdir=${_rcexecdir}
+ [ -n  ]
[ -n "${_rcflags}" ] && daemon_flags=${_rcflags}
+ [ -n  ]
[ -n "${_rclogger}" ] && daemon_logger=${_rclogger}
+ [ -n  ]
[ -n "${_rcrtable}" ] && daemon_rtable=${_rcrtable}
+ [ -n  ]
[ -n "${_rctimeout}" ] && daemon_timeout=${_rctimeout}
+ [ -n  ]
[ -n "${_rcuser}" ] && daemon_user=${_rcuser}
+ [ -n  ]

if [ -n "${_RC_DEBUG}" ]; then
    echo -n "${_name}_flags "
    [ -n "${_rcflags}" ] || echo -n "empty, using default "
    echo ">${daemon_flags}<"
fi
+ [ -n  ]

readonly daemon_class
+ readonly daemon_class
unset _rcexecdir _rcflags _rclogger _rcrtable _rctimeout _rcuser
+ unset _rcexecdir _rcflags _rclogger _rcrtable _rctimeout _rcuser
# the shell will strip the quotes from daemon_flags when starting a daemon;
# make sure pexp matches the process (i.e. doesn't include the quotes)
pexp="$(eval echo ${daemon}${daemon_flags:+ ${daemon_flags}})"
+ eval echo /usr/local/bin/fail2ban-client
echo /usr/local/bin/fail2ban-client
+ echo /usr/local/bin/fail2ban-client
+ pexp=/usr/local/bin/fail2ban-client
 
rc_bg=YES
+ rc_bg=YES
rc_reload=NO
+ rc_reload=NO
 
rc_pre() {
    install -d -o root -m 0700 /var/run/fail2ban
}
 
rc_start() {
    ${rcexec} "${daemon} start ${daemon_flags} ${_bg}"
}
 
rc_check() {
    pgrep -q -f "fail2ban-server"
}
 
rc_stop() {
    ${rcexec} "${daemon} stop"
}
 
rc_cmd $1
+ rc_cmd check
_enotsup=${rc_check}
fail2ban(failed).

8

Re: openbsd fail2ban(failed)

Neovana wrote:

Should the "-x" flag be added to the "/etc/rc.d/fail2ban" file? Or is the startup script not loading with the appropriate permissions? Or does the /var/run/fail2ban directory have incorrect permissions?

I added the "-x" flag and it works on OpenBSD 7.5 with upcoming iRedMail release.
Could you help verify it?

diff --git a/samples/fail2ban/openbsd/rc b/samples/fail2ban/openbsd/rc
index c17846b9..b64f690c 100755
--- a/samples/fail2ban/openbsd/rc
+++ b/samples/fail2ban/openbsd/rc
@@ -12,7 +12,7 @@ rc_pre() {
 }
  
 rc_start() {
-    ${rcexec} "${daemon} start ${daemon_flags} ${_bg}"
+    ${rcexec} "${daemon} -x start ${daemon_flags} ${_bg}"
 }
  
 rc_check() {

9

Re: openbsd fail2ban(failed)

OpenBSD mail.neovana.com 7.3 GENERIC.MP#5 amd64

% doas cat /etc/rc.d/fail2ban
#!/bin/ksh
 
daemon="/usr/local/bin/fail2ban-client"
 
. /etc/rc.d/rc.subr
 
rc_bg=YES
rc_reload=NO
 
rc_pre() {
    install -d -o root -m 0700 /var/run/fail2ban
}
 
rc_start() {
    ${rcexec} "${daemon} -x start ${daemon_flags} ${_bg}"
}
 
rc_check() {
    pgrep -q -f "fail2ban-server"
}
 
rc_stop() {
    ${rcexec} "${daemon} stop"
}
 
rc_cmd $1
% doas /etc/rc.d/fail2ban check
fail2ban(failed)

% doas /etc/rc.d/fail2ban start
fail2ban(failed)

10

Re: openbsd fail2ban(failed)

I did few more tests and it failed. Still working on it.

11

Re: openbsd fail2ban(failed)

ZhangHuangbin wrote:

I did few more tests and it failed. Still working on it.

I just discovered that fail2ban was also failing on Debian 12 (on a different instance - not an iRedMail server). Needed to modify /etc/fail2ban/jail.conf:

- backend=auto
+ backend=systemd

I don't think that's the problem with OpenBSD because as you know, fail2ban will load when you start it manually from the command line:

doas fail2ban-client -x start

Just figured that I'd mention it. Thank you for continuing to look into it.

12

Re: openbsd fail2ban(failed)

- Fail2ban works fine on Debian 12 with "backend = auto" in jail.conf during my local testings. And no other users reported this issue before.
- Still didn't figure out why it cannot start on OpenBSD (i tested it with OpenBSD 7.5).