There is no forwarding back to our server in place. Not something I would have thought of, but I did double check this.

I am hesitant to add ALLOWED_FORGED_SENDERS setting, as I feel it should not be needed, and I don't quite understand the implications.

The main thing I do not understand is why it only fails with a very small percentage of the emails, and re-sending it likely works. If something was misconfigured, would it not always fail?

554 5.7.1 : Recipient address rejected: SMTP AUTH is required for users under this sender domain

This error message was produced by iRedAPD, and you can find the reason and solution:
https://docs.iredmail.org/errors.html#r … der-domain

The point is not the solution here, but why the sender in this message contains email domain name hosted on your server. Does the external account forward / redirect the forwarded/redirected message back to your server?

About the SPF/DKIM thing, please make sure you have correct DNS records for your email domain names. Also, is it possible that the sender address in email was changed?

This is a bit of a long post, but I'm trying to add as much relevant information as I can think of...

A bit of history:
Say our mail domain is domain.com.  Some of our mailboxes are forwarding email to gapps (at user@gapps.domain.com), others to regular gmail/yahoo/whatever accounts, with or without also keeping a local copy.
Now, 99% of the time, this works as expected, but every once in a while, gapps is bouncing a message back with
   

Re-sending the same message will again fix it in 99% of the cases. Rarely it will have to be sent a third time.

Both SPF and DKIM are set correctly, but I doubt that's where the problem is because, again, it works correctly 99% of the time.

This seems to happen mostly (only?) with internal messages, and mostly (only?) when forwarding to gapps accounts.
It also may happen when sending to multiple recipients, all of which are forwarding to gapps, one will bounce, and all others go through. This in particular I find puzzling...

This situation has existed like this for years, without other issues, and has been mostly just annoying, but we can live with it.
The system works flawlessly otherwise.

The bigger problem:
Recently, a customer sent us this bounce message which is similar, but more serious than internal only ones.
(Note, this user forwards to a regular gmail account, not gapps like above, which may explain the difference in wording):

How can SPF/DKIM fail in less than 1% of all cases?

This is the first occurrence we know of that involves external email, or other destinations than gapps.

We would be OK with just ignoring this error, since gapps/gmail are just backup accounts in case things go horribly wrong, but the customer assumer (and rightfully so) we did not receive the email, which we did.

The questions:

1. Any theories what may be going on here (or how to further debug/fix/workaround)

2. Is there some way to do a retry or something before bouncing back to our customers

3. Any ideas on how to (temporary) prevent the user from receiving this bounce message completely, and just ignore it