1 Topic by raystrach (edited by raystrach 2018-07-27 10:39:00)

  • raystrach
  • Member
  • Offline
  • Registered: 2012-05-21
  • Posts: 37

Topic: trying to convert to sdbox mail format

==== REQUIRED BASIC INFO OF YOUR IREDMAIL SERVER ====
- iRedMail version (check /etc/iredmail-release): 0.9.8
- Linux/BSD distribution name and version: centos 7
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): martiadb
- Web server (Apache or Nginx): nginx
- Manage mail accounts with iRedAdmin-Pro? n
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

first of all thanks for this package.

i have set up quite a few iredmail systems before but this is the first with nginx and centos 7. this time i wanted to do quite a few extras.

so far i have managed to get on top of the different nginx subdomain structure and have them working quite well.

i have also got all the ssl issues sorted with letsencrypt

i also got adminer working so all good there as well

my final problem is with changing the mailbox format to dbox, more specifically sdbox

the reason i want to do this is that i want to store all attachments separate to the emails. ths can only be done using the dbox format, not maildir

i have set up the two configs in 10-mail.conf like so:

mail_location = sdbox:~/dbox

mail_attachment_dir = /mnt/Email_attach/vmail/%d/.attachments

the problem is that any new mailboxes (even with new domains) are still using the maildir format, so obviously the attachments directory is not working either as it is not supported by that format.

i have restarted dovecot and rebooted the server but still no success.

i tried a number of other minor things but none worked. there is not a lot out there on this topic

there does not appear to be any related errors in the logs

i read that the db mail record overrides the mailLocation setting but could not seem to figure out if that is even set.

can anyone offer some help, please.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,793

Re: trying to convert to sdbox mail format

Dovecot gets per-user mailbox format and folder name from SQL/LDAP query, and we improved this in development edition by 2 new SQL columns (for SQL backends) and 2 new LDAP attributes (for LDAP backends).

Check this __DRAFT__ iRedMail upgrade tutorial for details of this change.
https://docs.iredmail.org/upgrade.iredm … db-special

And check this tutorial to see how to switch mailbox format:
https://docs.iredmail.org/change.mailbox.format.html

I suggest you use this change instead of implementing it in your own way, it's should be easy and clear enough, and easy for upgrading iRedMail in the future.

3 Reply by raystrach (edited by raystrach 2018-07-28 19:27:53)

  • raystrach
  • Member
  • Offline
  • Registered: 2012-05-21
  • Posts: 37

Re: trying to convert to sdbox mail format

thanks zhang

i will give it a try and let you know how i went.

4 Reply by raystrach

  • raystrach
  • Member
  • Offline
  • Registered: 2012-05-21
  • Posts: 37

Re: trying to convert to sdbox mail format

hi zhang
i am working my way through this...

after a couple of mistakes (i misread your instruction on the extra line and changed mailbox.maildir , but should have left that alone) i have got it working so that when it is configured it will deliver mail correctly in the right mailbox with the correct format. so all good on that score - thanks.

what is not happening at present is that when i create an account through the admin panel, it defaults back to maildir. unless i was mistaken, i could not see anything to change these values in the database upon creation of the new email account.

can that be done without having to do it for every account, manually?

5 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,793

Re: trying to convert to sdbox mail format

Upcoming iRedAdmin-Pro release will have 2 new setting parameters also:

# Default mailbox format: maildir.
# All Dovecot supported mailbox formats are ok, e.g. mdbox.
# https://wiki2.dovecot.org/MailboxFormat
MAILBOX_FORMAT = 'maildir'

# Default folder used to store mailbox under per-user HOME directory.
# It will be appended to the `mail` variable returned by Dovecot SQL/LDAP
# query. for example, sql query in `/etc/dovecot/dovecot-mysql.conf`:
#
#   user_query = SELECT ...,  CONCAT(...) AS mail, ...
#
# Or LDAP query in `/etc/dovecot/dovecot-ldap.conf`:
#
#   user_attrs      = ...,=mail=%{ldap:mailboxFormat:maildir}:~/%{ldap:mailboxFolder:Maildir}/,...
MAILBOX_FOLDER = 'Maildir'

If you want to use mdbox (or other format besides 'maildir'), you can set it in iRedAdmin-Pro config file and restart uwsgi service to load the change. Then all newly created user accounts will use mdbox format.

raystrach wrote:

i am working my way through this...

I think our implementation and configuration is clear enough and easy to understand and manage, i suggest you follow our tutorial so that you can easily upgrade iRedMail by following our upgrade tutorial in the future if there's any change made to this part.

6 Reply by raystrach

  • raystrach
  • Member
  • Offline
  • Registered: 2012-05-21
  • Posts: 37

Re: trying to convert to sdbox mail format

thanks again.

unfortunately i do not have the pro version of iredadmin, so setting this up there was not an option.

however, after going through quite a number of scripts, i realised that you did not actually insert these values implicitly, but instead you let the default field value be inserted by the database.

that makes sense.

of course that meant that all i needed to do was to change the default field value and it should set things up the way i wanted and that was the case.

so now all that appears to be working perfectly. i will just have to make sure that i check those values whenever an update is done - that's ok.

then the only thing that was not working was the attachments being stored in my chosen directory through the mail_attachment_dir directive in 10-mail.conf.

after trying different things i realised that 10-mail.conf was probably not being used. i then put the directive in the dovecot.conf file and hey presto - it worked!

so to recap for those people who may want to do a similar thing (please note my configuration at the top of this thread)...

after setting up the mailserver in the normal way i did the update as suggested by zhang, above.

i then changed any mailboxes in the user database to the following field values for the mailbox table:
mailboxformat:sdbox //could be sdbox or mdbox - pros and cons for each - previously maildir
mailboxfolder:dbox //this can be anything - your choice - previously Maildir

i then changed the default value for these two fields to the value i input for the current record(s)

i then made the following changes to /etc/dovecot/dovecot.conf :
//this was previously there but i edited it - not 100% sure it does anything - zhang might like to comment

mail_location = sdbox:%Lh/dbox/


//add - this will put attachments in a file for each domain i manage

mail_attachment_dir = /mnt/email_attach/%d/

i restarted dovecot

# /bin/systemctl restart dovecot.service

it all seems to be working fully now although i am happy to be corrected on any points.

this has the advantage of being able to store the bulk of your email content on a separate disk.

also, as i understand it(and this appears correct), each attachment is checked for its uniqueness, so that only once instance of an attachment is saved to the server (i stand to be corrected on this).

the only thing that i am not 100% certain about are the other instances of where maildir is specified in a number of other files. i took it from zhang's instructions that they did not matter

hope this helps someone else as there is not a lot out there on this type of set up.

7 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,793

Re: trying to convert to sdbox mail format

raystrach wrote:

//this was previously there but i edited it - not 100% sure it does anything - zhang might like to comment

mail_location = sdbox:%Lh/dbox/

Dovecot can always get some parameters of mail_location settings from SQL/LDAP query, so the "sdbox:..." part here is useless, but you can still add other settings here, e.g. INDEX.

raystrach wrote:

the only thing that i am not 100% certain about are the other instances of where maildir is specified in a number of other files. i took it from zhang's instructions that they did not matter

Which places?

8 Reply by raystrach

  • raystrach
  • Member
  • Offline
  • Registered: 2012-05-21
  • Posts: 37

Re: trying to convert to sdbox mail format

hi zhang

sorry to take so long to respond. the "%Lh/Maildir" directory or other similar instances can be found in at least the following:

/functions/dovecot.sh
/functions/system_accounts.sh
/etc/postfix/mysql/virtual_mailbox_maps.cf

there are also a number of instances in the samples directory which of course would not affect anything

cheers

9 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,793

Re: trying to convert to sdbox mail format

raystrach wrote:

the "%Lh/Maildir" directory or other similar instances can be found in at least the following:
/functions/dovecot.sh
/functions/system_accounts.sh
/etc/postfix/mysql/virtual_mailbox_maps.cf
there are also a number of instances in the samples directory which of course would not affect anything

- functions/dovecot.sh: it sets default settings, but it's always overridden by the value returned by SQL/LDAP query.
- functions/system_accounts.sh: There's one command used to create mailbox for postmaster@<first-domain>. We have to choose one format for this account while setting up mail server. i think this is acceptable, right?
- /etc/postfix/mysql/virtual_mailbox_maps.cf: since we use Dovecot LDA or LMTP as local mail deliver agent, this file is only used by Postfix to query existing users as verification (to reject email sent to unknown addresses), so the maildir path it returns is never used. But if you set local mail deliver agent to 'virtual' (it's Postfix's builtin transport without quota support and more), this maildir path is used. If you return mailbox format which is not supported by Postfix, postfix cannot deliver the email at all, so this is a conflict. The fortunate thing is, in iRedMail, the returned maildir path is never used to deliver mail.

Hope it helps.

10 Reply by raystrach

  • raystrach
  • Member
  • Offline
  • Registered: 2012-05-21
  • Posts: 37

Re: trying to convert to sdbox mail format

thanks for that zhang

i have noted your observations, made some slight adjustments and now it is all working really well.

it seems a good set up and apparently the dbox format is very good with indexing so it should be fast, with the added bonus of being able to store those big attachments on cheaper storage, especially with identical attachments only held once.

cheers and thanks again.