User Tools

Site Tools


handling_mass-email_reports:exim_cyrus

Filtering at the MTA level

To meet the most of our demands the MTA level seems the best place to manage the email-stream. Within the MTA there are also various stages. So we want to interfere at the point where mail is designated for our server, the mail is not already split at a per user basis.
Some people have implemented hooks at the level where email is already split. At that point classic such as procmail and mailagent can be used. But again, they need per user configuration and I do not want that.

I was looking for a generic solution as not everybody uses the same MTA and MDA, so my thoughts were going towards Sieve as the preferred choice.
In out setup we have Exim as the MTA and Cyrus IMAP Server as the MDA. Both support a dialect :-( of Sieve :-)
As I made up my mind that the preferred level of intervention was the MTA I focused on that. And then a problem came up: although Sieve is supported, it is only supported by Exim for user-filters, not system-filters :-(
So the only way was to implement it in such a way that it would work on our setup but not not copy-paste-able to an environment say like Postfix with Courier. I bit a shame, but there is no other way. Nevertheless I hope this article and the shown configuration are insightful enough so you can implement it on the email-server setup of your choice.

Design

Global Design : Bacula Director ⇒ bsmtp ⇒ Exim → filtering → Exim ⇒ Cyrus shared IMAP folders
I decided to define a structure on Cyrus with the format of: shared.bacula.<servername>.all shared.bacula.<servername>.error shared.bacula.<servername>.warning Were all Bacula messages would go for historic reason go into the “all” folder and messages reporting an error would also be copied into “error” and messages containing a warning would also be copied into “warning”.
That way a new sysadmin or tapeoperator could have an insight of what happened in history and warnings and errors do blink out as they are copied in their own folder.

What do I have to do to make this work?
  • create a shared environment in the IMAP server
  • create shared mailfolders as specified per server (Bacula FD)
  • set the correct ACLs on these folders
  • make the assigned people subscribe to these folders
  • configure the IMAP server to allow posting to these shared folders
  • configure the SMTP server (MTA) to allow posting to these shared folders
  • set ACLs on postings to these shared folders
  • configure system-wide filters to catch Bacula email and deliver them to the correct shared IMAP folder

Implementation

Implemented on Debian Etch with Exim and Cyrus.

Bacula Director

Definition of Messages blocks in my bacula-director.conf file:

# Reasonable message delivery -- send most everything to email address
#  and to the console
Messages {
  Name = Standard
#
# NOTE! If you send to two email or more email addresses, you will need
#  to replace the %r in the from field (-f part) with a single valid
#  email address in both the mailcommand and the operatorcommand.
#  What this does is, it sets the email address that emails would display
#  in the FROM field, which is by default the same email as they're being
#  sent to.  However, if you send email to more than one address, then
#  you'll have to set the FROM address manually, to a single address. 
#  for example, a 'no-reply@mydomain.com', is better since that tends to
#  tell (most) people that its coming from an automated source.
#
  mailcommand = "/usr/lib/bacula/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula: %t %e of %c %
l\" %r"
  operatorcommand = "/usr/lib/bacula/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula: Intervent
ion needed for %j\" %r"
  #mail = bacula@localhost = all, !skipped            
  #operator = bacula@localhost = mount
  mail = bacula-sysman@localhost = all, !skipped            
  operator = bacula-operator@localhost = mount
  console = all, !skipped, !saved
#
# WARNING! the following will create a file that you must cycle from
#          time to time as it will grow indefinitely. However, it will
#          also keep all your messages if they scroll off the console.
#
  append = "/var/lib/bacula/log" = all, !skipped
}
#
# Message delivery for daemon messages (no job).
Messages {
  Name = Daemon
  mailcommand = "/usr/lib/bacula/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula daemon message
\" %r"
  mail = bacula@localhost = all, !skipped            
  console = all, !skipped, !saved
  append = "/var/lib/bacula/log" = all, !skipped
}

IMAP Server configuration

Obviously this is different for every type/brand of IMAP Server. For Cyrus there are some useful sites/pages also in regard to what we want to establish here.
Notably besides standard wiki, howto and docs on Cyrus:

  • An excellent presentation by Naked Ape's Wil Cooley
  • the Exim wiki's page about Cyrus/Exim integration by Andrzej Filip
Create shared mailfolders

This lead to using cyradm issuing:
cm shared
cm shared.bacula
cm shared.bacula.myfirstserver
cm shared.bacula.myfirstserver.all
cm shared.bacula.myfirstserver.warning
cm shared.bacula.myfirstserver.error

Obviously this should be repeated for every server (mysecondserver etc) a Bacula FD runs on.

Set appropriate ACLs

Next the ACLs should be defined using cyradm, gyrus or any suitable other tool of choice.
To quote the Cyrus docs: The identifier “anonymous” refers to the anonymous, or unauthenticated user. The identifier “anyone” refers to all users, including the anonymous user.
So it all depends on the (password)backend of your IMAP server and under which system-account Bacula is running (probably user “bacula” or user “root”) which one to choose.
For the lazy among us:
sam shared.bacula.myfirstserver.all anyone plrs
A restriction on who can post is made later in the configuration of the MTA (see below). For all shared folders ACLs should be defined and can be less exposed. However the user under which Bacula is running *must* be able to post to these folders. Either by specific configuration or by anonymous/anyone ACL configuration like above.
It's wise to let someone administrator the mailbox too (for instance user admin): sam shared.bacula.myfirstserver.all admin lrswipda

MTA configuration

Again, most of the configuration aspects are derived from the Exim wiki's page about Cyrus/Exim integration by Andrzej Filip.
However there are some other aspects which I will document.

Exim system filtering activation

In /etc/exim4/conf.d/main/03_exim4-config_tlsoptions :

system_filter = "/etc/exim4/system.filter"
system_filter_user =  Debian-exim
system_filter_group = Debian-exim
restriction of allowed senders in Exim

In /etc/exim4/conf.d/main/01_exim4-config_listmacrodefs :

domainlist cyrus_domains = localhost : example.org : example.com
addresslist shared_senders = bacula-sysman@example.com : bacula-operator@example.com
transport definition in Exim

In /etc/exim4/conf.d/transport/35_exim4-config-cyrus_delivery :

cyrus_delivery:
  debug_print = "T: cyrus_ltcp for $local_part@$domain"
  driver = smtp
  protocol = lmtp
  hosts = localhost
  allow_localhost

Note: Exim requires LMTP over TCP for shared IMAP folders. Sockets do not do the trick here. You can opt for anonymous LMPT delivery or not.

router definition in Exim

In /etc/exim4/conf.d/router/666_exim4-config_local_shared :

local_shared:
  debug_print = "R: local_shared for $local_part@$domain"
  domains = +cyrus_domains
  senders = +shared_senders
  driver = accept
  local_parts = \N^\+shared\..+\N
  transport = LOCAL_DELIVERY
ACL definition in Exim

In /etc/exim4/conf.d/acl/30_exim4-config_check_rcpt :

accept domains     = +cyrus_domains
       local_parts = \N^\+shared\..+\N
endpass

The actual filtering process

When looking at the actual filter things get interesting. I found some documentation on the subject of which messages can be send in the bacula Developers manual, however this is not very useful for two reasons:

  • status/errorcodes are not available at the state we are intercepting the mails
  • mail subjects can be user defined (see config of Bacula Director)

Yet I saw at least 16 different types of email from Bacula in my mailboxes. Especially messages with header subject “Bacula daemon message” can be all kinds of messages.

Exim systemwide filter for Bacula

Exim Filter Specification shows how to define filters. Emails to shared IMAP folders should have to format of: +shared.level1.level2@host
The example below shows the filering for one particular server. These definitions should be copied and adapted for other servers or be made more generic or comples.

In /etc/exim4/system_filter :

# Exim filter
####################################################################################
### system filtering : Bacula ######################################################
# Bacula message filtering to shared IMAP folders
# filtering based on headers although content is probably more reliable
# to maintain some kind of compatibility with Sieve
# (Sieve is not supported by Exim MTA as systemfilter, only as personal filter)
# Client: localhost
if $sender_address: matches "bacula@localhost" or   
   $sender_address: matches "bacula@myfirstserver" or
   $sender_address: matches "bacula-operator@example.com" or 
   $sender_address: matches "bacula-operator@localhost" or
   $sender_address: matches "bacula-operator@myfirstserver" or
   $sender_address: matches "bacula-sysman@example.com" or 
   $sender_address: matches "bacula-sysman@localhost" or
   $sender_address: matches "bacula-sysman@myfirstserver" then
  
  # message : "Intervention Needed"
  if $header_subject: contains "Intervention needed" then
    #deliver "+shared.bacula.myfirstserver.warning@example.com"
    deliver "+shared.bacula.myfirstserver.warning"
  endif
  
  # message : "Verify Differences"
  if $header_subject: contains "Verify Differences" then
    #deliver "+shared.bacula.myfirstserver.warning@example.com"
    deliver "+shared.bacula.myfirstserver.warning"
  endif
  # message : "Error"
  if $header_subject: contains "Error" then
    #deliver "+shared.bacula.myfirstserver.error@example.com"
    deliver "+shared.bacula.myfirstserver.error"
  endif
  # message : "Canceled"
  if $header_subject: contains "Canceled" then
    #deliver "+shared.bacula.myfirstserver.warning@example.com"
    deliver "+shared.bacula.myfirstserver.warning"
  endif
  # message : "Error"
  if $header_subject: contains "Error" then
    #deliver "+shared.bacula.myfirstserver.error@example.com"
    deliver "+shared.bacula.myfirstserver.error"
  endif
  # message : "Bacula daemon message"
  if $header_subject: contains "Verify Differences" then
    #deliver "+shared.bacula.myfirstserver.warning@example.com"
    deliver "+shared.bacula.myfirstserver.warning"
  endif
  # message : "Error"
  if $header_subject: contains "Error" then
    #deliver "+shared.bacula.myfirstserver.error@example.com"
    deliver "+shared.bacula.myfirstserver.error"
  endif
  # deliver to the "all" shared folder for historical reasons (logging/backtracing)
  #deliver "+shared.bacula.myfirstserver.all@example.com"
  deliver "+shared.bacula.myfirstserver.all"
  # now drop the message as it is already delivered to the right places
  #seen finish
endif
####################################################################################
handling_mass-email_reports/exim_cyrus.txt · Last modified: 2010/09/11 00:09 by olaf