User Tools

Site Tools


handling_mass-email_reports

Managing Bacula's informative email stream

Problem definition

Issue :
Bacula is great. It can be very informative what is going on. A nice option is that it can give status reports through logs or by email. No one reads logs unless something suspicious is expected due to a bombardment of info, so email is useful. However too much mail is not informative as it tends not be read and therefor it is not useful anymore.
Possible solutions :

  • Do no longer send email
  • Limited the amount of email which is send (Let Bacula be less informative)
  • Implement a mechanism to handle Bacula's email more efficiently so it gets noticed


A side-issue is that when your working-environment consist of multiple people handling Bacula mail gets forked. This seems a bit like a waste of resources. Also that way it is not clear if and when someone has handled the issue.

How to tackle this?

The first 2 options were not real options for me.

Email flow

What next? OK, let's check the flow of the email-reporting-process and determine where we can make a change and where that might lead us.
A general email-process can be described as : MUA ⇒ MTA ⇒ network ⇒ MTA ⇒ MDA ⇒ MUA
In my case Bacula-Director, MTA and MDA are all on one server, so: MUA ⇒ MTA ⇒ MDA ⇒ MUA
Terminology:

  • MUA : Mail User Agent
  • MTA : Mail Transfer Agent
  • MDA : Mail Delivery Agent

How this works can be read here among other places.

To illustrate: Bacula Director ⇒ Bacula bsmtp ⇒ Exim (MTA) ⇒ Cyrus (MDA) ⇒ email-client of choice (Thunderbird)
As we have a Tape-Operator and a Sys Admin and have Bacula messages configured in such a way that some messages go to both, some email is forked at the point of the MTA.

What kind of solution was I looking for?

I wanted:

  • a generic solution so you folks could benefit too
  • a solution that would allow for someone else to step in:
    • historic overview
    • minimal to non-reconfiguration to let this person have the insights of the sysadmin
  • preferably (but not a must) a solution which would limit the amount of forked mail

What I did not want:
Yes, I know, filtering at the mailclient can also be an option, but that would be on a per user basis and would not limit the amount of mail. Nor would it ease adding new sysadmins etc to the game. So that is a no go.

Considerations

So there are various stages in the email-flow-process where email could be managed differently than before

  • Bacula-director : Bacula would be less informative : no-go
  • Bacula's bsmtp tool: would require a rewrite of the tool : preferably not
  • MTA: sounds like an option
  • MDA: mail is already distributed at a per person/user-basis. No change of limiting the number of emailreceivers/requires a per user configuration : preferably not
  • MUA: as described above: I like IMAP mailfiltering, but this is not enough in this case (Exim/Cyrus): no go

Depending on what features are available in your MTA and MDA of choice, the above remarks on MTA and MDA are true or can be viewed a little more subtle.

Drawbacks
  • As mail is filtered and put in shared IMAP folders it can not be forwarded to other (external) mailaddresses. Maybe it can be done somehow but I have not looked at this (yet). So the only place where warnings and errors are collected are in the shared IMAP folders
  • People obviously still have to subscribe to these folders, look at them and react…..

Conclusion

Basically this implementation leads to what I want: multiple people can look into current and past Bacula events. The mails are not duplicated and warning and errors do stick out so should be more easily noted.

Obviously I must admit this is more of a basic description on how to use shared IMAP folders. I just found using it for storing Bacula status emails in a certain way as described a workable solution for my working environment. One could also opt for an entire different approach and skip email warning all together and use something like Syslog Next Generation or probably preverably http://www.rsyslog.com/RSyslogD and apply filters on those logs. If email is still preferred then you can send emails based on those results of course.

Currently two different aproaches based on what was technically possible on different servers were implemented. The Sieve implementation can be easily used in other MDAs supporting Sieve. Of course you could also use procmail or other filtering mechanisms. Please share them.

Final Notes

It could very well be that I made mistakes, that some configuration aspects can be optimized or even discarded. It took me some time to get things working and I am glad it does. Feel free to comment.

With this document I do not claim to have the ultimate solution to handle status emails from Daemons such as Bacula. This is just one approach; I welcome others to submit their alternative solutions, so people reading this wiki can make a solid choice of what to implement and weight the cons and pros of the solutions offered.

handling_mass-email_reports.txt · Last modified: 2010/09/11 00:21 by olaf