github-starred/sendmailanalyzer
2
0
Fork 0
mirror of https://github.com/darold/sendmailanalyzer.git synced 2026-05-15 14:15:56 -06:00
Code Issues 3 Projects Packages Wiki Activity Actions

Fix documentation about MAIL_GW and MAIL_HUB where ip addresses must be used instead of DNS name. Thanks to Roger for the report.

Browse source
This commit is contained in:
Darold Gilles 2015-03-18 11:47:29 +01:00
parent 0ae6cb958a
commit b3530d90e7
2 changed files with 122 additions and 119 deletions
Download patch file Download diff file Expand all files Collapse all files

233
README
View file

@ -2,20 +2,20 @@ NAME
SendmailAnalyzer - Sendmail/Postfix log analyzer
DESCRIPTION
SendmailAnalyzer as is name suggest is a Sendmail log analyzer. It
process maillog files and generate dynamic statistics in HTML and
graphical output. The reports are generated in real time so that it let
you know at any moment what is going on your mail servers. It use time
SendmailAnalyzer as its name suggests is a Sendmail log analyzer. It
processes maillog files and generates dynamic statistics in HTML and
graphical output. The reports are generated in real time so that it lets
you know at any moment what is going on your mail servers. It uses time
(hour, day, month and year views) and cross-linked navigation for easy
use.
SendmailAnalyzer is easy to install and highly configurable to match the
dozen of Sendmail possible configurations. It also support report for
dozen of Sendmail possible configurations. It also supports report for
all the major milter or sendmail filters like SpamAssassin, MailScanner,
Clamav, etc.
Collected data are stored in flat files that are automatically archived
or delete to keep disk space. All reports before the current day are
Collected data is stored in flat files that are automatically archived
or deleted to keep disk space. All reports before the current day are
cached to save system resources and are displayed in the 1 second into
your browser.
@ -33,13 +33,13 @@ DESCRIPTION
POSTFIX SUPPORT
SendmailAnalyzer is a statistical dedicated tool for Sendmail and it is
very good in this task. As many people ask me to have such free tool for
the Postfix MTA, Since release v7.0 SendmailAnalyzer now also support
the Postfix mail.log statistics report.
very good in this task. As many people asked me to have such free tool
for the Postfix MTA, Since release v7.0 SendmailAnalyzer now also
supports the Postfix mail.log statistics report.
Postfix is now fully supported, if you have any issue or unsupported
feature please let me know. Note that as I don't use postfix I may ask
you for log file to reproduce some issues or develop features.
Postfix is now fully supported, if you have any issues or unsupported
features please let me know. Note that as I don't use Postfix I may ask
you for log files to reproduce some issues or develop features.
FEATURES
It reports all you ever wanted to know about email trafic on your
@ -99,10 +99,10 @@ FEATURES
coming from or sent to your internal network / private domain.
Top Statistics
Once you have define in the configuration file the Top Max statistics to
show (25 by default), the Max Recipient for a message (25 by default),
the message Size Max (5Mb by default) you will see the top statistics
of:
Once you have defined in the configuration file the Top Max statistics
to show (25 by default), the Max Recipient for a message (25 by
default), the message Size Max (5Mb by default) you will see the top
statistics of:
* Top sender domain, top sender relay, top sender address.
@ -132,7 +132,7 @@ FEATURES
* Top Postgrey status, relay, senders and recipients.
Note: on daily view you can click on each of the reported element to see
the detailled informations. For example if you follow link on a sender
the detailed information. For example if you follow link on a sender
relay you will see all messages detailled information coming from that
relay. This kind of navigation is only available for the days of the
current month to keep disk space, memory usage and privacy.
@ -156,7 +156,7 @@ FEATURES
* Support parsing of compressed maillog file.
Milter / Filter supported
SendmailAnalyzer support some of the most used milter and filter for
SendmailAnalyzer supports some of the most used milter and filter for
spam and virus filtering. If you don't find yours drop me a line and it
will be included.
@ -183,9 +183,9 @@ FEATURES
New features
If you need new features and support for new/other milters or filters,
let me know. This help a lot to develop a better/useful tool. This piece
of software is widely use at my work (espacially for IT report) but this
reflect only a part of the Sendmail usage.
let me know. This helps a lot to develop a better/useful tool. This
piece of software is widely used at my work (espacially for IT report)
but this reflects only a part of the Sendmail usage.
Internationalization
SendmailAnalyzer can be translated to any language with your
@ -196,8 +196,8 @@ FEATURES
REQUIREMENT
SendmailAnalyzer can work in any platform where Sendmail and Perl could
run. What you need is a modern Perl distribution 5.8.x or more is good
but older version should also work.
run. What you need is a modern Perl distribution - 5.8.x or more is
good, but older versions should also work.
You need the following Perl modules. If they are not yet include in your
OS distribution you can always find them at http://search.cpan.org/
@ -214,7 +214,7 @@ REQUIREMENT
INSTALLATION
Generic install
Here is the generic installation steps, but if you want you can create
Here are the generic installation steps, but if you want you can create
and install your own distribution package, see "Package install" bellow.
1) Unpack the distribution tarball in the desired location as follow:
@ -224,8 +224,9 @@ INSTALLATION
perl Makefile.PL
make && make install
2) Follow the instruction given at the end of install. With this default
install everything will be installed under /usr/local/sendmailanalyzer.
2) Follow the instructions given at the end of install. With this
default install everything will be installed under
/usr/local/sendmailanalyzer.
3) Edit sendmailanalyzer.conf file to customize your SendmailAnalyzer
reports. See the configuration file and CONFIGURATION section bellow for
@ -303,8 +304,8 @@ INSTALLATION
/usr/bin/systemctl restart sendmailanalyzer.service > /dev/null 2>&1 || true
Package install
In the packaging/ directory you will find all scripts and file to
generate binary RPM, Slackware and Debian package. See README in this
In the packaging/ directory you will find all scripts and files to
generate a binary RPM, Slackware and Debian package. See README in this
directory.
Custom install
@ -341,13 +342,13 @@ INSTALLATION
DOCDIR=%{_docdir}/%{uname}-%{version} \
DESTDIR=%{buildroot} < /dev/null
See spec file in packaging/RPM for full RPM build script.
See the spec file in packaging/RPM for the full RPM build script.
USAGE
There's two way to use SendmailAnalyzer. If you don't need real time you
can run it each night so that maillog will be parsed and reports
generated once a day. Note that if you have a huge MTA load this not a
good solution.
There are two ways to use SendmailAnalyzer. If you don't need real time
you can run it each night so that maillog will be parsed and reports
generated once a day. Note that if you have a huge MTA load this is not
a good solution.
The other way is to run it in daemon mode, in this way it can parse huge
maillog (million line per day) preserving system resources.
@ -379,20 +380,20 @@ USAGE
given with the distribution (in start_script/ directory). See the README
file in that directory for more explanation about how to install.
It will start as 'sendmailanalyzer -f' that tell him to start in daemon
It will start as 'sendmailanalyzer -f' that tells it to start in daemon
mode (default), parse full maillog seeking after the last run ending
position (-f) and to open a pipe to a tail command on /var/log/maillog.
It will never end until you kill it or restart it.
To restart sendmailanalyzer use the SIGHUP signal as follow :
To restart sendmailanalyzer use the SIGHUP signal as follows :
/bin/kill -HUP `cat /var/run/sendmailanalyzer.pid`
or
/usr/bin/pkill -HUP sendmailanalyzer
This will force sendmailanalyzer to reread his configuration file and
reopen a pipe to the tail command on you mail log file. The originals
command line arguments that you've given ar startup^will be preserved.
This will force sendmailanalyzer to reread its configuration file and
reopen a pipe to the tail command on you mail log file. The original
command line arguments that you've given at startup will be preserved.
Important: If you have syslog rotate enable (I hope so :-) you will have
to restart SendmailAnalyzer after each log rotation to always tail the
@ -426,28 +427,28 @@ USAGE
process and the pipe to the tail command.
Caching
SendmailAnalyzer collect maillog entries to write datas to flat files,
when you run the CGI script sa_report.cgi it had to read each data files
SendmailAnalyzer collects maillog entries to write data to flat files,
when you run the CGI script sa_report.cgi it has to read each data file
for the given period to compute statistics and output HTML reports. This
can be enought for day views but when you jump to month view it cost a
can be enough for day views but when you jump to month view it costs a
lot in CPU and memory usage unless you have a home MTA.
To speed up things and free system resources you have to run the script
sa_cache each night by cron to create cache files. After that viewing a
month or year view take less than a second.
The script sa_cache must be run by cron as follow:
The script sa_cache must be run by cron as follows:
/usr/local/sendmailanalyzer/sa_cache >/dev/null 2>&1
If you have set per domain report sa_cache will create cache files for
each domains. These cache files are name cache.pm for the MTA global
statistics and cache.pmYOURDOMAIM.DOM for each domain report. To lowered
the memory footprint of the sa_cache program, since version 8.0 it start
computing cache file per hours.
each domains. These cache files are named cache.pm for the MTA global
statistics and cache.pmYOURDOMAIM.DOM for each domain report. To lower
the memory footprint of the sa_cache program, since version 8.0 it
starts computing cache file per hours.
Since version 4.0 sa_report.cgi will warm you to avoid out of memory
when your entering a month view without caching.
when you're entering a month view without caching.
Huge MTA activity
On MTA server with very huge activity you can experience out of memory
@ -464,13 +465,13 @@ USAGE
*/5 * * * * /usr/local/sendmailanalyzer/sa_cache --actual-day-only
It will only parse data stored in the current day so five minutes
interval may be enough for most case.
interval may be enough for most cases.
Database
SendmailAnalyzer store data into flat file database. Data are store in a
time hierarchical directory structure ending at daily level. This
structure is composed as follow : 'mailhost'/year/month/day/ In each day
repository you can find the following data files.
SendmailAnalyzer stores data into flat file database. Data is stored in
a time hierarchical directory structure ending at daily level. This
structure is composed as follows : 'mailhost'/year/month/day/ In each
day repository you can find the following data files:
senders.dat: senders informations.
recipient.dat: recipients informations.
@ -483,28 +484,29 @@ USAGE
auth.dat: SMTP auth message grabbed into the log file.
miltername.dat: message related to a milter, antivir or antispam.
The format of each file is explain in the SendmailAnalyzer code source.
The format of each file is explained in the SendmailAnalyzer code
source.
Archiving
When sa_cache is run and following the value of the FREE_SPACE
configuration option it will try to archive data older than the current
month. If FREE_SPACE is set to 'delete' sa_cache will simply remove the
data file from disk. If you set it to 'archive', sa_cache will build a
gzipped tarball for all daily data file into the corresponding month
directory and the remove date file from disk.
gzipped tarball for all daily data files into the corresponding month
directory and then remove data files from disk.
If you set it to 'none', data file are kept.
If you set it to 'none', data files are kept.
If you're primary concerned in disk space saving set it to 'delete'. If
you want to preserve data for a year or more you can safely set it to
If your primary concern is disk space saving set it to 'delete'. If you
want to preserve data for a year or more you can safely set it to
'archive'. For your information one of my server has 100,000 inbound
message a day and a year of 'archive' storage take around 1Gb and a
messages a day and a year of 'archive' storage take around 1Gb and a
'delete' storage around 250Mb.
One advantage of the 'archive' method is that you can replay the cached
stats (for example after an upgrade to fix a sa_cache bug :-). In this
case, you just have to delete any cache file and extract all tarbal as
follow :
case, you just have to delete any cache file and extract all tarballs as
follows:
find /path/to/SendmailReport/ -name "cache.pm*" | xargs -i rm -f {}
find /path/to/SendmailReport/ -name "history.tar.gz" | xargs -i \
@ -528,14 +530,14 @@ USAGE
CONFIGURATION
The default path to configuration file is /etc/sendmailanalyzer.conf If
you want to change this path, please edit cgi-bin/sa_report.cgi,
sa_cache to match you're need. For sendmailanalyzer use the --config|-c
sa_cache to match your needs. For sendmailanalyzer use the --config|-c
command line argument.
The configuration file consist in a text file with a configuration
The configuration file consists of a text file with a configuration
option in upper case and a value or list of value separated by a tab
character.
Here are the definition of all this configuration directives.
Here are the definitions of all those configuration directives.
System commands options
TAIL_PROG
@ -567,7 +569,7 @@ CONFIGURATION
Path to the maillog file to analyse. Can be overwritten with --log
or -l in sendmailanalyzer args. Default is /var/log/maillog. If the
extension is .gz SendmailAnalyzer will automatically use zcat to
parse the compressed log. For Postix you may use /var/log/mail.log
parse the compressed log. For Postfix you may use /var/log/mail.log
instead.
OUT_DIR
@ -584,9 +586,9 @@ CONFIGURATION
DELAY
Delay in second to flush collected data to disk. Can be overwritten
with --write-delay or -w in sendmailanalyzer args. Default is 60
seconds. During this time data are kept in memory to limit disk I/O
seconds. During this time data is kept in memory to limit disk I/O
and gain speed. If you experience an out of memory on huge mail
server adjust this value to something smaller depending of your
server adjust this value to something smaller depending on your
hardware configuration.
Reporting/display options
@ -600,7 +602,7 @@ CONFIGURATION
HTML_CHARSET
Used to define the HTML charset to use. Default is iso-8859-1, but
with cyrillics character you have to use utf-8 instead.
with cyrillic characters you have to use utf-8 instead.
URL_LOGO
Url to the barorng image. Default: salogo.gif
@ -630,31 +632,31 @@ CONFIGURATION
MBytes.
DOMAIN_REPORT
Compute statistics and cache for a list of domain and display a link
in the front page for a per domain access. See DOMAIN_USER if you
want to grant special access on these pages. You can have multiple
DOMAIN_REPORT lines. If you are running rsyslog with multiple host
use DOMAIN_HOST_REPORT instead. Example:
Compute statistics and cache for a list of domains and display a
link in the front page for a per domain access. See DOMAIN_USER if
you want to grant special access to these pages. You can have
multiple DOMAIN_REPORT lines. If you are running rsyslog with
multiple hosts use DOMAIN_HOST_REPORT instead. Example:
DOMAIN_REPORT domain1.com,domain2.com
DOMAIN_HOST_REPORT
Compute statistics and cache for the given host followed by a list
of domain and display a link in the front page for a per domain
of domains and display a link in the front page for a per domain
access under each host. You can have multiple DOMAIN_HOST_REPORT
lines. See DOMAIN_USER if you want to grant special access on these
lines. See DOMAIN_USER if you want to grant special access to these
pages. For example:
DOMAIN_HOST_REPORT host1 domain1.com,domain2.com
DOMAIN_HOST_REPORT host2 domain2.com,domain3.com
ANONYMIZE
This option allow the anonymization of the output, i-e it remove any
sender/recipient personal information from the report.
This option allows the anonymization of the output, i.e. it removes
any sender/recipient personal information from the report.
REPLACE_HOST
This option replace some hostname in all relay information for
anonymization You must used one REPLACE_HOST line per replacement.
This option replaces some hostname in all relay information for
anonymization. You must use one REPLACE_HOST line per replacement.
REPLACE_HOST internal.relay.dom external.relay.dom
@ -700,7 +702,7 @@ CONFIGURATION
args. Default is 0, go ahead with tail.
MTA_NAME
Syslog name of the MTA. Syslog write it to maillog with the pid as
Syslog name of the MTA. Syslog writes it to maillog with the pid as
... sendmail[1234] ... This is required to only parse relevant
lines. Can be overwritten with --sendmail or -s in sendmailanalyzer
args. Default is sendmail, some distro come with sm-mta instead.
@ -712,42 +714,42 @@ CONFIGURATION
Default: sm-mta|sendmail|postfix
MAILSCAN_NAME
Syslog name of MailScanner. Syslog write it to maillog with the pid
Syslog name of MailScanner. Syslog writes it to maillog with the pid
as ... MailScanner[1234] ... This is required to only parse relevant
lines Can be overwritten with --mailscanner or -m in
sendmailanalyzer args. Default is MailScanner.
AMAVIS_NAME
Syslog name of Amavis. Syslog write it to maillog with the pid as
Syslog name of Amavis. Syslog writes it to maillog with the pid as
... amavis[1234] ... This is required to only parse relevant lines.
Default is amavis.
MD_NAME
Syslog name of MimeDefang. Syslog write it to maillog with the pid
Syslog name of MimeDefang. Syslog writes it to maillog with the pid
as ... mimedefang.pl[1234] ... This is required to only parse
relevant lines based on parsing mimedefang log generated by method
md_graphdefang_log() Default is mimedefang.pl.
CLAMD_NAME
Syslog name of Clamd. When using Mailscanner with clamd if you want
virus report you must configure clamd to log with syslog and use
Syslog name of Clamd. When using Mailscanner with clamd, if you want
virus reports, you must configure clamd to log with syslog and use
LOG_MAIL. Default value is 'clamd' (... clamd[1234] ...). Can be
overwritten with --clamd or -n.
POSTGREY_NAME
Syslog name of Postgrey or sqlgrey. Syslog write Postgrey to maillog
with the pid as follow: ... postgrey[1234] ... and sqlgrey as
follow: ... sqlgrey: ... This is required to only parse relevant
Syslog name of Postgrey or sqlgrey. Syslog writes Postgrey to
maillog with the pid as follows: ... postgrey[1234] ... and sqlgrey
as follow: ... sqlgrey: ... This is required to only parse relevant
logged lines. Can be overwritten with --postgrey or -g. Default is
set to postgrey|sqlgrey
SPAMD_NAME
Syslog name of Spamd. Syslog write it to maillog with the pid as
Syslog name of Spamd. Syslog writes it to maillog with the pid as
follow: ... spamd[1234] ... This is required to only parse relevant
logged lines Can be overwritten with --spamd. Default is spamd.
LOCAL_DOMAIN
Comma separated list of internal domain to be used when
Comma separated list of internal domains to be used when
SendmailAnalyzer is running on a mail host which received message
from any side. SA can't know what message are internal or external
in this case, so the only way to know if a mail come from Internet
@ -770,16 +772,17 @@ CONFIGURATION
LOCAL_HOST_DOMAIN sysloghost2 domain3.com,domain4.com
MAIL_HUB
FQDN comma separated list of internal mail hubs, aka: where email
are redirected if the host is a gateway. For example:
Comma separated ip addresses list of internal mail hubs, aka: where
email are redirected if the host is a gateway. For example:
mailhost.mydom.dom This directive is very important to help
SendmailAnalyzer to find the direction of incoming and outgoing
message.
MAIL_GW
FQDN comma separated list of MTA gateways where external mail comes
from. This directive is very important to help SendmailAnalyzer to
find the direction of incoming and outgoing message.
Comma separated ip addresses list of MTA gateways where external
mail comes from. This directive is very important to help
SendmailAnalyzer to find the direction of incoming and outgoing
message.
DEFAULT_DOMAIN
Default domain or hostname to add to an email address if there's
@ -789,31 +792,31 @@ CONFIGURATION
domain but you may want to change this domain, so use this directive
SPAM_DETAIL
This directive allow report for Spam details. Enable by default.
This allow you to see complete detail of your favorite antispam as
well as score, cache hit and autolearn if your antispam report it.
This directive allows report for Spam details. Enable by default.
This allows you to see complete detail of your favorite antispam as
well as score, cache hit and autolearn if your antispam reports it.
To disable set it to 0, you will save disk space.
SMTP_AUTH
This directive allow report for SMTP authentication. Enable by
This directive allows report for SMTP authentication. Enabled by
default. This allow you to see per authent type (server or client)
user and relay statistics. If you not use SMTP Auth set it to 0 to
disable this feature. These stats are not available in per domain
user and relay statistics. If you do not use SMTP Auth set it to 0
to disable this feature. These stats are not available in per domain
views.
MERGING_HOST
Use this directive to combined multiple mailhost report on a single
report. This allow you to aggregate multiple mailhost that syslogs
to a remote server throught rsyslog to have only one
SendmailAnalyzer report. The value must only use alphanumeric
character as it is used to create subdirectory.
Use this directive to combine multiple mailhost reports on a single
report. This allows you to aggregate multiple mailhost that syslogs
to a remote server through rsyslog to have only one SendmailAnalyzer
report. The value must only use alphanumeric characters as it is
used to create a subdirectory.
SKIP_RCPT_RELAY
Use this to set the recipient relay used for local delivery if your
message appears twice in details view and in messaging, sender and
recipient counter. This is especially right when with postfix
configuraed to have local delivery via dovecot service. Default:
dovecot, that mean that recipient log lines with relay=dovecot will
configured to have local delivery via dovecot service. Default:
dovecot, that means that recipient log lines with relay=dovecot will
instruct sendmailanalyzer to skip those messages.
EXCLUDE_TO
@ -839,22 +842,22 @@ CONFIGURATION
disable this feature.
ADMIN
List of admins username separated by comma that must have full
access to all report. The username is checked again the http
REMOTE_USER environment variable. Default is every one can access,
in this case you may want to add a .htaccess.
List of admin usernames separated by a comma that must have full
access to all report. The username is checked against the http
REMOTE_USER environment variable. By default anyone can access, in
this case you may want to add a .htaccess file.
DOMAIN_USER
List of per user domain access control. The first field is the
username and the second field (separated by tabulation) is a comma
separated list of domain name to be allowed to this user. You could
separated list of domain names to be allowed to this user. You could
add as many lines of DOMAIN_USER as you want in the configuration
file.
ACCESS CONTROL
Access control is based on the REMOTE_USER environment variable stored
by the httpd server during an htaccess Authentication. If this variable
is not set, there is full access for every one.
is not set, there is full access for anyone.
AUTHOR
Gilles Darold <gilles @nospam@ darold.net>

8
doc/sendmailanalyzer.pod
View file

@ -935,15 +935,15 @@ For example:
=item MAIL_HUB
FQDN comma separated list of internal mail hubs, aka: where email are
redirected if the host is a gateway. For example: mailhost.mydom.dom
Comma separated ip addresses list of internal mail hubs, aka: where email
are redirected if the host is a gateway. For example: mailhost.mydom.dom
This directive is very important to help SendmailAnalyzer to find the
direction of incoming and outgoing message.
=item MAIL_GW
FQDN comma separated list of MTA gateways where external mail comes from.
This directive is very important to help SendmailAnalyzer to find the
Comma separated ip addresses list of MTA gateways where external mail comes
from. This directive is very important to help SendmailAnalyzer to find the
direction of incoming and outgoing message.
=item DEFAULT_DOMAIN