INSTALL smtp-poplock v2.06 Overview -------- I've tried to error on the side of detail in this section. (-; I know how hard installing badly documented software can be. Also, don't be turned off by the volume of the text below, the installation process is actually rather easy. Step by Step Outline -------------------- (1) Un-tar the files into /usr/src/smtp-poplock-2.xx (2) Install the package files (3) Add in the authorization logging modules (4) Configure the logging tools (a) The POP3 program writes to syslogd which writes to a fifo read by readog. (b) The POP3 program writes to syslogd which writes to a file. This file is then monitored by "tail -f" which sends its output to readlog. (c) The POP3 program writes to a file which is monitored by "tail -f" which sends its output to readlog. (d) The POP3 program writes to a fifo which is directly read by readlog. (5) Configure cron to run fifo-safety (if necessary) (6) Setup syslogd to log to a fifo (if necessary) (7) Add /usr/sbin/readlog to the system startup scripts (a) Use the /etc/rc.d/rc?.d system. (b) Use the /etc/rc.d/rc.local script. (8) Add /usr/sbin/relaylock as a wrapper to qmail-smtp. (a) qmail-smtp running from tcpserver under supervise (b) qmail-smtp running from inetd (9) Setup the control/rcpthosts file. (10) Create log parsing anonymous sub in smtp-poplock.conf (11) Test the setup from end to end. (12) Report success Step by step details -------------------- (1) Un-tar the files into /usr/src/smtp-poplock-2.xx, for example: $ cd /usr/src $ ungzip smtp-poplock-2.xx.tar.gz -c | tar xf - $ cd smtp-poplock-2.xx (2) Install the package files: $ make install (3) Add in the authorization logging modules If you are using qmail-pop3d, modify the inetd or tcpserver setup so that it will log user authentication to the syslogd device or a file. The qmail POP3 server is commonly invoked as qmail-popup CHANGEME checkpassword qmail-pop3d Maildir with CHANGEME replaced by the fully qualified domain name of the local host. You want to change this to: (all on one line) ----- /var/qmail/bin/qmail-popup CHANGEME /var/qmail/bin/tcp-env /usr/sbin/logpopauth-pre /PATH/TO/YOUR/checkpassword /usr/bin/logpopauth-post /var/qmail/bin/qmail-pop3d Maildir ----- (This can be either in your inetd.conf file or in the tcpserver command depending on your setup.) The default configuration is to log the data to syslogd. See the $logpopauth_where configuration in /etc/smtp-poplock.conf. (4) Configure the logging tools Time to make some decisions about how smtp-poplock interfaces into your system and edit the /etc/smtp-poplock.conf file. First, go and read the /etc/smtp-poplock.conf file for the internal documentation in it, so you know what the heck I'm talking about. There are a few different ways to setup the system. These are based on the various ways that the POP3/IMAP server may log information and that non all syslogd programs can log to a fifo. Here is a narrative to help you figure out which setup out of the four mentioned below to use: First, lets assume your POP3 program writes to syslogd, which is quite probable. If so, then you want to setup syslogd to log the mail to a fifo in addition to the file. Readlog then reads this pipe to get the data. This is option (a). If your syslogd is not capable of logging to a fifo or you don't like this, then use a "tail -f" process to monitor the mail logfile being written to by syslogd. This is not as preferable as a pipe, because "tail -f" is not a salient process and can sometimes cause a few second delay. If you are using "tail -f" special attention has to be given to when your log is rotated. (More on how to do this in step seven.) This is option (b). Possibly your POP3/IMAP server does not log to syslogd, but logs to a file. If so, just make readog monitor this file with a "tail -f" command. This is option (c). However, the same concerns stated above about "tail -f" exist, so it would be nice to make the POP3/IMAP server log to a file which is a fifo being read by readlog. Note that this will throw away any logging data. (Or you could modify readlog to write it back out again, or create a pipe to the tee command.) This is option (d). Option (d) is best used when you are using qmail-pop3d and don't really care about storing the POP3 access logs. It's a cool setup because the data does not have to pass _through_ syslogd, which is often slow and clunky. But it's a direct pipe. Okay, here are the summaries of the different setup options, with the configuration detailed: (all $logpopauth_where examples assume you are using qmail-pop3d) (a) The POP3 program writes to syslogd which writes to a fifo read by readog. $readfile = "/var/smtp-poplock/fifo"; $readfile_isfifo = 1; $logpopauth_where = "| /var/qmail/bin/splogger qmail-pop3d"; Configure syslog to write to fifo: YES Run fifo-safety every min from cron: YES Requires handling of log rotation: NO (b) The POP3 program writes to syslogd which writes to a file. This file is then monitored by "tail -f" which sends its output to readlog. $readfile = "tail -f /var/log/maillog |"; $readfile_isfifo = 0; $logpopauth_where = "| /var/qmail/bin/splogger qmail-pop3d"; Configure syslog to write to fifo: NO Run fifo-safety every min from cron: NO Requires handling of log rotation: YES (c) The POP3 program writes to a file which is monitored by "tail -f" which sends its output to readlog. $readfile = "tail -f /var/log/POP3program-logfile |"; $readfile_isfifo = 0; $logpopauth_where = "/var/log/pop3program-logfile"; Configure syslog to write to fifo: NO Run fifo-safety every min from cron: NO Requires handling of log rotation: YES (d) The POP3 program writes to a fifo which is directly read by readlog. $readfile = "/var/smtp-poplock/fifo"; $readfile_isfifo = 1; $logpopauth_where = "/var/smtp-poplock/fifo"; Configure syslog to write to fifo: NO Run fifo-safety every min from cron: YES Requires handling of log rotation: NO Configure your POP3 program to log to the file /var/smpt-poplock/fifo. Steps five and six directly below will be dependent on which option you chose above. (5) Configure cron to run fifo-safety If you chose option (a) or (d) in step four, you will need to setup fifo-safety to run every minute from cron to protect the fifo from filling up and blocking. To make cron run fifo-safety as root once a minute. I added the following to my /etc/crontab file: $ vi /etc/crontab and insert: ----- # below line for smtp-poplock package * * * * * root fifo-safety ----- However, your crond might be different. This works for Vixie Cron which is the most popular cron daemon. Check the crond(8) manpage to see if you have a Vixie Cron. (6) Setup syslogd to log to a fifo If you chose option (a) in step three, you will need to setup syslogd to log to a fifo. To make syslog do log to a fifo on my system (RH Linux), I added the following line to the system logger configuration file: $ vi /etc/syslog.conf and insert: ----- # Send mail message to fifo for use in SMTP access filtering mail.* |/var/smtp-poplock/fifo ----- If you are running FreeBSD, then things are slightly different. Check in the INSTALL.systype file for user-reported notes specific to FreeBSD and other operating systems. Consult your man page for detailed documentation on your syslogd. Remember to HUP the syslogd process so that it re-reads the configuration file. $ kill -HUP ` cat /var/run/syslogd.pid ` (7) Add /usr/sbin/readlog to the system startup scripts so that it will start when the system boots. This can be done with or without supervise program from the daemontools package by Dan Bernstein, the author of qmail. I recomend getting the daemaontools package and using supervise to run readlog and tcpserver to run qmail-smtpd later on. Okay, enough pushing DJB software. If you want daemontools, you can get it from: ftp://koobera.math.uic.edu/www/daemontools.html IMPORTANT NOTE: This included scripts that use supervise only work for the daemontools package version 0.53 and not later versions. In version 0.60 Dan went and greatly changed how supervise works (http://cr.yp.to/daemontools/upgrade.html). (If you would like to update these scripts for the latest version of supervise, feel free to send me a patch.) There are two ways I will list here to have readlog started in the system startup. I ended up using the first solution, because my system logger is started through the same mechanism. I think it makes sense to have readlog start at the same time as syslogd, as they are tightly bound through the fifo. I have readlog and syslogd starting consecutively in the system boot. This makes me feel good, but is not required. (-; (a) Use the /etc/rc.d/rc?.d system. This makes readlog start from the /etc/rc.d/rc?.d system. Note that the first command is dependent on if you have the daemontools package with supervise installed. $ mkdir /var/run/readlog # with supervise $ cp readlog-starter_supervise /etc/rc.d/init.d/readlog # with supervise $ cp readlog-starter_nosupervise /etc/rc.d/init.d/readlog # without supervise $ chmod 755 /etc/rc.d/init.d/readlog $ /sbin/chkconfig --add readlog Now that it is installed in the system start scripts, go and start readlog manually: $ /etc/rc.d/init.d/readlog start (b) Use the /etc/rc.d/rc.local script. $ mkdir /var/run/readlog # with supervise $ cp readlog-starter_supervise /usr/sbin/readlog-starter # with supervise $ cp readlog-starter_nosupervise /usr/sbin/readlog-starter # without supervise $ chmod 755 /usr/sbin/readlog-starter $ vi /etc/rc.d/rc.local and insert: ----- /usr/sbin/readlog-starter start ----- Now that it is installed in the system start scripts, go and start readlog manually: $ /usr/sbin/readlog-starter start If you chose either option (b) or (c) in step four, where "Requires handling of log rotation: YES" is noted, then special setup is required to make sure that readlog is restarted after the logfile you are running a "tail -f" on is rotated. On a Red Hat Linux system, if you are running a "tail -f" of /var/log/messages add a command to restart readlog (this command depends on if you used option (a) or (b) above to start readlog) to the postrotate section for /var/log/messages in the /etc/logrotate.d/syslog file. Specifically, /etc/logrotate.d/syslog should have a section that looks like this: If using option (a) from this step, "use the /etc/rc.d/rc?.d system": ----- /var/log/messages { postrotate /usr/bin/killall -HUP syslogd /etc/rc.d/init.d/readlog restart endscript } ----- If using option (b) from this step, "Use the /etc/rc.d/rc.local script": ----- /var/log/messages { postrotate /usr/bin/killall -HUP syslogd /usr/sbin/readlog-starter restart endscript } ----- These are only examples. Customize this to have readlog restarted whenever the file it is tailing is rotated. This file may be something other than /var/log/messages, and my be rotated through a different system. Make sure you tie into whatever system is being used for the log rotation. This is important. Restarting readlog is required for a new "tail -f" process to be started and start looking at the new file syslog is writing to. If you don't do this the existing "tail -f" will continue to monitor the old file to which nothing is being written. (8) Add /usr/sbin/relaylock as a wrapper to qmail-smtp. Now, you might be running qmail-smtp from inetd or from tcpserver. Because of this, there are two different ways to do this step. For more info about running qmail-smtpd see: ftp://koobera.math.uic.edu/www/qmail/faq/servers.html#tcpserver-smtpd NOTE: Any logging done by qmail-smtp is ignored by smtp-poplock. (a) qmail-smtp running from tcpserver under supervise IMPORTANT NOTE: The included scripts that use supervise only work for the daemontools package version 0.53 and not later versions. In version 0.60 Dan went and greatly changed how supervise works (http://cr.yp.to/daemontools/upgrade.html). (If you would like to update these scripts for the latest version of supervise, feel free to send me a patch.) Here, you need to run the following command in the system startup which starts up a supervised tcpserver which will fork off the relaylock wrapper to qmail-smtpd: 1) Using splogger: (part of the qmail package) ----- supervise /var/run/qmail-smtpd \ tcpserver \ -u ` perl -e 'print scalar getpwnam qmaild' ` \ -g ` perl -e 'print scalar getgrnam nofiles' ` \ -c 40 \ -v \ 0 smtp \ /var/qmail/bin/tcp-env \ relaylock \ /var/qmail/bin/qmail-smtpd \ 2>&1 | /var/qmail/bin/splogger smtpd 3 & ----- NOTE: the included smtp-starter script uses this first method with smtpd logging going through the syslogd. If you use option two, edit that file. 2) Using cyclog: (part of the daemontools package) ----- supervise /var/run/qmail-smtpd \ tcpserver \ -u ` perl -e 'print scalar getpwnam qmaild' ` \ -g ` perl -e 'print scalar getgrnam nofiles' ` \ -c 40 \ -v \ 0 smtp \ /var/qmail/bin/tcp-env \ relaylock \ /var/qmail/bin/qmail-smtpd \ 2>&1 | setuser qmaill accustamp | \ setuser qmaill cyclog -s5000000 -n5 \ /var/log/qmail/qmail-smtpd & ----- This can be run from the system startup scripts from /etc/rc.d/rc.local or from the /etc/rc.d/rc?.d system. 1) /etc/rc.d/rc?.d system $ mkdir /var/run/qmail-smtpd $ install -m 755 smtpd-starter /etc/rc.d/init.d/smtpd $ /sbin/chkconfig --add smtpd $ /etc/rc.d/init.d/smtpd start 2) /etc/rc.d/rc.local file $ mkdir /var/run/qmail-smtpd $ install -m 755 smtpd-starter /usr/sbin $ vi /etc/rc.d/rc.local and insert: ------ /usr/sbin/smtpd-starter start ------ $ /usr/sbin/smtpd-starter start (b) qmail-smtp running from inetd This also works just as well, but can cause problems under high load because some inetd versions just can't handle it. Also this does not set a top limit on the number of simultaneous qmail-smtpd processes. To install this do the following: $ vi /etc/inetd.conf and insert: (all on one line) ----- smtp stream tcp nowait qmaild /var/qmail/bin/tcp-env tcp-env /usr/sbin/relaylock /var/qmail/bin/qmail-smtpd ----- One user has reported that /usr/sbin/tcpd is required on Red Hat 5.2, so you might need the following: (all on one line) ----- smtp stream tcp nowait qmaild /usr/sbin/tcpd tcpd /var/qmail/bin/tcp-env /usr/sbin/relaylock /var/qmail/bin/qmail-smtpd ----- (9) Setup the control/rcpthosts file. The qmail control file /var/qmail/control/rcpthosts specifies which domains qmail-smtpd will receive mail for. I.e. qmail-smtpd will reject any envelope recipient address with a domain not listed in rcpthosts. However, the environment variable RELAYCLIENT overrides this and allows relaying. This is the variable that we set in the relaylock wrapper to specify that a connection is allowed to relay. Russ Nelson put a tip up on www.qmail.org on a way you can make a rcpthosts file: $ cd /var/qmail/control $ sed 's/:.*//' < virtualdomains | cat - locals | sort > rcpthosts Then, all you need to do, is to add any hosts that you are a secondary MX for, or you receive mail for, then forward via smtproutes. ( I got this tip from http://qmail-docs.surfdirect.com.au/docs/qmail-antirelay.html which is a good file on the basics of configuring qmail as a selective relay. ) WARNING: If you don't setup this file everyone will be able to relay anyway. If you setup this file with a subset of the domains you receive mail for, then your mail server will reject that mail. Make sure you do this right. NOTE: If you have more than 50 domains in control/rcpthosts, consider using control/morercpthosts and the qmail-newmrh program. See the qmail-smtpd main page for more info. (10) Create log parsing anonymous sub in smtp-poplock.conf It is possible that that default log-line parsing built into readlog will not work with your POP3 server. When this happens, just write your own parsing subroutine in the smtp-poplock.conf file. It is the parse_log_sub configuration command. How will you know that the default log parsing is not working? Well, the default parsing uses the following perl regular expressions: /(?:pop3|imap).*\Wlogin\D.*(\d+\.\d+.\d+.\d+)/i /qmail-pop3d authenticated host:\s+(\S+)/i where $1 becomes the name of the authenticated host. Look at the log lines produced by your POP3/IMAP server and see if they match these lines. Or, if you don't know perl regular expressions, just proceed onto step eight where you will test the system end to end. If things don't work, find someone who knows perl regular expressions to help you out. You may find a working log parsing sub in the file INSTALL.parse_log_sub which contains log parsing subs contributed by users. IMPORTANT: If you create a new parse_log_sub to support your server, e-mail it to dharris@drh.net, so it can be included in the INSTALL.parse_log_sub file. Make sure to tell me the name and version of your POP3/IMAP server and what system you are running, so that others can benefit from the work. (11) Test the setup from end to end. Make sure that: (1) you can't relay mail for an IP address that is not in the "authorized hosts database", (2) hosts which authenticate via POP or IMAP are added to the database, and (3) hosts that are in the database can relay mail. A useful tool is showallowed, which shows the current hosts in the database. For example: $ showallowed ipaddr access window expires ---------------- ---------------------------- 24.7.44.201 Sat Jan 9 00:24:08 1999 Also, you can clear out the database by running the following command: $ > /var/smtp-poplock/dbfile If you run into any problems that you think are system dependent send me an e-mail at dharris@drh.net, and I'll see if we can get it working. (12) Additional configuration. Now that the system is working properly, there are a few other things you can configure. The /etc/smtp-poplock.satic_allowed file lets you enter ipaddresses that will always be allowed to relay. They do not require pop authentication but rather are trusted from their ipaddress. You might use this to allow your internal network to relay. Specify addresses on per line in the /etc/smtp-poplock.static_allowed file. Specify subnets as either "192.168.0.0/255.255.255.0" or "192.168.0.0/24". (13) Report success $ ( echo "smtp-poplock working by YOUR NAME on YOUR SYSTEM TYPE"; uname -a ) \ | mail smtp-poplock-success@davideous.com Replace "YOUR SYSTEM TYPE" with the most reasonably specific description of your OS that you have. Such as "Red Hat Linux 5.2" or "FreeBSD x.x". The problem is there are differences between various versions which are not listed with "uname -a". IMPORTANT: If you solved any interesting problems where the information would be helpful to others, please e-mail me at dharris@drh.net, and I'll add the information to the INSTALL file or the INSTALL.systype file. This is important because there is no way that I can test this on all the system types out there. I need your help to make this package run smoothly for everyone out there.