cbpolicyd Greylisting on Zimbra

Greylisting refuses mail from a sender it has not seen before, temporarily, and waits to see whether they come back. A real mail server queues the message and retries a few minutes later. A lot of spam software does not, because retrying costs time and state that a throwaway sender does not want to spend.

It is the most effective and the most disruptive thing in cbpolicyd, in roughly equal measure.

Part of the series that starts with cbpolicyd on Zimbra: An Introduction.

Enable it

zmprov ms $(zmhostname) zimbraCBPolicydGreylistingEnabled TRUE
zmmtactl restart

There is also a training mode, which records what greylisting would have done without actually deferring anything:

zmprov ms $(zmhostname) zimbraCBPolicydGreylistingTrainingEnabled TRUE

Run in training mode first. It costs you nothing and tells you how much of your real mail would have been delayed.

How the triplet works

Greylisting tracks a triplet: the client address, the sender and the recipient. The first time cbpolicyd sees a given combination it returns a 4xx and records it. If the same triplet comes back after GreylistPeriod seconds, it is accepted and remembered.

greylisting_tracking holds those triplets, with FirstSeen, LastUpdate, Tries and Count.

The settings

The greylisting table has more columns than any other module, and NULL in any of them means inherit rather than off.

Column	What it does
UseGreylisting	Actually greylist, 1 or 0
GreylistPeriod	Seconds a new triplet must wait before being accepted
Track	What to key on, normally SenderIP:/24
GreylistAuthValidity	How long a proven triplet stays proven
GreylistUnAuthValidity	The same for triplets that never came back
UseAutoWhitelist	Promote well-behaved servers automatically
AutoWhitelistPeriod	How far back to look when deciding
AutoWhitelistCount	How many good triplets before whitelisting
AutoWhitelistPercentage	What share of them must have been good
UseAutoBlacklist	The same machinery in reverse

Track should be SenderIP:/24, not a single address. Large senders deliver from a pool of machines, so the retry frequently arrives from a different IP than the first attempt. Keyed on the exact address, that looks like a brand new triplet and the mail is deferred again, and again. Masking to a /24 treats the pool as one sender.

A working configuration

INSERT INTO policies (Name, Priority, Description, Disabled)
  VALUES ('Greylisting inbound', 25, 'Greylist unauthenticated inbound mail', 0);

INSERT INTO policy_members (PolicyID, Source, Destination, Disabled)
  VALUES ((SELECT ID FROM policies WHERE Name = 'Greylisting inbound'),
          '$-',                  -- unauthenticated sessions only
          '%internal_domains',   -- to our own domains
          0);

INSERT INTO greylisting
  (PolicyID, Name, UseGreylisting, GreylistPeriod, Track,
   GreylistAuthValidity, GreylistUnAuthValidity,
   UseAutoWhitelist, AutoWhitelistPeriod, AutoWhitelistCount, AutoWhitelistPercentage,
   Disabled)
  VALUES
  ((SELECT ID FROM policies WHERE Name = 'Greylisting inbound'),
   'Inbound greylisting',
   1,
   240,          -- 4 minutes before a retry is accepted
   'SenderIP:/24',
   2592000,      -- proven triplets stay proven for 30 days
   14400,        -- unproven ones expire after 4 hours
   1,            -- auto-whitelist good senders
   604800,       -- looking back over a week
   50,           -- after 50 triplets
   80,           -- of which 80% behaved
   0);

The $- in Source is the important part. It restricts this to unauthenticated sessions, so your own users submitting mail are never greylisted. Greylisting an authenticated user means their client shows a send failure, which is unacceptable.

Whitelisting senders that retry badly

Some large senders retry from unpredictable networks or with long backoffs. Put them in greylisting_whitelist, which takes a CIDR or a reversed hostname pattern:

INSERT INTO greylisting_whitelist (Source, Comment, Disabled)
  VALUES ('209.85.128.0/17', 'Google', 0);

INSERT INTO greylisting_whitelist (Source, Comment, Disabled)
  VALUES ('40.92.0.0/15', 'Microsoft', 0);

Maintaining that list by hand is a real ongoing cost, and it is the main reason people eventually turn greylisting off. The auto-whitelist settings above exist to reduce it, by promoting servers that have proven themselves.

Watching it work

-- triplets currently waiting
SELECT TrackKey, Sender, Recipient,
       datetime(FirstSeen, 'unixepoch')  AS FirstSeen,
       datetime(LastUpdate, 'unixepoch') AS LastUpdate,
       Tries, Count
  FROM greylisting_tracking
 ORDER BY LastUpdate DESC LIMIT 20;

-- servers promoted automatically
SELECT TrackKey, datetime(Added, 'unixepoch') AS Added,
       datetime(LastSeen, 'unixepoch') AS LastSeen
  FROM greylisting_autowhitelist ORDER BY LastSeen DESC LIMIT 20;

A healthy system shows triplets with Count climbing and a steadily growing autowhitelist. Triplets with high Tries and Count at zero are senders being deferred repeatedly and never getting through, which is what you are looking for when someone reports missing mail.

The trade-offs, honestly

Every new correspondent is delayed. The first message from anyone you have never heard from arrives minutes late, and that includes password resets, one-time codes, order confirmations and the message from the person you are about to interview. Users notice this, and they report it as the mail server being broken.

Some senders never retry. A poorly written notification system or a transactional provider with no queue will simply drop the message. You will not find out; the sender will not tell you. This is the risk that matters, and it falls on exactly the automated mail people most rely on.

It is less effective than it was. Greylisting worked spectacularly when spam came from simple software. Modern spam largely comes through compromised accounts and real mail servers that retry correctly, and passes straight through.

Zimbra already filters. Amavis and SpamAssassin run on every message. Greylisting adds a layer in front of them that costs latency rather than CPU, and the question is whether your remaining spam problem justifies it.

Given all of that, greylisting is worth trying when spam is getting through despite filtering, and worth skipping when it is not. Run training mode for a fortnight, look at what it would have delayed, and decide with data.

To turn it off again:

zmprov ms $(zmhostname) zimbraCBPolicydGreylistingEnabled FALSE
zmmtactl restart

Next in this series

cbpolicyd SPF Checking, which decides on sender authorisation during the session rather than after delivery.