PMDF popstore & MessageStore Manager's Guide
PMDF-POP-6.0


Previous | Contents


Chapter 1
Overview

1.1 Introduction

The PMDF popstore is a message store streamlined for use with POP3 clients. It is distinct from the Berkeley and VMS MAIL mail box message stores traditionally used on UNIX and OpenVMS platforms. For a given message, a single copy is stored for all recipients. Moreover, the message is stored in a ready-to-download format; i.e., the server can just map the file into memory and send it down the TCP connection without the need for any pre-processing of the message data as is the case with many stores such as Berkeley and VMS MAIL mail boxes.

The popstore is primarily designed for scaleability. Central database files, a principal cause of bottlenecks in high volume settings, are avoided.¹ In a similar vein, the underlying message store itself may be spread across any number of disks.

The major components of the popstore are described below.

Legacy and popstore POP3 server

A dual-store, multi-threaded POP3 server is provided on UNIX and OpenVMS platforms which supports both the PMDF popstore and the legacy UNIX Berkeley and OpenVMS MAIL mailbox formats.

MessageStore and popstore POP3 server

A new dual-store, multi-threaded POP server is provided on all platforms which supports both the popstore and MessageStore. This server includes security and performance enhancements not possible while maintaining support for legacy mailbox formats.

Poppassd server

A multi-threaded poppassd server for users of Mulberry, Eudora, and other clients which support the ad hoc poppassd protocol for changing passwords.

Web-based user interface

A basic web-based user interface is provided. This interface allows users to use a web-client to change their password as well as see basic usage information about their popstore account. They can also read and delete messages stored for their account. For details, see Chapter 5 .

Web-based management utility

A web-based management utility to manage the popstore. The utility presents itself as a multi-threaded CGI accessed through the PMDF HTTP server. Popstore users with management privileges may use this interface to monitor and manage the popstore. This utility is extremely reconfigurable; the entire interface can be changed around to suit a site's needs. See Chapter 4 for a description of this interface.

Command line management utility

A command line oriented management utility. Users with operating system privileges as well as popstore users who have been granted popstore management privileges may use the utility. See Chapters 6 or 7 for information about this utility.

Migration utility

A utility is provided to migrate the mail inboxes for login accounts to the popstore. The utility can create a popstore account for each migrated user, migrate their mail inbox, and then establish mail forwarding from their login account's message store to the popstore. See Chapter 8 for details about this utility.

Forwarding database

A forwarding database which allows mail for popstore users, fictitious or otherwise, to be automatically redirected elsewhere. Consult Section 1.5 for further details.

Delivery channel

A master channel to deliver inbound messages to the popstore. Inbound messages for the popstore are queued to this channel by PMDF. The channel is then run by PMDF to deliver the messages to the popstore. See Section 10.1 for details.

Message bouncer

This is a job which runs periodically and either returns or deletes old stored messages which have "expired". This channel is best likened to the PMDF RETURN job. This job is used to "time out" old messages which popstore users have not deleted. If an old message has never been read, it is returned as undelivered. If it has been read, it is deleted. Note that the popstore can be configured to never delete old messages and just keep messages around indefinitely. See Section 10.2 for further discussion.

Rewrite rule

A special rewrite rule is used by PMDF so that it can, when presented with a popstore address, immediately check to see if it is valid or not (e.g., is it a valid recipient address, is the recipient allowed to receive new messages, etc.). This allows the various incoming mail streams to reject up front invalid messages for the popstore thereby obviating cases where the message is received only to then have to be bounced. Section 10.1.1 contains additional information on this rewrite rule.

API

An API for sites who want to generate their own management, accounting, billing, logging, etc. facilities. In addition, agents which access the popstore or manipulate user accounts may be written using the API. See Chapter 12 for further details.


Note

¹ Note that the popstore does use some databases. However, these databases are only used to expedite management operations such as listing accounts and maintaining information about management groups. They are not used for non-management operations and therefore do not impact the performance of the popstore.


1.2 Licensing

Although installed as part of the base PMDF-MTA and PMDF-ACCESS products, the PMDF popstore is licensed separately. The popstore may, however, be used without a license: sites without a PMDF-POPSTORE license may create up to ten popstore user accounts plus a default account. A PMDF-POPSTORE license enables a site to create more than ten user accounts.

From the interactive command line management utilities, the show -count_users command (SHOW/COUNT_USERS on OpenVMS) may be used to display the number of currently defined accounts as well as the limit allowed by your license. The web-based management utility can also display this information; see the "License limits" link in the menu at the top of the main page.

1.3 Accounts

Each user of the popstore has a popstore account. Accounts have several key attributes:
Attribute Description
Name The username used to identify the account.
Password The secret used to access the messages stored for the account. Note that passwords are case sensitive. The popstore stores the passwords in an encrypted form.
Group Management group to which the account belongs. Use of management groups is optional.
Quota A storage quota which limits the total amount of messages which an account may store.
Usage flags Flags used to control usage of the account. For example, whether or not the account may receive new messages.
Accounting information Accounting information such as time of last connect, total connect time, current storage, etc.
Accounts are created with either the web-based or command line management utilities. Sites may also develop their own utilities using the API.

Of particular importance is the account name and password. The account name specifies the mailbox for the popstore account. That is, if the popstore has the domain name pop.acme.com then a popstore account with the name jdoe would have the e-mail address

jdoe@pop.acme.com 
Also, the popstore supports the concept of subaddresses. If an address contains a plus sign, "+", in the local part then the plus sign and any characters to the right of it up to the at sign are ignored. For instance, the user jdoe may wish to identify mail they receive from the HBD mailing list by subscribing themselves to that list with the address
jdoe+hbd@pop.acme.com 
Mail coming in to the popstore with that address will then be delivered to the account jdoe and not jdoe+hbd.

POP users access the mail stored for their account by supplying to their POP3 client their popstore account name and password. The rules for account names and passwords are given in Section 1.3.1 and Section 1.3.2 .

Note that there is a special account --- the default account --- which is created at the time that the popstore is configured. The settings for the default account serve as account defaults. When new accounts are created, their defaults are copied from the default account.

The complete list of account fields is given in Table 1-1 . Their interpretation and usage are made clear throughout the remainder of this manual.

Table 1-1 popstore account fields
Field name Description
version Data structure version number indicating what revision of the popstore account data structure is used by the account.
flags Account usage flags.
ulen Length in bytes of the value stored in the username field.
plen Length in bytes of the value stored in the password field. The value stored in the plen field is stored in an encrypted form.
olen Length in bytes of the value stored in the owner field.
slen Length in bytes of the value stored in the private field.
username The account's name. Maximum length of this field is 32 bytes.
password The account's password. Maximum length of this field is 32 bytes. The value stored in this field is encrypted.
owner Information about the account's owner. Maximum length of this field is 40 bytes.
private Site-defined data field. Maximum length of this field is 64 bytes. The contents of this field may be set through the API or either of the management interfaces.
quota The account's primary storage quota measured in bytes. A value of zero indicates unlimited storage quota.
overdraft The account's overdraft quota measured in bytes.
last_billing Time when billing information for the account was last generated. When the account is created, this value is set to the creation time of the account.
total_connections Total number of connections made to the account.
last_connect Time when the user last connected to the account with a POP3 client.
last_disconnect Time when the user last disconnected from the account with a POP3 client.
total_connect Total time, in seconds, spent connected to the account.
past_block_days Storage block days for previously stored and since deleted or returned messages. Does not include storage values for messages currently being held in the store.
past_block_days_remainder Roundoff from the past_block_days field. The roundoff is measured in units of byte minutes.
message_count Count of messages presently stored for the account.
quota_used Total size in bytes of the messages presently being stored for the account.
messages_received Cumulative number of messages which have been stored for the account.
bytes_received Cumulative number of message bytes which have been stored for the account.
glen Length in bytes of the value stored in the group field.
group Management group to which this account belongs. Maximum length of this field is 16 bytes.

1.3.1 Account naming

The popstore supports three different naming schemes: two are case-insensitive while the third is case sensitive. Note that the character set used for account names is controlled with the USERNAME_CHARSET option discussed in Section 3.4 . The choice of character set is only of relevance to USERNAME_STYLE 1 and 2. And, for those two, is only used when doing case-insensitive comparisons of account names (e.g., are the account names Sue and sue identical).

1.3.1.1 The preferred naming scheme

The preferred naming scheme corresponds to an option setting of USERNAME_STYLE=3 in the popstore option file. This option is required by the PMDF MessageStore and will be included in newly generated PMDF configurations.

With this setting,

However, for interoperability with Internet protocols it is safer to confine usernames to alphanumeric characters and
- _ . 

If you wish to have case-insenstive behavior, create all your accounts using lower-case names, and use

$\$U$_ 
in appropriate domain rewriting rules to convert usernames to lower case on message delivery. Also, set TRANSLATE=ASCII-NOCASE in your security.cnf file to convert login usernames to lower case.

1.3.1.2 The default naming scheme

For backwards compatability, the default naming scheme corresponds to an option setting of USERNAME_STYLE=2 in the popstore option file. That option file is described in Chapter 3 .

In the default scheme, account names may be up to 32 characters long and may contain any of the characters from the set

0 1 2 3 4 5 6 7 8 9 . _ - 
a b c d e f g h i j k l m n o p q r s t u v w x y z 
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z 
Since account names are treated as being case insensitive, the names Anne and anne specify the same account.³ Account names should not begin with an underscore character, "_".²

1.3.1.3 Another obsolete naming scheme

A third, and largely obsolete naming scheme corresponds to the option setting of USERNAME_STYLE=1 in the popstore option file. That option file is described in Chapter 3 .

In this naming scheme, account names may contain any printable character in the DEC MCS or any one of the ISO Latin 1 through Latin 9 character sets (ISO 8559-1 through ISO 8859-9). That is, any character with decimal ordinal value in the ranges 33---126 or 161---255 (0x21---0x7E or 0xA1---0xFF in hexadecimal). On UNIX and NT platforms, the account names may be up to 32 characters long; on OpenVMS platforms up to 19 characters. Note that the names may contain spaces and punctuation characters. Again, the names are treated as being case insensitive. The account names should not begin with an underscore character, "_", or contain plus signs, "+".².

When a name is specified, it is converted to an account name using the following five steps: