Configuring Inbound Email

Phorge Administrator and User Documentation (Configuration)

This document contains instructions for configuring inbound email, so users may interact with some Phorge applications via email.

Preamble

Phorge can process inbound mail in two general ways:

Handling Replies: When users reply to email notifications about changes, Phorge can turn email into comments on the relevant discussion thread.

Creating Objects: You can configure an address like bugs@yourcompany.com to create new objects (like tasks) when users send email.

In either case, users can interact with objects via mail commands to apply a broader set of changes to objects beyond commenting. (For example, you can use !close to close a task or !priority to change task priority.)

To configure inbound mail, you will generally:

• Configure some mail domain to submit mail to Phorge for processing.
• For handling replies, set metamta.reply-handler-domain in your configuration.
• For handling email that creates objects, configure inbound addresses in the relevant application.

See below for details on each of these steps.

Configuration Overview

Usually, the most challenging part of configuring inbound mail is getting mail delivered to Phorge for processing. This step can be made much easier if you use a third-party mail service which can submit mail to Phorge via webhooks.

Some available approaches for delivering mail to Phorge are:

The remainder of this document walks through configuring Phorge to receive mail, and then configuring your chosen transport to deliver mail to Phorge.

Configuring "Reply" Email

By default, Phorge uses a noreply@phorge.example.com email address as the "From" address when it sends mail. The exact address it uses can be configured with metamta.default-address.

When a user takes an action that generates mail, Phorge sets the "Reply-To" address for the mail to that user's name and address. This means that users can reply to email to discuss changes, but: the conversation won't be recorded in Phorge; and users will not be able to use email commands to take actions or make edits.

To change this behavior so that users can interact with objects in Phorge over email, change the configuration key metamta.reply-handler-domain to some domain you configure according to the instructions below, e.g. phorge.example.com. Once you set this key, email will use a "Reply-To" like T123+273+af310f9220ad@phorge.example.com, which -- when configured correctly, according to the instructions below -- will parse incoming email and allow users to interact with Differential revisions, Maniphest tasks, etc. over email.

If you don't want Phorge to take up an entire domain (or subdomain) you can configure a general prefix so you can use a single mailbox to receive mail on. To make use of this set metamta.single-reply-handler-prefix to the prefix of your choice, and Phorge will prepend this to the "Reply-To" mail address. This works because everything up to the first (optional) '+' character in an email address is considered the receiver, and everything after is essentially ignored.

Configuring "Create" Email

You can set up application email addresses to allow users to create objects via email. For example, you could configure bugs@phorge.example.com to create a Maniphest task out of any email which is sent to it.

You can find application email settings for each application at:

Home → Applications → Select an Application → Configure

Not all applications support creating objects via email.

In some applications, including Maniphest, you can also configure Herald rules with the [ Content source ] and/or [ Receiving email address ] fields to route or handle objects based on which address mail was sent to.

You'll also need to configure the actual mail domain to submit mail to Phorge by following the instructions below. Phorge will let you add any address as an application address, but can only process mail which is actually delivered to it.

Security

The email reply channel is "somewhat" authenticated. Each reply-to address is unique to the recipient and includes a hash of user information and a unique object ID, so it can only be used to update that object and only be used to act on behalf of the recipient.

However, if an address is leaked (which is fairly easy -- for instance, forwarding an email will leak a live reply address, or a user might take a screenshot), anyone who can send mail to your reply-to domain may interact with the object the email relates to as the user who leaked the mail. Because the authentication around email has this weakness, some actions (like accepting revisions) are not permitted over email.

This implementation is an attempt to balance utility and security, but makes some sacrifices on both sides to achieve it because of the difficulty of authenticating senders in the general case (e.g., where you are an open source project and need to interact with users whose email accounts you have no control over).

You can also set metamta.public-replies, which will change how Phorge delivers email. Instead of sending each recipient a unique mail with a personal reply-to address, it will send a single email to everyone with a public reply-to address. This decreases security because anyone who can spoof a "From" address can act as another user, but increases convenience if you use mailing lists and, practically, is a reasonable setting for many installs. The reply-to address will still contain a hash unique to the object it represents, so users who have not received an email about an object can not blindly interact with it.

If you enable application email addresses, those addresses also use the weaker "From" authentication mechanism.

Testing and Debugging Inbound Email

You can use the bin/mail utility to test and review inbound mail. This can help you determine if mail is being delivered to Phorge or not:

phorge/ $ ./bin/mail list-inbound   # List inbound messages.
phorge/ $ ./bin/mail show-inbound   # Show details about a message.

You can also test receiving mail, but note that this just simulates receiving the mail and doesn't send any information over the network. It is primarily aimed at developing email handlers: it will still work properly if your inbound email configuration is incorrect or even disabled.

phorge/ $ ./bin/mail receive-test   # Receive test message.

Run bin/mail help <command> for detailed help on using these commands.

Mailgun Setup

To use Mailgun, you need a Mailgun account. You can sign up at http://www.mailgun.com. Provided you have such an account, configure it like this:

• Configure a mail domain according to Mailgun's instructions.
• Add a Mailgun route with a catch_all() rule which takes the action forward("https://phorge.example.com/mail/mailgun/"). Replace the example domain with your actual domain.
• Configure a mailer in cluster.mailers with your Mailgun API key.

Use of Mailgun is discouraged because of concerns that they may not be a trustworthy custodian of sensitive data. See https://phurl.io/u/mailgun for discussion and context.

Postmark Setup

To process inbound mail from Postmark, configure this URI as your inbound webhook URI in the Postmark control panel:

https://<phorge.yourdomain.com>/mail/postmark/

See also the Postmark section in Configuring Outbound Email for discussion of the remote address whitelist used to verify that requests this endpoint receives are authentic requests originating from Postmark.

SendGrid Setup

To use SendGrid, you need a SendGrid account with access to the "Parse API" for inbound email. Provided you have such an account, configure it like this:

• Configure an MX record according to SendGrid's instructions, i.e. add phorge.example.com MX 10 mx.sendgrid.net. or similar.
• Go to the "Parse Incoming Emails" page on SendGrid (http://sendgrid.com/developer/reply) and add the domain as the "Hostname".
• Add the URL https://phorge.example.com/mail/sendgrid/ as the "Url", using your domain (and HTTP instead of HTTPS if you are not configured with SSL).
• If you get an error that the hostname "can't be located or verified", it means your MX record is either incorrectly configured or hasn't propagated yet.
• Set metamta.reply-handler-domain to phorge.example.com (whatever you configured the MX record for).

That's it! If everything is working properly you should be able to send email to anything@phorge.example.com and it should appear in bin/mail list-inbound within a few seconds.

Local MTA: Installing Mailparse

If you're going to run your own MTA, you need to install the PECL mailparse extension. In theory, you can do that with:

$ sudo pecl install mailparse

You may run into an error like "needs mbstring". If so, try:

$ sudo yum install php-mbstring # or equivalent
$ sudo pecl install -n mailparse

If you get a linker error like this:

PHP Warning:  PHP Startup: Unable to load dynamic library
'/usr/lib64/php/modules/mailparse.so' - /usr/lib64/php/modules/mailparse.so:
undefined symbol: mbfl_name2no_encoding in Unknown on line 0

...you need to edit your php.ini file so that mbstring.so is loaded before mailparse.so. This is not the default if you have individual files in php.d/.

Local MTA: Configuring Sendmail

Before you can configure Sendmail, you need to install Mailparse. See the section "Installing Mailparse" above.

Sendmail is very difficult to configure. First, you need to configure it for your domain so that mail can be delivered correctly. In broad strokes, this probably means something like this:

• add an MX record;
• make sendmail listen on external interfaces;
• open up port 25 if necessary (e.g., in your EC2 security policy);
• add your host to /etc/mail/local-host-names; and
• restart sendmail.

Now, you can actually configure sendmail to deliver to Phorge. In /etc/aliases, add an entry like this:

phorge: "| /path/to/phorge/scripts/mail/mail_handler.php"

If you use the PHABRICATOR_ENV environmental variable to select a configuration, you can pass the value to the script as an argument:

.../path/to/mail_handler.php <ENV>

This is an advanced feature which is rarely used. Most installs should run without an argument.

After making this change, run sudo newaliases. Now you likely need to symlink this script into /etc/smrsh/:

sudo ln -s /path/to/phorge/scripts/mail/mail_handler.php /etc/smrsh/

Finally, edit /etc/mail/virtusertable and add an entry like this:

@yourdomain.com phorge@localhost

That will forward all mail to @yourdomain.com to the Phorge processing script. Run sudo /etc/mail/make or similar and then restart sendmail with sudo /etc/init.d/sendmail restart.