Configuring Outbound Email

Phorge Administrator and User Documentation (Configuration)

Instructions for configuring Phorge to send email and other types of messages, like text messages.

Overview

Phorge sends outbound messages through "mailers". Most mailers send email and most messages are email messages, but mailers may also send other types of messages (like text messages).

Phorge can send outbound messages through multiple different mailers, including a local mailer or various third-party services. Options include:

See below for details on how to select and configure mail delivery for each mailer.

For email, Postmark is recommended because it makes it easy to set up inbound and outbound mail and has a good track record in our production services. Other services will also generally work well, but they may be more difficult to set up.

For SMS, Twilio or SNS are recommended. They're also your only upstream options.

If you have some internal mail or messaging service you'd like to use you can also write a custom mailer, but this requires digging into the code.

Phorge sends mail in the background, so the daemons need to be running for it to be able to deliver mail. You should receive setup warnings if they are not. For more information on using daemons, see Managing Daemons with phd.

Outbound "From" and "To" Addresses

When Phorge sends outbound mail, it must select some "From" address to send mail from, since mailers require this.

When mail only has "CC" recipients, Phorge generates a dummy "To" address, since some mailers require this and some users write mail rules that depend on whether they appear in the "To" or "CC" line.

In both cases, the address should ideally correspond to a valid, deliverable mailbox that accepts the mail and then simply discards it. If the address is not valid, some outbound mail will bounce, and users will receive bounces when they "Reply All" even if the other recipients for the message are valid. In contrast, if the address is a real user address, that user will receive a lot of mail they probably don't want.

If you plan to configure inbound mail later, you usually don't need to do anything. Phorge will automatically create a noreply@ mailbox which works the right way (accepts and discards all mail it receives) and automatically use it when generating addresses.

If you don't plan to configure inbound mail, you may need to configure an address for Phorge to use. You can do this by setting metamta.default-address.

Configuring Mailers

Configure one or more mailers by listing them in the the cluster.mailers configuration option. Most installs only need to configure one mailer, but you can configure multiple mailers to provide greater availability in the event of a service disruption.

A valid cluster.mailers configuration looks something like this:

[
  {
    "key": "mycompany-postmark",
    "type": "postmark",
    "options": {
      "domain": "mycompany.com",
      "api-key": "..."
    }
  },
  ...
]

The supported keys for each mailer are:

• key: Required string. A unique name for this mailer.
• type: Required string. Identifies the type of mailer. See below for options.
• priority: Optional string. Advanced option which controls load balancing and failover behavior. See below for details.
• options: Optional map. Additional options for the mailer type.
• inbound: Optional bool. Use false to prevent this mailer from being used to receive inbound mail.
• outbound: Optional bool. Use false to prevent this mailer from being used to send outbound mail.
• media: Optional list<string>. Some mailers support delivering multiple types of messages (like Email and SMS). If you want to configure a mailer to support only a subset of possible message types, list only those message types. Normally, you do not need to configure this. See below for a list of media types.

The type field can be used to select these mailer services:

• ses: Use Amazon SES.
• sendgrid: Use SendGrid.
• postmark: Use Postmark.
• twilio: Use Twilio.
• sns: Use Amazon SNS.
• mailgun: Use Mailgun.

It also supports these local mailers:

• sendmail: Use the local sendmail binary.
• smtp: Connect directly to an SMTP server.
• test: Internal mailer for testing. Does not send mail.

You can also write your own mailer by extending PhorgeMailAdapter.

The media field supports these values:

• email: Configure this mailer for email.
• sms: Configure this mailer for SMS.

Once you've selected a mailer, find the corresponding section below for instructions on configuring it.

Setting Complex Configuration

Mailers can not be edited from the web UI. If mailers could be edited from the web UI, it would give an attacker who compromised an administrator account a lot of power: they could redirect mail to a server they control and then intercept mail for any other account, including password reset mail.

For more information about locked configuration options, see Configuration Guide: Locked and Hidden Configuration.

Setting cluster.mailers from the command line using bin/config set can be tricky because of shell escaping. The easiest way to do it is to use the --stdin flag. First, put your desired configuration in a file like this:

[
  {
    "key": "test-mailer",
    "type": "test"
  }
]

Then set the value like this:

phorge/ $ ./bin/config set --stdin cluster.mailers < mailers.json

For alternatives and more information on configuration, see Configuration User Guide: Advanced Configuration

Mailer: Postmark

Postmark is a third-party email delivery service. You can learn more at https://www.postmarkapp.com/.

To use this mailer, set type to postmark, then configure these options:

• access-token: Required string. Your Postmark access token.
• inbound-addresses: Optional list<string>. Address ranges which you will accept inbound Postmark HTTP webook requests from.

The default address list is preconfigured with Postmark's address range, so you generally will not need to set or adjust it.

The option accepts a list of CIDR ranges, like 1.2.3.4/16 (IPv4) or ::ffff:0:0/96 (IPv6). The default ranges are:

[
  "50.31.156.6/32",
  "50.31.156.77/32",
  "18.217.206.57/32",
  "3.134.147.250/32"
]

The default address ranges were last updated in December 2021, and were documented at: https://postmarkapp.com/support/article/800-ips-for-firewalls

Mailer: Mailgun

Use of Mailgun is discouraged because of concerns that they may not be a trustworthy custodian of sensitive data. See https://phurl.io/u/mailgun for discussion and context.

Mailgun is a third-party email delivery service. You can learn more at https://www.mailgun.com. Mailgun is easy to configure and works well.

To use this mailer, set type to mailgun, then configure these options:

• api-key: Required string. Your Mailgun API key.
• domain: Required string. Your Mailgun domain.
• api-hostname: Optional string. Defaults to "api.mailgun.net". If your account is in another region (like the EU), you may need to specify a different hostname. Consult the Mailgun documentation.

Mailer: Amazon SES

Amazon SES is Amazon's cloud email service. You can learn more at https://aws.amazon.com/ses/.

To use this mailer, set type to ses, then configure these options:

• access-key: Required string. Your Amazon SES access key.
• secret-key: Required string. Your Amazon SES secret key.
• region: Required string. Your Amazon SES region, like us-west-2.
• endpoint: Required string. Your Amazon SES endpoint, like email.us-west-2.amazonaws.com.

Mailer: Twilio

Twilio is a third-party notification service. You can learn more at https://www.twilio.com/.

To use this mailer, set type to twilio, then configure these options:

• account-sid: Your Twilio Account SID.
• auth-token: Your Twilio Auth Token.
• from-number: Number to send text messages from, in E.164 format (like +15551237890).

Mailer: Amazon SNS

Amazon SNS is Amazon's cloud notification service. You can learn more at https://aws.amazon.com/sns/. Note that this mailer is only able to send SMS messages, not emails.

To use this mailer, set type to sns, then configure these options:

• access-key: Required string. Your Amazon SNS access key.
• secret-key: Required string. Your Amazon SNS secret key.
• endpoint: Required string. Your Amazon SNS endpoint.
• region: Required string. Your Amazon SNS region.

You can find the correct region value for your endpoint in the SNS documentation.

Mailer: SendGrid

SendGrid is a third-party email delivery service. You can learn more at https://sendgrid.com/.

You can configure SendGrid in two ways: you can send via SMTP or via the REST API. To use SMTP, configure Phorge to use an smtp mailer.

To use the REST API mailer, set type to sendgrid, then configure these options:

• api-key: Required string. Your SendGrid API key.

Older versions of the SendGrid API used different sets of credentials, including an "API User". Make sure you're configuring your "API Key".

Mailer: Sendmail

This requires a sendmail binary to be installed on the system. Most MTAs (e.g., sendmail, qmail, postfix) should install one for you, but your machine may not have one installed by default. For install instructions, consult the documentation for your favorite MTA.

Since you'll be sending the mail yourself, you are subject to things like SPF rules, blackholes, and MTA configuration which are beyond the scope of this document. If you can already send outbound email from the command line or know how to configure it, this option is straightforward. If you have no idea how to do any of this, strongly consider using Postmark instead.

To use this mailer, set type to sendmail, then configure these options:

• message-id: Optional bool. Set to false if Phorge will not be able to select a custom "Message-ID" header when sending mail via this mailer. See "Message-ID Headers" below.

Mailer: SMTP

You can use this adapter to send mail via an external SMTP server, like Gmail.

To use this mailer, set type to smtp, then configure these options:

• host: Required string. The hostname of your SMTP server.
• port: Optional int. The port to connect to on your SMTP server.
• user: Optional string. Username used for authentication.
• password: Optional string. Password for authentication.
• protocol: Optional string. Set to tls or ssl if necessary. Use ssl for Gmail.
• message-id: Optional bool. Set to false if Phorge will not be able to select a custom "Message-ID" header when sending mail via this mailer. See "Message-ID Headers" below.

Disable Mail

To disable mail, just don't configure any mailers. (You can safely ignore the setup warning reminding you to set up mailers if you don't plan to configure any.)

Testing and Debugging Outbound Email

You can use the bin/mail utility to test, debug, and examine outbound mail. In particular:

phorge/ $ ./bin/mail list-outbound   # List outbound mail.
phorge/ $ ./bin/mail show-outbound   # Show details about messages.
phorge/ $ ./bin/mail send-test       # Send test messages.

Run bin/mail help <command> for more help on using these commands.

By default, bin/mail send-test sends email messages, but you can use the --type flag to send different types of messages.

You can monitor daemons using the Daemon Console (/daemon/, or click Daemon Console from the homepage).

Priorities

By default, Phorge will try each mailer in order: it will try the first mailer first. If that fails (for example, because the service is not available at the moment) it will try the second mailer, and so on.

If you want to load balance between multiple mailers instead of using one as a primary, you can set priority. Phorge will start with mailers in the highest priority group and go through them randomly, then fall back to the next group.

For example, if you have two SMTP servers and you want to balance requests between them and then fall back to Postmark if both fail, configure priorities like this:

[
  {
    "key": "smtp-uswest",
    "type": "smtp",
    "priority": 300,
    "options": "..."
  },
  {
    "key": "smtp-useast",
    "type": "smtp",
    "priority": 300,
    "options": "..."
  },
  {
    "key": "postmark-fallback",
    "type": "postmark",
    "options": "..."
  }
}

Phorge will start with servers in the highest priority group (the group with the largest priority number). In this example, the highest group is 300, which has the two SMTP servers. They'll be tried in random order first.

If both fail, Phorge will move on to the next priority group. In this example, there are no other priority groups.

If it still hasn't sent the mail, Phorge will try servers which are not in any priority group, in the configured order. In this example there is only one such server, so it will try to send via Postmark.

Message-ID Headers

Email has a "Message-ID" header which is important for threading messages correctly in mail clients. Normally, Phorge is free to select its own "Message-ID" header values for mail it sends.

However, some mailers (including Amazon SES) do not allow selection of custom "Message-ID" values and will ignore or replace the "Message-ID" in mail that is submitted through them.

When Phorge adds other mail headers which affect threading, like "In-Reply-To", it needs to know if its "Message-ID" headers will be respected or not to select header values which will produce good threading behavior. If we guess wrong and think we can set a "Message-ID" header when we can't, you may get poor threading behavior in mail clients.

For most mailers (like Postmark, Mailgun, and Amazon SES), the correct setting will be selected for you automatically, because the behavior of the mailer is knowable ahead of time. For example, we know Amazon SES will never respect our "Message-ID" headers.

However, if you're sending mail indirectly through a mailer like SMTP or Sendmail, the mail might or might not be routing through some mail service which will ignore or replace the "Message-ID" header.

For example, your local mailer might submit mail to Mailgun (so "Message-ID" will work), or to Amazon SES (so "Message-ID" will not work), or to some other mail service (which we may not know anything about). We can't make a reliable guess about whether "Message-ID" will be respected or not based only on the local mailer configuration.

By default, we check if the mailer has a hostname we recognize as belonging to a service which does not allow us to set a "Message-ID" header. If we don't recognize the hostname (which is very common, since these services are most often configured against the localhost or some other local machine), we assume we can set a "Message-ID" header.

If the outbound pathway does not actually allow selection of a "Message-ID" header, you can set the message-id option on the mailer to false to tell Phorge that it should not assume it can select a value for this header.

For example, if you are sending mail via a local Postfix server which then forwards the mail to Amazon SES (a service which does not allow selection of a "Message-ID" header), your smtp configuration in Phorge should specify "message-id": false.

Next Steps

Continue by:

• Configuring Inbound Email so users can reply to email they receive about revisions and tasks to interact with them; or
• learning about daemons with Managing Daemons with phd; or
• returning to the Configuration Guide.