Enabling HTTPS Engagement Tracking on SparkPost

SparkPost supports HTTPS engagement tracking for all self-service customers. This article describes how to use a Content Delivery Network (CDN) to enable SSL engagement tracking for your domain. After completing the steps below, your email recipients will see HTTPS links in the email you send. When they visit a tracked link, your CDN will handle the SSL connection, then pass the HTTP request on to SparkPost. SparkPost will record the click event and redirect the recipient to the original URL.

If you already have non-secure tracking domains in live service, with a CNAME direct to SparkPost's tracking endpoints, you need to plan for what happens when users click on links in previously-delivered emails.

The simplest approach is to leave the current tracking-domain in place, and set up a new, sibling tracking-domain pointing to your CDN. For example, if you are using click.mycompany.com with a direct CNAME, set up the proxy with click2.mycompany.com. This enables you to test your setup is working before switching over to use it for Production traffic in SparkPost.

If you want to end up with your CDN serving the original domain:

• You'll need your CDN to handle both port 80 (HTTP) and port 443 (HTTPS) requests, so that links in previously delivered mails continue to work
• To minimize disruption, we recommend you test your setup on a sibling domain before switching
• You need to consider your certificates (which may be specific to your subdomain, or may use subdomain wild-card)
• You need to change your DNS setting on the original tracking domain(s) to point to the CDN, only when your setup is tested and working.

In order for HTTPS engagement tracking to be enabled on SparkPost, our service needs to present a valid certificate that will be trusted by the email recipient’s browser. SparkPost does not manage certificates for customer engagement tracking domains, as we are not the record owner for our customers’ domains.

Use a CDN such as Cloudflare, Fastly or AWS Cloudfront to manage certificates and keys for any custom engagement tracking domains. These services forward requests onward to SparkPost so that HTTPS tracking can be performed.

This document includes step by step guides for the following CDNs.

• CloudFlare:
  • Create a Domain
    • (Cloudflare certificates are auto-issued)
• AWS CloudFront:
  • Create a Domain
  • Issue a Certificate
• Fastly:
  • Create a Domain
  • Issue a Certificate
• Google Cloud Platform:
  • Create a Domain
  • Issue a Certificate
• Microsoft Azure:
  • Create a Domain
  • Issue a Certificate

If you are using a CDN not listed here, the steps will differ in workflow. Please refer to your CDN documentation and contact their respective support departments if you have any questions.

This address is configured as the address your CDN forwards HTTPS requests to, usually known as the "origin server".

After configuring your CDN, you need instruct SparkPost to encode your links using HTTPS and verify your domain - instructions here.

Updated for Cloudflare web UI as of June 2023.

1. Create (or log in to your existing) CloudFlare account.

2. In CloudFlare, select the Websites management menu on the left. If your organizational domain is already shown on the app, click on it. Otherwise choose Add a Site. Enter your domain and confirm.

   CloudFlare will scan your existing DNS provider to collect records, and prompt you to change to use specifically-named CloudFlare nameservers (yours may be different to the example shown below). You will require a login to your existing DNS provider to be able to change them; beyond the scope of this document.

   Wait for the changes to take effect. You can monitor this in the CloudFlare dashboard.

   Note that (as the above screen mentions) the old nameservers may still be cached in the Internet and will take time to update. You can check the nameservers that a machine sees using dig NS:

   dig NS myexample.com

   ; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.68.rc1.87.amzn1 <<>> NS myexample.com
   ;; global options: +cmd
   ;; Got answer:
   ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4325
   ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
   
   ;; QUESTION SECTION:
   ;myexample.com.		IN	NS
   
   ;; ANSWER SECTION:
   myexample.com.	300	IN	NS	clay.ns.cloudflare.com.
   myexample.com.	300	IN	NS	coco.ns.cloudflare.com.
   
   ;; Query time: 8 msec
   ;; SERVER: 172.31.0.2#53(172.31.0.2)
   ;; WHEN: Tue Dec 21 18:02:00 2021
   ;; MSG SIZE  rcvd: 92
   

3. Check, and if necessary add the tracking domain CNAME.

   In CloudFlare, go to the DNS management menu. If you already have a plain (HTTP) tracking domain set up with SparkPost, it should be already present in the records. Check that it's set to "Proxied".

   If you are setting up a new tracking domain, then select the "Add record" option:

   • Select CNAME as record type.

   • Enter the subdomain you have chosen (in our example, this is track) in the Name field.

   • Specify the Target as the correct SparkPost tracking endpoint address for your service. See the possible endpoint addresses here.

   • Ensure Proxy status is enabled.

   • Select Save.

4. Check that CloudFlare is set to use HTTPS.

   In CloudFlare, go to the SSL/TLS management menu. You should see an Overview screen showing the encryption mode as Full.

   More information on CloudFlare SSL options can be found in this article.

   After a few minutes, you can verify that the routing is correct using ping to your tracking domain. See also troubleshooting tips.

   Cloudflare does not offer control of cache "time to live" (TTL) on free accounts. This may mask repeat opens/clicks, as described here. If you have a paid account, under Caching on the left side menu, check and set your TTL value.

5. Confirm that User-Agent and Host HTTP headers from original requests are correctly forwarded to SparkPost. In CloudFlare, go to the Rules management menu.

   In Page Rules, be sure that there are no configured page rules with the Host Header Override setting.

   In Transform Rules, be sure that there are no configured transform rules that modify the User-Agent and Host headers.

   In Origin Rules, be sure that there are no origin rules configured to rewrite the Host header.

   If your CloudFlare account has Load Balancing enabled, go to Traffic management menu on the left and select Load Balancing. Be sure that the option to override the Host header is disabled in your load balancer.

6. Follow these steps to update and verify your tracking domain.

Updated for CloudFront Console v3, as of July 2021.

Note: If you utilize CloudFront as your CDN to manage certificates and keys for any custom engagement tracking domains, it will result in a loss of user agent data. We include steps below to minimize data loss.

The following is a sample guide for use with AWS CloudFront only; please note, the steps to configure your chosen CDN will likely differ from CloudFront in workflow. Please refer to your CDN documentation and contact their respective support departments if you have any questions. For up to date information on creating a distribution via CloudFront, please refer to the AWS docs.

1. Login with your credentials onto AWS console and navigate to the CloudFront v3 console.

2. Choose Create Distribution:

3. Origin

   • Origin Domain: enter the correct endpoint address for your service, see here.

   • Origin Path: leave blank.

   • Protocol: choose "Match viewer". This provides backwards compatibility with HTTP as well as HTTPS.

   • Leave the port numbers at default.

   • Choose minimum version of TLSv1.2. Note this sets what AWS CloudFront uses to communicate with SparkPost, it does not limit what your distribution offers to clients.

   • Optionally, change the name (you can leave this at default).

   • Under "Add custom header", click "Add header". Enter X-Forwarded-Host as the header name and your custom tracking domain as the header value.

   • Leave "Enable Origin Shield" disabled.

   • Skip the "Additional settings".

4. Default cache behavior

   • "Path Pattern" - use default setting of (*).

   • "Compress objects automatically" - use default setting of "Yes". (This doesn't really affect engagement tracking, as large objects are not transferred).

   • "Viewer Protocol Policy" - use default setting "HTTP and HTTPS". This enables your distribution to serve non-secure links that may be already out there in your delivered email, prior to enabling this.

   • "Allowed HTTP methods" - use default setting of "GET, HEAD".

   • "Restrict viewer access" - use default setting of "No".

   • "Cache key and origin requests" - use default setting of "Cache policy and origin request policy (recommended)".

   • "Cache policy" - choose "Create policy". This opens a new tab.

   • Give your policy a name, e.g. "SparkPost_ET_short_cache". Spaces are not permitted in names.

   • Set Minimum TTL to 1 second

   • Set Maximum TTL to 10 seconds

   • Set default TTL to 10 seconds. For more information see Cache Time To Live (TTL) settings.

   Cache Key Settings:

   • Set Headers, Query Strings and Cookies to None.

   • Leave Compression Support on defaults.

   • Click "Create" (on first time) / "Save Changes" (if modifying).

   • You can close this tab.

   Continuing on your original tab, "Cache key and origin requests":

   • Under Origin Request policy - choose "Create policy". This opens a new tab.

   • Give your policy a name and optional description.

   • Under Origin request settings, "Headers", select "Include the following headers".

   • Select "Add custom".

   • Enable forwarding of the User-Agent header. Type in User-Agent and click "Add". This allows User-Agent data to be present in your engagement events received from SparkPost.

   • Under "Query strings", select "Include the following query strings".

   • Under "Add query string", enter target.

   • Leave Cookies set to default (None). Your origin request settings should now look like this.

   • Click "Create" (on first time) / "Save Changes" (if modifying).

   • You can close this tab.

5. Continuing on your original tab, under "Function associations - optional":

   • Leave these settings at default ("No association").

6. Settings:

   • "Price class" - we recommend leaving this set to default ("Use all edge locations").

   • AWS WAF web ACL - leave at default

   • Alternate domain name (CNAME) - leave this blank for now.

   • Under "Custom SSL Certificate", select Custom SSL Certificate - Upload certificates as needed.

     If you want to have AWS create a new certificate within AWS instead of importing an existing one, click "Request certificate" and follow the steps here before continuing.

   • Leave the other settings at default / recommended values.

   • At the bottom of the page, press Create Distribution. This returns you to the main CloudFront Distributions list.

7. Create, or update, a CNAME record with your DNS service so that requests to your tracking domain are routed to your CloudFront distribution. This will be specific to your DNS service.

   • Get the "Domain Name" for your distribution from the Distributions page. You can use the square "copy" button.

   • Create the CNAME record within your DNS service (this will be specific to your provider). If you have a TTL (time to live) field, we suggest to set this to 1 hour.

     Example DNS provider CNAME setup

   • Go back into your CloudFront distribution, select "Edit", and add the CNAME record. Choose "Add item". Type in your custom tracking domain, without the leading https:// - e.g. track.mycompany.com.

   • Select "Save changes". The update will take a few minutes to deploy.

   • You can verify that the routing is successful using ping on your created record. You should see a response from CloudFront.

8. Follow these steps to update and verify your tracking domain.

Once your CNAME is set up with your DNS provider, instead of providing an existing certificate, you can have AWS issue a certificate for your custom tracking domain.

1. Navigate to the AWS Certificate Manager (ACM). Choose Request a Certificate, then select Request a public certificate.

   Add your domain name, select Next.

2. Choose DNS validation. Add tags if you wish. Select Review, then Confirm and Request.

3. On your DNS provider, create the CNAME records that are used by AWS to validate that the domain is yours.

4. Check that the certificate is shown with status "Issued", with Validation status of "Success".

1. Navigate to CloudFront. Select your distribution, then select "Edit":

2. Enter your domain name, select "Custom SSL certificate", and select from the drop-down list.

3. At the bottom of the page, click on the "Save Changes" button.

4. Follow these steps to update and verify your tracking domain, as this requires the certificate to be present and valid.

Updated June 2023. Images and descriptions follow the current Fastly web UI.

Sign up for Fastly or log in to an existing account.

1. Select the Deliver tab on the Dashboard, then click the Create a delivery service button. Give your service a name in the Options menu by clicking Edit service name.

2. In the Domains section, insert your tracking domain into the provided field and click Add. A subdomain (like click.domain.com, rather than just domain.com) is recommended.

3. Select Origins in the left-side menu. In the Hosts section, add the correct tracking endpoint for your service (also known as hostname), see possible values here.

   Fastly detects that SparkPost supports TLS, and shows the host entry like below. Optionally you can use the "pencil" edit icon to set a meaningful name.

   Fastly default settings pass the User-Agent and X-Forwarded-For HTTP headers through to SparkPost engagement tracking as expected.

4. Click Settings in the left-side menu, and scroll down to the Fallback TTL section. Click on the "pencil" icon to set the Fallback TTL to 10 seconds.

5. Still in the Settings page, be sure the Override host option is disabled. For each CDN request, the Host HTTP header should be forwarded to SparkPost in order for your domain to be identified by engagement tracking. If this option is enabled, requests to SparkPost won't contain your host value.

6. Activate your service by clicking on the Activate button in the top right corner of the page.

1. Under the Secure tab, select TLS management. If you have no TLS domains, click Get started. Otherwise, click the Secure another domain in the upper-right corner of the page.

2. Enter your tracking domain and click Add. Let's Encrypt certificates are free, and can be auto-renewed by Fastly via an additional CNAME record that you will need to create with your DNS provider. You can upload your own private key & certificate instead of using Let's Encrypt.

   Once finished, click Submit.

3. For Let's Encrypt option: verify your domain ownership creating a CNAME record with your DNS provider using the values provided by Fastly.

4. After you create the CNAME, Fastly will request the certificate to Let's Encrypt.

   After a short time, the certificate should be issued as below:

5. Select More Details... and look for CNAME records. This is the address the Fastly will use to serve your incoming requests.

6. Create the CNAME record for the tracking domain within your DNS service (this will be specific to your provider) pointing to the CNAME address provided by Fastly (as seen above). If you have a TTL (time to live) field, we suggest to set this to 1 hour. You can verify that the routing is successful using ping on your created record.

7. Follow these steps to update and verify your tracking domain.

Fastly keeps previous versions of your configuration, and can show the "diff" between them. You can also set up advanced routing rules using the VCL language, and monitor statistics on served requests.

Updated for Google Cloud Platform as of July 2023.

Unlike some other services, Google Cloud Platform (GCP) can route tracking domains to SparkPost via an "external" HTTPS load-balancer, with certificate and routing rules. This is conceptually simpler than using a CDN in front of SparkPost tracking, as there is no caching Time to Live to consider.

GCP organizes resources under named projects.

1. From the top menu, select an existing project, or create a new project.

2. On the main menu (top left), scroll down and select Network Services then Load balancing.

   It will take a few minutes for a new project to become ready for adding services.

   You may see a message such as "Compute Engine is getting ready". Refresh your browser to continue.

3. Click the Create load balancer button at the top.

   You will see three options as below. Choose Aplication Load Balancer (HTTP/S) and Start configuration.

4. On the question Internet facing or internal only, choose From Internet to my VMs or serverless services.

   Regarding Global or Regional, choose the best option for your application. In this guide, we will proceed with a Global external Application Load Balancer. Select Continue.

5. Give your load balancer a meaningful name.

   Note the remaining setup steps:

   • Frontend configuration (which includes the certificate);
   • Backend configuration (which will be SparkPost's engagement tracking endpoint) and
   • Routing rules.

   We now configure each of these, then create the load-balancer.

6. Frontend configuration

   • Enter a name.

     For Protocol, select HTTPS (includes HTTP/2).

   • Select the Certificate field and click on Create new certificate. Choose a name to identify your certificate. If you have an existing certificate for your tracking domain, you can upload it via this dialog.

     Otherwise choose the Create Google-managed certificate option. This has the advantage that GCP will handle your renewals. Under Domains, enter your tracking domain and select Create.

   • For a new Google-managed certificate, additional steps are necessary after you review and finalize. The certificate will be available only after you point your domain to the frontend service.

7. Backend configuration

   • Choose Backend services & backend buckets / Create a backend service.

     Give the backend service a name, e.g. "sparkpost-engagement-tracking".

     For Backend type, choose Internet network endpoint group.

     For Protocol, choose HTTPS. Leave Named port and Timeout at defaults.

   • In the New backend dialog, choose Create Internet network endpoint group. This will open a new browser tab.

   • Give your Network Endpoint Group a name:

     Set Default port to 443.

     On Add through, leave this set at Fully qualified domain name and port.

     On Fully qualified domain name, add the correct endpoint address for your service, see here.

   • Select Create. You should now see your "Network Endpoint Group" exists.

   • Close this tab, and return to your previous tab.

     Unfortunately this does not auto-refresh; however, start typing the name of the Network Endpoint Group you just created, and it will appear. Click on Done.

   • Leave Enable Cloud CDN unchecked and the other settings at defaults.

   • Scroll to the end of the page and select Create. This returns you to the Create a new global external load balancer view, showing with blue check marks that Frontend configuration, Backend configuration and Routing rules are done.

   • For Routing rules mode, leave Simple host and path rule option selected. This default configuration passes all traffic on the load balancer through to our back end; this is sufficient.

   • Be sure that no Load Balancer configurations are set up to change the Host HTTP header in client requests. Both Host and User-Agent headers must be forwarded to SparkPost for Engagement Tracking to work as expected.

8. Review and finalize

   Select Review and Finalize. Your configuration should now look like this:

   Choose Create. After a few seconds, you should see the following status.

Creating a new certificate is done through the HTTP(S) load balancer configuration. On the main menu (top left), navigate to Network Services then Load balancing. Select your load balancer by clicking on its name.

1. If you don't have a named certificate present under the Frontend section, follow step Frontend configuration above to begin the process.

2. Once you have a named certificate on your frontend, it should look like this. It may take a few minutes after creating the load balancer for the IP:Port to appear.

   Click on the certificate name (underlined in blue). You should see the status similar to this:

3. Take the IP address from the IP:Port value above, and use it to create your DNS record.

   Point your tracking domain toward the load-balancer frontend with an A record. The entry will vary depending on your DNS provider; for example, on GoDaddy, you omit the organizational domain from the "Host" field, i.e. type in only the subdomain part (here, we're using the subdomain "gcp").

   Save your record. It will typically take from a few minutes up to several hours before the record is published and visible. While you're waiting, Google Cloud Platform will show the Domain Status as FAILED_NOT_VISIBLE with a yellow warning icon; this is expected.

   If your A record is correct, Google Cloud Platform will activate the certificate and make it visible on the screen. The green check mark indicates the domain/certificate is active.

   It can take a further few minutes after this before the certificate is fully active on the endpoint. You can check this using the troubleshooting tips.

4. Once the certificate is fully active, follow these steps to update and verify your tracking domain.

Microsoft Azure offers a range of load-balancer types. Most of these are for routing to destinations within Azure. The Azure Front Door service provides SSL offload with certificate, custom domains, path-based routing, and forwarding to external destinations such as SparkPost tracking. It is a global service, not tied to any specific Azure region.

The steps below are based on this guide to creating a Front Door instance.

1. From the home page or the Azure menu (top left), select Resource Groups. If you don't have one already, create a Resource Group for your project. Give this a name. Choose a region; this affects only where the Azure project metadata is stored.

2. Select "Review and Create", then "Create". This should return you back to your list of Resource Groups.

3. From the home page or the Azure menu, select Create a resource. Select Media > Front Door and CDN profiles.

   Choose Explore other offerings, choose Azure Front Door (classic) and then Continue.

   Select your resource group. Select "Next: Configuration".

4. In Frontends/domains, select + to open "Add a frontend host". Give your host a name - this needs to be a valid, unique subdomain of the domain .azurefd.net. Choosing a name based on your custom tracking domain should help to ensure uniqueness; you should see a green check mark appear on the right. (We will change this later to be your actual custom domain.)

5. Next, we create a backend pool that contains just the SparkPost tracking domain. In Backend pools, select + to open Add a backend pool. Give this a name.

6. Select "Add a backend". Set the backend host type to be "Custom host". Set the backend host name to be the correct endpoint address for your service, see here.

   • The backend host header field will be automatically filled in for you. Leave the HTTP port and HTTPS port settings at defaults. Click Add.

   • Leave the Health probe inactive, as there is only one backend.

   • This will forward the Host and User-Agent HTTP headers to SparkPost properly, which is necessary for Engagement Tracking to work as expected.

7. On "Routing rules", select +. Give your rule a name.

   Leave "Accepted protocol" as the default "HTTP and HTTPS". Ensure your "Frontends/domains" setting is your previously configured subdomain name.

   Set the Path to /* to match all incoming requests. Leave Route type set to the default "Forward", and set Forwarding Protocol to "Match request". Select "Add".

8. Select "Review + create", then "Create".

   You should see "Deployment is in progress", followed by a "deployment complete" message.

   Your front door is now active on the subdomain we set up, and can be checked using curl with added path /f/a/b/c/d, for example:

   curl -v https://my-tracking-domain.azurefd.net/f/a/b/c/d

   You should see a default 302 response from SparkPost via your Front Door.

The front door created so far has a certificate for *.azurefd.net. Here we set up a custom domain and enable a matching certificate for the custom domain. The following steps are based on this tutorial followed by this tutorial, taking the option "Use a certificate managed by Front Door" and the regular CNAME verification method.

1. Create a CNAME DNS record, pointing your custom domain to your Azure front door. The entry will vary depending on your DNS provider; for example, on GoDaddy, you omit the organizational domain from the "Host" field, i.e. type in only the subdomain part; in this test example, the tracking subdomain is azure, yours will be specific to your own name.

   Save your record. It will typically take from a few minutes up to several hours before the record is published and visible.

2. Sign in to the Azure portal and browse to the Front Door containing the frontend host that you want to map to a custom domain. Select the Front Door designer.

3. On the "Frontends/domains" panel, select + to add a custom domain. On "Custom host name", enter your tracking domain. Select "Add".

4. The Front Door designer will show a warning about your new domain not yet having a default route. Select "Routing rules". Update your routing rule to select your custom domain, and deselect your previously set azurefd.net domain.

5. On Frontends/domains, select your new domain. Update the "CUSTOM DOMAIN HTTPS" setting to be "Enabled", with minimum TLS version 1.2. Select "Update".

6. The Front Door designer shows that there are pending changes. Select "Save". After this is completed, select your custom domain. You should see that the certificate provisioning progressing through these stages.

7. After a few minutes, you should see the status of your custom domain certificate change to "complete".

   You can check the certificate is being served correctly on your domain using the troubleshooting tips.

8. Once the certificate is fully active, follow these steps to update and verify your tracking domain.

CDNs apply caching, with a "Time to Live" (TTL) for each unique request URL. When a request is first fetched, it is cached. Within the TTL, later requests may be served to the client from the CDN cache, without touching the SparkPost endpoint. The client is redirected to the landing page as usual, but a side-effect is that SparkPost does not record the repeat opens/clicks.

SparkPost engagement tracking URLs are unique to a particular recipient, message, and (for links) each individual link in the message.

A TTL of zero means "always pass through to the origin", which is, perhaps surprisingly, not ideal. Some inbound mail gateways repeatedly scan email links, which leads to erroneous high event counts.

To enable SparkPost to record human-driven repeat opens/clicks, while screening robot-driven repeat opens/clicks, we suggest setting the default TTL to 10 seconds.

If you have previously created a tracking domain (whether verified or unverified), and wish to switch it from insecure (the default) to secure, use the Update a Tracking Domain API PUT call, to update the tracking domain with the "secure": true string.

1. Run the PUT call with the following data:

   {
       "secure"  : true
   }
   

   Note: If you would like this tracking domain to be the default, please add "default": true to the JSON object above, before updating the domain.

   Detailed information on this operation can be found in our API documentation here.

2. Navigate to the Tracking Domains section in the UI and click the "test" verification button.

3. Send a test email, examine the internals of the email and check the expected HTTPS URLs using your tracking domain are present.

4. Check that clicking the links takes you to the expected landing page.

You can test that your tracking domain is correctly routed to SparkPost with the following command:

The output will show the TLS negotiation including info on the certificate served by your CDN for your domain. Example:

If forwarding is configured properly, you will see a 302 response relayed back from the SparkPost endpoint. The server header value below indicates that your CDN is routed to a SparkPost endpoint. Bear in mind CloudFlare may overwrite this header. There may be slight differences between headers returned by EU (eu.spgo.io) and US (spgo.io).