Skip to content
This repository was archived by the owner on Feb 6, 2026. It is now read-only.

Commit 7e11042

Browse files
Merge pull request #40 from williamhakim10/feature/integrate-playbook
[Feature, N/A] Integrate playbook (newsletterguide.org) into report emails
2 parents 431df2f + 784fc7a commit 7e11042

5 files changed

Lines changed: 54 additions & 11 deletions

File tree

.gitignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
node_modules/
22
*.exe
3-
*.png
43
__pycache__/
54
*.pyc
65
*.db
@@ -14,3 +13,4 @@ benchmarks-env/
1413
.coverage
1514
.DS_Store
1615
scratch/
16+
app/static/charts/*.png

app/static/img/light-bulb.png

5.01 KB
Loading
192 KB
Loading

app/templates/email-base.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
<p style="text-align:left;margin-bottom:2em">
99
<img style="width:175px;" alt="logo" src="{{ url_for('static', filename='img/logo.png', _external=True) }}">
1010
</p>
11-
<p style="color:#a71930;text-align:center;font-weight:400;font-size:2rem;font-family:Montserrat,Arial,sans-serif;margin-bottom:1.7rem">{{ title }}</p>
11+
<p style="color:#a71930;text-align:center;font-weight:400;font-size:2.5rem;line-height:3.5rem;font-family:Montserrat,Verdana,sans-serif;margin-bottom:1.7rem">{{ title }}</p>
1212
{% block content %}{% endblock %}
1313
</body>
1414
</html>

app/templates/report-email.html

Lines changed: 52 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,32 +1,75 @@
11
{% extends "email-base.html" %}
22

33
{% block content %}
4-
<p style="text-align:center">
4+
<table style="max-width:400px;margin-left:auto;margin-right:auto;margin-bottom:2.5em;border:1px solid #ddd;padding:7.5px 15px;border-radius:2px;background-color:#eee;">
5+
<tr>
6+
<td style="vertical-align: middle"><img style="max-height:50px" alt="Newletter Guide Logo" src="{{ url_for('static', filename='img/newsletter-guide-logo.png', _external=True) }}"></td>
7+
<td style="vertical-align: middle;padding-left:10px;"><a style="font-family:Montserrat,Verdana,sans-serif;color:#a71930;font-size:22px;line-height:27.5px;text-decoration:none;" target="_blank" rel="noopener" href="https://newsletterguide.org/?utm_source=email-benchmarking-tool&utm_medium=email">Check out our new <span style="text-decoration:underline">email newsletters playbook!</span></td>
8+
</tr>
9+
</table>
10+
<p style="text-align:center">
511
<img style="min-width:400px;max-width:800px;width:100%;" alt="Chart A: List Size" src="{{ url_for('static', filename='charts/' + list_id + '_size_' + epoch_time + '.png', _external=True) }}">
612
</p>
7-
<p style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:2.5em;margin-left:auto;margin-right:auto;max-width:1200px;">Chart A compares the total number of current subscribers on your list to the mean number of current subscribers across all lists we're tracking in our database. (This number may be slightly smaller than the number you're used to seeing as it does not include "Pending" email addresses.) A better way of thinking about your list is taking into account the current status of every email address you’ve ever acquired (see Chart B, below).</p>
13+
<p style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:1.25em;margin-left:auto;margin-right:auto;max-width:1200px;">Chart A compares the total number of current subscribers on your list to the mean number of current subscribers across all lists we're tracking in our database. (This number does not include "Pending" email addresses.)<p>
14+
<table style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:2.5em;margin-left:auto;margin-right:auto;max-width:1200px;padding:7.5px 15px 14px 15px;border:1px solid #ddd;border-radius:2px;background-color:#eee">
15+
<tr>
16+
<td><img alt="Tips Icon" style="max-height:25px;" src="{{ url_for('static', filename='img/light-bulb.png', _external=True) }}">
17+
<span style="color:#a71930;font-size:22px;line-height:22px;display:inline-block;padding-left:5px;padding-bottom:10px;">Our Tips</span></td>
18+
</tr>
19+
<tr>
20+
<td>The Shorenstein Center’s <a target="_blank" rel="noopener" href="https://newsletterguide.org/?utm_source=email-benchmarking-tool&utm_medium=email" style="text-decoration:underline;color:#a71930;">Email Newsletter Playbook</a> contains a section on <a target="_blank" rel="noopener" href="https://newsletterguide.org/#section-5-email-monetization-conversion/?utm_source=email-benchmarking-tool&utm_medium=email" style="text-decoration:underline;color:#a71930;">email acquisition tactics</a>, such as signup field/signup lightbox placement and off-platform acquisition strategies. That being said, all subscribers are not created equal. We encourage you to challenge your team to think about your list in terms of the current status of every email address you’ve ever acquired (see Chart B, below).</td>
21+
</tr>
22+
</table>
823
<p style="text-align:center">
924
<img style="min-width:400px;max-width:1200px;width:100%;" alt="Chart B: List Composition" src="{{ url_for('static', filename='charts/' + list_id + '_breakdown_' + epoch_time + '.png', _external=True) }}">
1025
</p>
1126
<p style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:1em;margin-left:auto;margin-right:auto;max-width:1200px;">Chart B breaks down the total number of unique email addresses in the entire list into percentages. In this case, the entire list refers to all email addresses ever acquired, both currently and formerly subscribed. MailChimp has four possible values for list member status:</p>
1227
<ul style="font-family:Montserrat,Verdana,sans-serif;margin-left:auto;margin-right:auto;margin-bottom:1em;max-width:1200px;"><li>Subscribed: current subscribers</li><li> Unsubscribed: subscribers who removed themselves from list or whom the list owner removed</li><li>Cleaned: subscribers whom MailChimp automatically removed from your list after a number of email bounces</li><li>Pending: semi-subscribers stuck in the limbo of double opt in—or, someone who gave their email address but did not hit the confirmation button in their email inbox</li></ul>
13-
<p style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:2.5em;margin-left:auto;margin-right:auto;max-width:1200px;">For more information about how MailChimp breaks down your list, refer to the following support material: <a target="_blank" rel="noopener" href="https://mailchimp.com/help/about-your-contacts/" style="text-decoration:none;color:#a71930;">MailChimp: About Your Contacts</a> and <a target="_blank" rel="noopener" href="https://mailchimp.com/help/about-cleaned-contacts/" style="text-decoration:none;color:#a71930;">MailChimp: About Cleaned Contacts</a>.</p>
28+
<p style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:1.25em;margin-left:auto;margin-right:auto;max-width:1200px;">For more information about how MailChimp breaks down your list, refer to the following support material: <a target="_blank" rel="noopener" href="https://mailchimp.com/help/about-your-contacts/" style="text-decoration:underline;color:#a71930;">MailChimp: About Your Contacts</a> and <a target="_blank" rel="noopener" href="https://mailchimp.com/help/about-cleaned-contacts/" style="text-decoration:underline;color:#a71930;">MailChimp: About Cleaned Contacts</a>.</p>
29+
<table style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:2.5em;margin-left:auto;margin-right:auto;max-width:1200px;padding:7.5px 15px 14px 15px;border:1px solid #ddd;border-radius:2px;background-color:#eee">
30+
<tr>
31+
<td><img alt="Tips Icon" style="max-height:25px;" src="{{ url_for('static', filename='img/light-bulb.png', _external=True) }}">
32+
<span style="color:#a71930;font-size:22px;line-height:22px;display:inline-block;padding-left:5px;padding-bottom:10px;">Our Tips</span></td>
33+
</tr>
34+
<tr>
35+
<td>You should pay particular attention this chart if you see large percentages of "pending" or "unsubscribed" email addresses. Pendings can be caused by a number of factors, including confusing language around confirmation for double opt-in, bot traffic, or deliverability. Consider using a service like <a target="_blank" rel="noopener" href="https://neverbounce.com/" style="text-decoration:underline;color:#a71930;">NeverBounce</a> to test which of your pending users are valid and active email addresses, and consider reaching out to your unsubscribers with a survey request to learn more about why they didn’t love your email product.</td>
36+
</tr>
37+
</table>
38+
<p style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:2.5em;margin-left:auto;margin-right:auto;max-width:1200px;"></p>
1439
<p style="text-align:center">
15-
<img style="min-width:400px;max-width:800px;width:100%;margin-top:-25px;margin-bottom:-50px;" alt="Chart C: List Open Rate" src="{{ url_for('static', filename='charts/' + list_id + '_open_rate_' + epoch_time + '.png', _external=True) }}">
40+
<img style="min-width:400px;max-width:800px;width:100%;" alt="Chart C: List Open Rate" src="{{ url_for('static', filename='charts/' + list_id + '_open_rate_' + epoch_time + '.png', _external=True) }}">
1641
</p>
17-
<p style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:2.5em;margin-left:auto;margin-right:auto;max-width:1200px;">Chart C shows your List Open Rate. MailChimp calculates List Open Rate by taking the mean of your past Campaign Open Rates over the life of your list. Each Campaign Open Rate is calculated by dividing the number of recipients who opened the campaign email by the number of emails delivered. For more information on how MailChimp calculates open rates, refer to <a target="_blank" rel="noopener" href="https://shorensteincenter.org/email-analysis-research-guide#21_Inconsistently_Labeled_and_Defined_Email_Metrics" style="text-decoration:none;color:#a71930;">Section 2.1 of our Research Guide</a>. While List and Campaign Open Rates are the traditional way of looking at your email performance, these metrics lose a large part of the story. A better way to look at your List Open Rate is through a distribution of your subscribers' individual unique open rates (see Chart D, below).</p>
42+
<p style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:2.5em;margin-left:auto;margin-right:auto;max-width:1200px;">Chart C shows your List Open Rate. MailChimp calculates List Open Rate by taking the mean of your past Campaign Open Rates over the life of your list. Each Campaign Open Rate is calculated by dividing the number of recipients who opened the campaign email by the number of emails delivered. For more information on how MailChimp calculates open rates, refer to <a target="_blank" rel="noopener" href="https://shorensteincenter.org/email-analysis-research-guide#21_Inconsistently_Labeled_and_Defined_Email_Metrics/?utm_source=email-benchmarking-tool&utm_medium=email" style="text-decoration:underline;color:#a71930;">Section 2.1 of our Research Guide</a>. While List and Campaign Open Rates are the traditional way of looking at your email performance, these metrics lose a large part of the story. As with list size (Chart A, above), a better way to look at your List Open Rate is through a distribution of your subscribers' individual unique open rates (see Chart D, below).</p>
1843
<p style="text-align:center">
1944
<img style="min-width:400px;max-width:1200px;width:100%;" alt="Chart D: Distribution of Subscribers by Open Rate" src="{{ url_for('static', filename='charts/' + list_id + '_open_rate_histogram_' + epoch_time + '.png', _external=True) }}">
2045
</p>
21-
<p style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:2.5em;margin-left:auto;margin-right:auto;max-width:1200px;">Chart D shows the distribution of open rates among current subscribers on your list. (MailChimp calculates each user's open rate by dividing the total number of emails a user has opened by the total number of emails successfully delivered to him or her.) This histogram is created through binning, which groups together consecutive continuous numbers into discrete bins. The x axis shows the range that each bin contains. As an example, the leftmost bin contains subscribers with an open rate between 0% and 10%. The rightmost bin contains subscribers with an open rate between 90% and 100%. The y axis shows the number of current subscribers who fall into each bin. Open rates generally trend downward before upticking between 80-100%. For a more comprehensive look at typical distributions, refer to <a target="_blank" rel="noopener" href="https://shorensteincenter.org/email-analysis-research-guide#Notebook_1_Section_34_Subscriber_Engagement_Distributions" style="text-decoration:none;color:#a71930;">Section 3.4 of our Research Guide</a>.</p>
46+
<p style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:2.5em;margin-left:auto;margin-right:auto;max-width:1200px;">Chart D shows the distribution of open rates among current subscribers on your list. (MailChimp calculates each user's open rate by dividing the total number of emails a user has opened by the total number of emails successfully delivered to him or her.) This histogram is created through binning, which groups together consecutive continuous numbers into discrete bins. The x axis shows the range that each bin contains. As an example, the leftmost bin contains subscribers with an open rate between 0% and 10%. The rightmost bin contains subscribers with an open rate between 90% and 100%. The y axis shows the number of current subscribers who fall into each bin. Open rates generally trend downward before upticking between 80-100%. For a more comprehensive look at typical distributions, refer to <a target="_blank" rel="noopener" href="https://shorensteincenter.org/email-analysis-research-guide#Notebook_1_Section_34_Subscriber_Engagement_Distributions/?utm_source=email-benchmarking-tool&utm_medium=email" style="text-decoration:underline;color:#a71930;">Section 3.4 of our Research Guide</a>.</p>
2247
<p style="text-align:center">
2348
<img style="min-width:400px;max-width:1200px;width:100%;" alt="Chart E: Percentage of Subscribers with User Unique Open Rate >80%" src="{{ url_for('static', filename='charts/' + list_id + '_high_open_rt_pct_' + epoch_time + '.png', _external=True) }}">
2449
</p>
25-
<p style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:2.5em;margin-left:auto;margin-right:auto;max-width:1200px;">Chart E shows your most engaged subscribers: those who open between 80% and 100% of your emails. These subscribers deserve greater scrutiny: what drives their activity? It is worth trying to figure out how to move more of your subscribers into this portion of your list.</p>
50+
<p style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:1.25em;margin-left:auto;margin-right:auto;max-width:1200px;">Chart E shows your most engaged subscribers: those who open between 80% and 100% of your emails.</p>
51+
<table style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:2.5em;margin-left:auto;margin-right:auto;max-width:1200px;padding:7.5px 15px 14px 15px;border:1px solid #ddd;border-radius:2px;background-color:#eee">
52+
<tr>
53+
<td><img alt="Tips Icon" style="max-height:25px;" src="{{ url_for('static', filename='img/light-bulb.png', _external=True) }}">
54+
<span style="color:#a71930;font-size:22px;line-height:22px;display:inline-block;padding-left:5px;padding-bottom:10px;">Our Tips</span></td>
55+
</tr>
56+
<tr>
57+
<td>We recommend surveying these users. Who are they? What are their habits? What else do they like to read? What do they want out of an email product? What might compel them to support or contribute to your work? Then, develop a few user personas. You should use this data to target your acquisition efforts towards subscribers who are likely to be be highly engaged. Alternatively, try to figure out what separates users in this category from semi-engaged subscribers (those who open ~30-80% of your emails). Might they be better served with a different newsletter or product from your newsroom?</td>
58+
</tr>
59+
</table>
2660
<p style="text-align:center">
2761
<img style="min-width:400px;max-width:1200px;width:100%;" alt="Chart F: Percentage of Subscribers who did not Open in last 365 Days" src="{{ url_for('static', filename='charts/' + list_id + '_cur_yr_inactive_pct_' + epoch_time + '.png', _external=True) }}">
2862
</p>
29-
<p style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:2.5em;margin-left:auto;margin-right:auto;max-width:1200px;">Chart F shows your current subscribers who haven't opened one of your emails within the past 365 days. Inactive subscribers can cloud your metrics and make it harder to understand your list dynamics.</p>
30-
<p style="font-family:Montserrat,Verdana,sans-serif;font-size:11px;color:#555;margin-left:auto;margin-right:auto;max-width:1200px;">Dataset average represents the mean of all lists in our dataset. &middot; We update cached data at least once every 30 days. &middot; We acknowledge the frequency with which emails are sent can affect metrics, with daily sends having lower engagement than weekly or monthly sends. We do not take this into account at the time of this analysis.<br><br>To learn more about analyzing the health of your email list using data science tools, please see our <a target="_blank" rel="noopener" href="https://shorensteincenter.org/email-analysis-research-guide/" style="text-decoration:none;color:#a71930;">Research Guide</a>. For more information on how we use and store your data, check out our <a target="_blank" rel="noopener" href="{{ url_for('privacy', _external=True) }}?utm_source=report_email&utm_medium=email" style="text-decoration:none;color:#a71930;">Privacy Policy</a>.<br><br>If you would no longer like to hear from us, please email us at <a target="_blank" rel="noopener" href="mailto:contact@emailbenchmarking.com" style="text-decoration:none;color:#a71930">contact@emailbenchmarking.com</a>.
63+
<p style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:1.25em;margin-left:auto;margin-right:auto;max-width:1200px;">Chart F shows your current subscribers who haven't opened one of your emails within the past 365 days. Inactive subscribers can make it harder to understand your list dynamics as well as affect your email deliverability (i.e. increase the probability that your emails are relegated to spam).</p>
64+
<table style="font-family:Montserrat,Verdana,sans-serif;margin-bottom:2.5em;margin-left:auto;margin-right:auto;max-width:1200px;padding:7.5px 15px 14px 15px;border:1px solid #ddd;border-radius:2px;background-color:#eee">
65+
<tr>
66+
<td><img alt="Tips Icon" style="max-height:25px;" src="{{ url_for('static', filename='img/light-bulb.png', _external=True) }}">
67+
<span style="color:#a71930;font-size:22px;line-height:22px;display:inline-block;padding-left:5px;padding-bottom:10px;">Our Tips</span></td>
68+
</tr>
69+
<tr>
70+
<td>We recommend that newsrooms routinely and manually remove inactive users from their list after sending them a re-engagement campaign. This is also called "cleaning" your list. If you have never cleaned your list, we strongly recommend that you set aside some time to do so. Then, monitor to see if and when the inactive or pending segment grows. We generally recommend a list clean twice a year (around every six months).</td>
71+
</tr>
72+
</table>
73+
<p style="font-family:Montserrat,Verdana,sans-serif;font-size:11px;color:#555;margin-left:auto;margin-right:auto;max-width:1200px;">Dataset average represents the mean of all lists in our dataset. &middot; We update cached data at least once every 30 days. &middot; We acknowledge the frequency with which emails are sent can affect metrics, with daily sends having lower engagement than weekly or monthly sends. We do not take this into account at the time of this analysis.<br><br>To learn more about analyzing the health of your email list using data science tools, please see our <a target="_blank" rel="noopener" href="https://shorensteincenter.org/email-analysis-research-guide/?utm_source=email-benchmarking-tool&utm_medium=email" style="text-decoration:underline;color:#a71930;">Research Guide</a> as well as our comprehensive <a target="_blank" rel="noopener" href="https://newsletterguide.org/?utm_source=email-benchmarking-tool&utm_medium=email" style="text-decoration:underline;color:#a71930;">Playbook</a> for email newsletters.<br><br>For more information on how we use and store your data, check out our <a target="_blank" rel="noopener" href="{{ url_for('privacy', _external=True) }}?utm_source=report-email&utm_medium=email" style="text-decoration:underline;color:#a71930;">Privacy Policy</a>.<br><br>If you would no longer like to hear from us, please email us at <a target="_blank" rel="noopener" href="mailto:contact@emailbenchmarking.com" style="text-decoration:underline;color:#a71930">contact@emailbenchmarking.com</a>.
3174
</p>
3275
{% endblock %}

0 commit comments

Comments
 (0)