Notification systems on any device or platform are proven to increase conversions, but they should only be considered when they help a user achieve a goal or when they can be applied without feeling abrupt or intrusive. If your product team is considering building push notifications, you should rigorously test how they impact the user experience before you roll them out into production code.

 

Why bother testing notifications?

Notifications are a super contentious issue in the digital space and the need for them has been under the microscope for decades. If they aren’t directly relevant to a user’s needs or demand attention too often, they’ll be dismissed as invasive and aggressive. But the opposite is also true — when they work for the user, they work for the business.

Rolling out notifications or popups can be a big development undertaking when you’re still trying to test how they will work in the user experience. They’ll also be difficult to accurately test in low fidelity formats, which make them a great candidate for A/B testing with real world users.

Below is some example code to quickly test the effectiveness of push notifications in a targeted desktop audience.

 

Trigger a notification on page load

If you’re setting up a web push notification experiment, your target audience should be desktop users on modern browsers like Chrome, Firefox, and Edge that we know support push notifications. If users on browsers that don’t support web push notifications are exposed to the experiment, they will skew the data and disqualify the test, so it’s best to keep this test highly targeted.

Add the notification code to the variant experience in the test, so the control group aren’t prompted to accept notifications.

When a notification is granted by a targeted user, you can display an icon or logo, title, and some body copy in the alert. Then when a user agrees to accept the notification, you can do something like redirect to a specific page, show an element on the page, or change a URL parameter to fire in your optimisation software.

Here’s an example of a notification asking users to sign up for price alerts on load. I’ve used a document event listener in the example, but you may not need this.

JS


document.addEventListener('DOMContentLoaded', function() {
if (Notification.permission !== 'granted') {
Notification.requestPermission();
else {
var notify = new Notification('Sign up for price alerts', {
icon: 'full-path-to-your-logo.png',
body: 'We'll let you know every time the price drops.',
});
notify.onclick = function() {
// Do something like redirect to a page or set URL parameter
// eg. window.open("https://your-url.com");
};
}
});

 

Trigger a notification on click

Here’s the same notification example as above, but triggered when a user clicks a specific button.

HTML


<button onclick="notifyMe()">Price alerts</button>

JS


function notifyMe() {
if (Notification.permission !== 'granted')
Notification.requestPermission();
else {
var notify = new Notification('Sign up for price alerts', {
icon: 'full-path-to-your-logo.png',
body: 'We'll let you know every time the price drops.',
});
notify.onclick = function() {
// Do something like redirect to a page or set URL parameter
// eg. window.open("https://your-url.com");
};
}
}

 

These examples are just scratching the surface of what you can do with web push notifications, so please check out the API documentation for more current and detailed info.

 

Measuring your experiment

Your experiment should be measuring the probability of users reaching your experiment goal with notifications verses without notifications. The experiment goal could be a page view, a click event, a page event, or a conversion. This type of experiment would work well coupled with qualitative research like surveys or contextual inquiries, so make sure to validate these experiments with other data sources.

If your experiment results are statistically significant, you may have moved the needle that bit closer to understanding what helps your users. The value of continuously testing notification systems is it helps you understand exactly how, when, and why users prefer to be notified of an event and when you’re better off just leaving them alone.