[Duplicate] Notify users trapped on suppression list

[yaelberger-commits]

Notify users trapped on suppression list can't reset password or regain access without support

If a user has their email address is on the suppression list (due to email server is down, incorrect email response codes, overactive email filter etc.) they will never get an email from Notify again. If they try to reset their password, they won't be able to receive the password reset email from us.

Bug Severity

SEV-2, impacting only users who end up on the suppression list, but makes it impossible for those users to use Notify, reset their password or receive any communications through Notify, without reaching out through support to be removed from the suppression list.

To Reproduce

Steps to reproduce the behavior:

1. Add a test account to the suppression list
2. Click on password reset link in Notify from that test user account
3. Do not receive a password reset email

Instructions for co-pilot

1. You will need to make a PR in both notification-admin and notification-api
   Add a box under /service-settings where a user can input an email-address and if the email-address is on the supression list, we can remove it
   Check that the email_address is one that this particular service has sent to through the notify system (you will have to check the notifications table)
   Send a suppor tticket to freshdesk indicating that the user has removed this email address from the supression list
   Test all of this (there are makefiles in both api and admin)
   you will likely need a webcall in admin to api
   api will have to call boto to remove the address from the supression list

???

Impact

Due to the lack of reliability of GC Email servers, a temporary failure may exclude some one from getting future Notify messages, a bigger problem than just one user.

If applicable

Impact on Notify users:

Impact on Recipients:

Impact on Notify team: We have to manually remove users from the suppression, manually reset passwords and communicate the issue. We receive more support tickets

Screenshots

https://cds-snc.freshdesk.com/a/tickets/9075

Additional context

If GC Notify sends an email an email address that doesn't work, for example Joe.fakestaff@fakedept-pasvrai.gc.ca, that address gets added automatically by Amazon SES to a suppression list so that future emails sent to that email address automatically fail, preventing the aggregate GC Notify bounce rate from going up, threatening the service.
So, if the email server fakedept-pasvrai.gc.ca blocks the email to that address for any reason, such as the email server is down, incorrect email response codes, or overactive email filter. Then poor Joe.fakestaff will get added to the suppression list, automatically by SES and never get another email from GC Notify again. Until our mutual develop friends remove them from the suppression list.

This isn't the first time unfortunately that we block notify users and recipient based on bad email server response, they try to reset their password and then it never get sent. It might be worth tackling this as a bug, to think of an escape route and surface these potential trapped users.

• We can remove emails from the suppression list as many times as we want from our account level suppression list.

• We can't remove emails from the global suppression list anymore. The account level suppression list supersedes the global one now, so removing from the account level one is fine enough.

• The SMS phone numbers from the opt-out list can only be removed once per month per number.

Ideas

These are our account suppression list
We could have an automated alert and action where if a user is on the suppression list and tries to log in or reset password, they automatically get removed from the suppression list and maybe sent a notification to try to log in again?

[amazingphilippe]

Curious to know what the exact process is for un-suppressing a user.

Idea: set a message in the Notify UI to let users know that they are on the suppression list. Within that message, what action could the user take to resolve this issue?

Is it better for us to have a tool that compares the user list to the suppressed list? And then have an alert whenever a match is found?

[jimleroyer]

@amazingphilippe Yeah, we can treat Notify's users with special handling for matching against suppression list (vs rest of recipients who are not Notify users), because they should be able to receive emails from us. This could be an alarm, we might want to know when that happens because sending an email too many times to an invalid address will cause a bounce and decrease our reputation (and potentially lead to Notify emails' sending capabilities disabled with AWS). We'd need to be careful with any automated action that would remove from the suppress list for that reason.

If we'd want the user to be able to take an action, he'd need to be logged into Notify but it's possible the user can't login anymore due to this use case. 😞

[yaelberger-commits]

It's a high priority to discuss impact and possible solutions, but not necessarily to fix immediately

[sastels]

Might be a good idea to have a warning alarm if Notify tries to send a login code or reset link to someone on the suppression list.

[yaelberger-commits]

Self-service solution we would need to replicate AWS feature ourselves

[yaelberger-commits]

Hey team! Please add your planning poker estimate with ZenHub @andrewleith @jimleroyer @sastels @jzbahrai

[yaelberger-commits]

Please add your planning poker estimate with ZenHub @amazingphilippe