Wishlist/feature request - reduce number of emails sent to users

[aputtu]

We receive reports (id 32634) on occasions, where users get overwhelmed by the number of emails received.
There seems to be a use for:

1. Alerts regarding security issues.
2. Warnings regarding various areas of concern.
3. Information on file changes and similar functional usage.

In addition there are the type of emails, where e.g. the system will send out mails when a connection fails or similar. We will want to warn users at time, but (most often) not have them receiving excessive amount of identical warnings.

Questions to raise:

• Can we rate-limit or compound outgoing mails, so we don't send out excessive number of identical alerts/warning.
• Would it make sense to make built-in information on activity instead of sending emails?
• Option to opt-out of certain types of emails

Not sure which part of the issues that belong to Migrid development and which part that belongs to server administrator.

[jonasbardino]

I'm pretty sure 99% of these reports are about the built in notifications system warning about valid issues like repeated failed SFTP/WebDAVS login attempts and often the resulting case of hitting the login rate limit. Such emails are typically triggered when a client has set up an SFTP or WebDAVS network drive with automatic retry - and it keeps trying login despite repeat errors. Common service login error causes include when main user account expired (ERDA/SIF FAQ) or when the required twofactor session expired. The latter is mandatory for GDP sites like SIF and optional on general sites like ERDA.

The notification system already does internal batching to avoid sending an email on every error, but if the client keeps failing login it will result in additional emails every once in a while until the client stops hammering. This batching window may be adjusted but a number of these messages are about e.g. expired main account due to lack of web activity, so it does not make sense to inform about it on web. We really only can email or completely suspend the account to get user attention in such cases.

Apart from perhaps improving the actual warnings emails and related documentation I think @Rehr is best acquainted with the notification code and any frequency adjustments.

[Martin-Rehr]

The notification interval is hard coded to 60 seconds and was chosen to give users a quick response upon failed logins. Similar errors are batched within the interval.

We could extend the system to suppress similar errors that occurs across notification intervals but I'll say this goes at the end of the nice-to-have list, since users are only bothered if one of their own clients keeps hammering on the system.

Last but not least we could use fail2ban to block users with clients "on the loose".

[aputtu]

We also get report (internal ticket 34246) that some project owners state that they get too many emails coming from GDPR projects, when changes happen.

I will update the request, but will benefit from thoughts on the following.

So more specifically, three types activities of interest, that leads to notifications:

• common service login errors (mandatory, and do not cause many issues I believe)
• emails when failed SFTP/WebDAVS login occurs (60 sec. interval hard-coded)
• notification on changes in (GDPR) projects sent to owners (for some overwhelming)

Suggested goals:
Users get clear instructions in notification on what the issue is, and how to mitigate it, when needed.
We want to avoid desensitization as a result of too many false positives security alerts.

Considerations and possible actions:

• stats on notifications typically sent project owners
• revise notification texts
• adjust our rate-limiting of identical messages (e.g. system failing to connect network drive)
• if possible, send first notification, then trigger rate-limiting at 10 minutes interval, will inform fast + still encourage action
• grade in regards to severity on what and when to send:
  • high priority always sent immediately first time
  • batching for similar notifications ("10 files were added to project, 3 files were deleted within the last hour")
  • notifications on web option for changes, but the value again depend on project type and activity level

Off-shoot:
Reconsider how we provide network drives to the majority of users, ie. central administered setup of RClone network drive?