At Norwegian (bokmål) Wikipedia I made a kind of notification gadget. I only call it a kind of as it was rather simplistic. It is although a few lessons learned from the system. The system was made over several iterations and emerged from an initial idea of a Gtalk-ish type of interface. After a while I realized that it only reimplemented what was going on at the talk pages, and as such wasn't very much better than the pages it tried to replace. So I ended up with something that is really just a very simplistic notification system.
The gadget keeps a list of visited pages and compares this to the change-feed, and reporting hits on changed pages. Initially the feed is pulled often, but after a short time the iterations slow down. It also use a lazy pull of the watchlist-feed to augment the list of pages to monitor. The list of pages is maintained as a large cookie-structure. Reports are small yellow stickers that slides into view down in the right corner, and they are driven by a timer that maintains its state across page reloads. The whole system is pretty simple.
It seems like there are two different types of notifications; broadcast and private. Broadcast (public) notifications are basically just the RC-feed, and the user (private) notifications are something happening due to some private settings like the watchlist-feed. The events triggering a private notification often comes from a public notification, and often a private notification can be reformulated as a public notification. It seems like a common case is that a category can be added to a notification and then a user can register for a public notification instead of the same notification being handled as a private notification.
In general private notifications are bad, they can't be reused and can only be used while the user is logged in. While the user may register for public notifications that is delivered in a server-driven push-mode, it is also possible to use a client-driven pull-mode for public notifications. An other distinction is that notifications pushed from the server can be said to be permanent while the notifications pulled by the client can be said to be transient. The distinction isn't quite so simple because the client can store information on a permanent basis, but it is a good approximation.
According to this the notifications at Norwegian (bokmål) Wikipedia uses public notifications from the RC-feed in a client-side pull-mode, and private notifications from the watchlist-feed and that to in a client-side pull-mode.
The state should not be maintained as cookies, this has several problems and it is really bad for privacy. Instead the state should be maintained in local or session storage. The watch list should be transferred at the beginning of the session and stored in session storage, or if the APi is updated it should update previously stored entries in local storage. The later imply that the API can report changes to the raw watchlist after a specific timestamp that is stored in local storage in the client. In addition the local storage should probably be lazy updated to catch changes outside visited pages, or perhaps be updated through server push of new data.
The yellow stickers in the present GUI should be synchronized with the proposals for the Athena design, or better be made configurable. It is especially important that the graceful enabling of Athena is supported. At least two levels should be supported; a count of unread notifications and a sticker-like notification. The later will probably not be used together with a notification count.
Today the sticker will go away as the timers says no matter if the user hovers on the sticker or not. Because the sticker contains clickable links it should brobably not be removed if the user hovers the pointer above it. It should go in a hold mode. It should also be a hide-button or -link in it so it can be removed if it covers important parts of the window. If the user keeps on typing it should probably be removed pretty fast, while if he does not type it can be hold visible somewhat longer.
Changes to the user own pages should always be reported, but the user own changes should not be reported. In this aspect the current implementation fails.
The changes-feed should include the categories of the pages, this makes a lot of notifications accessible through the changes-feed that otherwise must be private. It should also be possible to filter the changes-feed on categories at the server, even if this does not seem to be really useful. Most of the time the user will just listen in to everything on the changes-feed and filter locally in the client.
The client will pull data from the server according to events emerging locally. Such queries are usually ordinary API-requests for public data, even if the event generating the request are private.