The Crux of Linux Notifier Chains

Linux is monolithic like any other kernel. Its subsystems or modules help to keep the kernel light by being flexible enough to load and unload at runtime. In most cases, the kernel modules are interconnected to one another. An event captured by a certain module might be of interest to another module. For instance, when a USB device is plugged to your kernel, the USB core driver has to communicate to the bus driver sitting at the top. This will allow the bus driver to take care of the rest. Another classic example would be of interfaces. Many kernel modules would be looking for a network interface state change. The lower level module that detects the network interface state change, would communicate this information to the other modules.

Typically, communication systems implement request-reply messaging, or polling. In such models, a program that receives a request will have to send the data available since the last transaction. Such methods sometimes require high bandwidth or they waste polling cycles.

Linux uses a notifier chain, a simple list of functions that is executed when an event occurs. These notifier chains work in a publish-subscribe model. This model is more effective when compared to polling or the request-reply model. In a publish-subscribe model, the ‘client’ (subscriber) that requires notification of a certain event, ‘registers’ itself with the ‘server’ (publisher). The server will inform the client whenever an event of interest occurs. Such a model reduces the bandwidth requirement or the polling cycle requirement, as the client no longer requests for new data regularly.

Notifier chains

Linux uses notifier chains to inform asynchronous events or status, through the function calls registered. The data structure is defined in include/linux/notifier.h:

struct notifier_block {
      int (*notifier_call)(struct notifier_block *, unsigned long, void *);
      struct notifier_block *next;
      int priority;
};

The notifier data structure is a simple linked list of function pointers. The function pointers are registered with ‘functions’ that are to be called when an event occurs. Each module needs to maintain a notifier list. The functions are registered to this notification list.

The notification module (publisher) maintains a list head that is used to manage and traverse the notifier block list. The function that subscribes to a module is added to the head of the module’s list by using the xxxxxx_notifier_chain_register API and deletion from the list is done using xxxxxx_notifier_chain_unregister.

When an event occurs which is of interest to a particular list, then the xxxxxx_notifier_call_chain API is used to traverse the list and service the subscribers. The ‘xxxxxx_’ in the above APIs represents the type of notifier chains.

Let us now look at the different types of notifier chains in the following section.

Types of notifier chains

Notifier chains are broadly classified based on the context in which they are executed and the lock/protect mechanism of the calling chain. Based on the need of the module, the notifiers can be executed in the process context or interrupt/atomic context. Thus, notifier chains are classified into four types:

Atomic notifier chains: As the name indicates, this notifier chain is executed in interrupt or atomic context. Normally, events that are time critical, use this notifier. This also means it is a non-blockable call. Linux modules use atomic notifier chains to inform watchdog timers or message handlers. For example, register_keyboard_notifier uses atomic_notifier_chain_register to get called back on keyboard events. This notifier is usually called from the interrupt context.

Blocking notifier chains: A blocking notifier chain runs in the process context. The calls in the notification list could be blocked as it runs in the process context. Notifications that are not highly time critical could use blocking notifier chains. Linux modules use blocking notifier chains to inform the modules on a change in QOS value or the addition of a new device. For example, usb_register_notify uses blocking_notifier_chain_register to inform either USB devices or buses being added or removed.

Raw notifier chains: A raw notifier chain does not manage the locking and protection of the callers. Also, there are no restrictions on callbacks, registration, or de-registration. It provides flexibility to the user to have individual lock and protection mechanisms. Linux uses the raw notifier chain in low-level events. For example, register_cpu_notifier uses raw_notifier_chain_register to pass on CPU going up/down information.

SRCU notifier chains: Sleepable Read Copy Update (SRCU) notifier chains are similar to the blocking notifier chain and run in the process context. It differs in the way it handles locking and protection. The SRCU methodology brings in less overhead when we notify the registered callers. On the flip side, it consumes more resource while unregistering. So it is advisable to choose this methodology where we use the notifier call often and where there’s very little requirement for removing from the chain. For example, Linux module uses srcu_notifier_chain_register in CPU frequency handling.

Using notifier chains

Let us consider two modules: a publisher and a subscriber. The publisher module has to maintain and export a ‘notification head’. Generally, this is exported through an interface function that helps the subscriber to register itself with the publisher. The subscriber has to provide a callback function through notifier_block. Let us now look at how a publisher and a subscriber work using blocking notifier chains.

Assume a scenario in which an action needs to be taken by a module when a USB device is plugged into the kernel. Any USB activity is first detected by the USB core of the Linux kernel. The USB core has to ‘publish’ a notification list head to inform new USB devices of activity in the kernel. Thus the USB core becomes the publisher.

The USB core publishes its notification list through the following interface function and the notifier list data structure (snippet of drivers/usb/core/notify.c file):

 18 static BLOCKING_NOTIFIER_HEAD(usb_notifier_list);
 19
 20 /**
 21 * usb_register_notify - register a notifier callback whenever a usb change happens
 22 * @nb: pointer to the notifier block for the callback events.
 23 *
 24 * These changes are either USB devices or busses being added or removed.
 25 */
 26 void usb_register_notify (struct notifier_block *nb)
 27 {
 28 blocking_notifier_chain_register (&usb_notifier_list, nb);
 29 }
 30 EXPORT_SYMBOL_GPL(usb_register_notify);

The first step of the publisher is to provide a notifier list. The usb_notifier_list is declared as the notifier list head for the USB notification. An interface function that exports the USB notification list is also provided by the USB core. It is a better programming practice to provide an interface function than exporting a global variable.

Now, we can see how to write an example USB hook module that ‘subscribes’ to the USB core. The first step is to declare a handler function and initialise it to a notifier_block type variable. In the following example (a sample usbhook.c file), usb_notify is the handler function and it is initialised to a notifier_block type variable usb_nb.

/*
 * usbhook.c - Hook to the usb core
 */
static int usb_notify(struct notifier_block *self, unsigned long action, void *dev)
{
    printk(KERN_INFO “USB device added \n”);
    switch (action) {
    case USB_DEVICE_ADD:
        printk(KERN_INFO “USB device added \n”);
        break;
    case USB_DEVICE_REMOVE:
        printk(KERN_INFO “USB device removed \n”);
        break;
    case USB_BUS_ADD:
        printk(KERN_INFO “USB Bus added \n”);
        break;
    case USB_BUS_REMOVE:
        printk(KERN_INFO “USB Bus removed \n”);
    }
    return NOTIFY_OK;
}
static struct notifier_block usb_nb = {
     .notifier_call = usb_notify,
};
int init_module(void)
{
    printk(KERN_INFO “Init USB hook.\n”);
    /*
* Hook to the USB core to get notification on any addition or removal of USB devices
    */
    usb_register_notify(&usb_nb);

    return 0;
}

void cleanup_module(void)
{
    /*
     * Remove the hook
    */
    usb_unregister_notify(&usb_nb);

    printk(KERN_INFO “Remove USB hook\n”);
}

MODULE_LICENSE(“GPL”);

The above sample code registers the notifier_block usb_nb using the interface function of the USB core. The interface function adds the usbhook function to the usb_notifier_list of the USB core. Now we are set to receive the notification from the USB core.

When a USB device is attached to the kernel, the USB core detects it and uses the blocking_notifier_call_chain API to call the registered subscribers:

60 void usb_notify_add_bus(struct usb_bus *ubus)
61 {
62 blocking_notifier_call_chain(&usb_notifier_list, USB_BUS_ADD, ubus);
63 }

This blocking_notifier_call_chain will call the usb_notify function of the USB hook registered in usb_notifier_list.

The above sample briefs you on the infrastructure of Linux blocking notifier chains. The same methodology can be used for other types of notifier chains.

Linux kernel modules are loosely coupled and get loaded and unloaded runtime with ease. An effective methodology is required to communicate between these modules. Linux notifier chains do this effectively. They are mainly brought in for network devices and can be effectively used by other technologies as well. As developers, we have to look for such utility functions that are available in the kernel and use them effectively in our designs instead of reinventing the wheel.