很长时间没有继续这个源码分析了,原因是到了主流业务,对底层的驱动知识不太了解,也没有太多时间。
在上一节中分析到
STATIC void drbd_connector_callback(struct cn_msg *req, struct netlink_skb_parms *nsp)
mdev = ensure_mdev(nlp);
在这个调用中,会进行设备的注册和驱动的加载。这一节重点分析struct drbd_conf* drbd_new_device(unsigned int minor)方法。该方法主要是一个块设备的驱动。关于块设备的驱动程序的编写,可以参考CU上面的赵磊的帖子,该帖子绘声绘色的讲解了如何从0基础开始编写块设备驱动:。
对于每一个块设备,会进行一系列的初始化,会启动3个内核线程:
drbd_thread_init(mdev, &mdev->receiver, drbdd_init); drbd_thread_init(mdev, &mdev->worker, drbd_worker); drbd_thread_init(mdev, &mdev->asender, drbd_asender);
其中drbd_init线程负责与对端建立连接,是接受进程也是初始化进程,所以该进程的一些命名有一些奇怪。在启动完这三个线程后,主线程继续注册块设备驱动。
struct drbd_conf* drbd_new_device(unsigned int minor) { struct drbd_conf* mdev; struct gendisk* disk; struct request_queue* q; /* GFP_KERNEL, we are outside of all write-out paths */ mdev = kzalloc(sizeof(struct drbd_conf), GFP_KERNEL); if (!mdev) return NULL; if (!zalloc_cpumask_var(&mdev->cpu_mask, GFP_KERNEL)) goto out_no_cpumask; mdev->minor = minor; drbd_init_set_defaults(mdev); q = blk_alloc_queue(GFP_KERNEL); if (!q) goto out_no_q; mdev->rq_queue = q; q->queuedata = mdev; blk_queue_max_segment_size(q, DRBD_MAX_SEGMENT_SIZE); disk = alloc_disk(1); if (!disk) goto out_no_disk; mdev->vdisk = disk; set_disk_ro(disk, TRUE); disk->queue = q; disk->major = DRBD_MAJOR; disk->first_minor = minor; disk->fops = &drbd_ops; sprintf(disk->disk_name, "drbd%d", minor); disk->private_data = mdev; mdev->this_bdev = bdget(MKDEV(DRBD_MAJOR, minor)); /* we have no partitions. we contain only ourselves. */ mdev->this_bdev->bd_contains = mdev->this_bdev; q->backing_dev_info.congested_fn = drbd_congested; q->backing_dev_info.congested_data = mdev; blk_queue_make_request(q, drbd_make_request_26); blk_queue_bounce_limit(q, BLK_BOUNCE_ANY); blk_queue_merge_bvec(q, drbd_merge_bvec); q->queue_lock = &mdev->req_lock; /* needed since we use */ /* plugging on a queue, that actually has no requests! */ q->unplug_fn = drbd_unplug_fn; mdev->md_io_page = alloc_page(GFP_KERNEL); if (!mdev->md_io_page) goto out_no_io_page; if (drbd_bm_init(mdev)) goto out_no_bitmap; /* no need to lock access, we are still initializing this minor device. */ if (!tl_init(mdev)) goto out_no_tl; mdev->app_reads_hash = kzalloc(APP_R_HSIZE * sizeof(void *), GFP_KERNEL); if (!mdev->app_reads_hash) goto out_no_app_reads; mdev->current_epoch = kzalloc(sizeof(struct drbd_epoch), GFP_KERNEL); if (!mdev->current_epoch) goto out_no_epoch; INIT_LIST_HEAD(&mdev->current_epoch->list); mdev->epochs = 1; return mdev; /* out_whatever_else: kfree(mdev->current_epoch); */ out_no_epoch: kfree(mdev->app_reads_hash); out_no_app_reads: tl_cleanup(mdev); out_no_tl: drbd_bm_cleanup(mdev); out_no_bitmap: __free_page(mdev->md_io_page); out_no_io_page: put_disk(disk); out_no_disk: blk_cleanup_queue(q); out_no_q: free_cpumask_var(mdev->cpu_mask); out_no_cpumask: kfree(mdev); return NULL; }
这个方法一个主要的作用是构造一个gendisk对象,并返回。让外层调用adddisk()完成块设备驱动的添加。在块设备驱动中比较重要的是块设备IO处理函数,DRBD定义了自己的块设备处理函数:drbd_make_request_26,块设备I/O队列处理函数的输入是bio,拿到一个bio后,会先分析该bio是否需要拆分:
/* to make some things easier, force alignment of requests within the * granularity of our hash tables */ s_enr = bio->bi_sector >> HT_SHIFT; e_enr = (bio->bi_sector + (bio->bi_size >> 9) - 1) >> HT_SHIFT; if (likely(s_enr == e_enr)) { dev_err(DEV, "drbd_make_request_26 2\n"); inc_ap_bio(mdev, 1); return drbd_make_request_common(mdev, bio); }