linux中的网络层设计是一种面向对象的设计结构.
一些key:
Device or Interface:
一个网络接口是可以接收或发送packet. 包括, 网卡和lo.
Protocol:
每一种协议描述了一种网络语言. 在linux中, 一个协议是一组代码向socket层提
供服务.
Socket:
一个socket是一个网络连接, 向用户程序提供类似于文件I/O风格的服务. 在
kernel中, 个socket是一对结构描述了高层的socket接口和低层的socket接口.
sk_buff:
所有网络层使用的buffer都是sk_buffs. 由于这种统一性, 存在一组例程为整个网
络层服务. sk_buffs为网络层提供了通用的缓冲和流控机制.
一个sk_buff带有一块的控制结构. 它有两组例程库. 第一组用于操纵由
sk_buffs组成的双向链表, 第二组用于操纵其控制的内存. 这种双向链表按一般网络操作的
方式进行优化, 通常是从表头取数据向表尾加数据.
下面是一个例子:
void append_frame(char *buf, int len)
{
struct sk_buff *skb = alloc_skb(len, GFP_ATOMIC);
if (skb == NULL)
my_dropped++;
else {
skb_put(skb, len);
memcpy(skb->data, data, len);
skb_append(&my_list, skb);
}
}
void process_queue(void)
{
struct sk_buff *skb;
while ((skb = skb_dequeue(&my_list)) != NULL) {
process_data(skb);
kfree_skb(skb, FREE_READ);
}
}
append_frame()有点象网卡驱动程序通过中断来取得一个报文.
process_frame()有点象将报文传递给相应的协议.
在 net/core/dev.c 中的 netif_rx() 和 net_bh() 中也有类似的程序, 由于他们需要处理
将报文正确投递和流控所以代码较复杂.
skb_put() 增长数据区的长度来为memcpy准备空间. 许多的网络操作需要加入一些桢头, 这
可以使用skb_push来将数据区向后推, 为头留出空间.
请参见下图:
----------------------------------------
| head | data | |
----------------------------------------
skb_put
-----------------------------------------
| head | data | put_data | |
-----------------------------------------
skb_push
------------------------------------------
| head | push_data | data | put_data | |
------------------------------------------
还有一个函数skb_reserve()用于在添加数据前, 移动头指针来保留空间.
参见下面的例程:
skb = alloc_skb(len+headspace, GFP_KERNEL);
skb_reserve(skb, headspace);
skb_put(skb,len);
memcpy_fromfs(skb->data, data, len);
pass_to_m_protocol(skb);
看一看下面的函数:
o skb_dequeue()
struct sk_buff *__skb_dequeue(struct sk_buff_head *list);
用于从一个链表中取得数据, 第一个参数为一个链表. 可以使用
skb_queue_head 和 skb_queue_tail 来向链表中添加数据.
o skb_queue_head()
void skb_queue_head(struct sk_buff_head *list, struct sk_buff *newsk);
用于将一个缓冲区放到链表的开头.
o skb_queue_tail()
void skb_queue_tail(struct sk_buff_head *list, struct sk_buff *newsk);
用于将一个缓冲区放到链表的结尾.
o skb_unlink()
void skb_unlink(struct sk_buff *skb);
从链表中删除一个缓冲区无论它在那个链表中. 该缓冲区并不被释放掉.
o skb_insert skb_append
void skb_insert(struct sk_buff *old, struct sk_buff *newsk);
void skb_append(struct sk_buff *old, struct sk_buff *newsk);
由于一些网络需要对它们的数据进行排序, 这两个函数是将一个新包加在某个
包的前面或后面.
o alloc_skb()
#define alloc_skb(size, priority)
(struct sk_buff *) kmalloc(size,priority)
用于创建一个新的sk_buff. 通常作如下的操作 : skb->free=1; 表明它是自由的.
o kfree_skb()
void kfree_skb(struct sk_buff *skb);
用于释放一个sk_buff.
o skb_clone()
struct sk_buff *skb_clone(struct sk_buff *skb, int gfp_mask);
作一份sk_buff的拷贝, 但并不拷贝数据区.
o skb_copy()
int skb_copy_datagram(struct sk_buff *skb, int offset,
char *to, int size);
拷贝一个sk_buff的内容.
Higher Level Support Routines
-----------------------------
其中的sock_queue_rcv_skb() 用于处理接收到的数据并有流控处理. 它通常有下面的风格:
sk = my_find_socket(whatever);
if (sock_queue_rcv_skb(sk,skb) == -1) {
myproto_stats.dropped++;
kfree_skb(skb, FREE_READ);
return;
}
该函数会阻止大量的数据进入socket. 当到达一定的限度时, 数据会被抛弃.
在发送方, sock_alloc_send_skb() 用于处理一些标志, 如: 非阻塞等.
skb=sock_alloc_send_skb(sk,....)
if(skb == NULL)
return -err;
skb->sk = sk;
skb_reserve(skb, headroom);
skb_put(skb,len);
memcpy(skb->data, data, len);
protocol_do_something(skb);
比较重要的一行是 skb->sk = sk;
sock_alloc_send_skb() 改变了缓冲区指向的socket, 通过这一行我们告诉内核作
kfree_skb()时, 要有sk作确认. 这样, 当一个设备发送完一个包并且释放了它, 用户可以
继续发更多的包.
Network Devices
---------------
所有的linux网络设备使用相同的函数接口(面向对象的).
drivers/net/skeleton.c 包含了一个网络设备的轮廓.
Basic Structure
---------------
-------------------- -----------------------------
| dev_queue_xmit | | netif_rx receivers frames |
| delivers packets | | and queues them up. |
-------------------- -----------------------------
||||||| ||||||||
----------------------------------------------------------------
| Methods and Variables (struct device) |
----------------------------------------------------------------
||||||| ||||||
------------------ ------------------- ---------------------
| Initialisation | | hard_start_time | | mydev_interrupt |
| Routine | | delivers frames | | collects received |
------------------ ------------------- | frames |
||||||| ---------------------||||||| ||||||
----------------------------------------------------------------
| Physical Device Media |
----------------------------------------------------------------
每个网络设备处理数据从层到物理介质的传输并从上接收数据, 接收到的数据被放
到网络层, 由netif_rx()完成, 改函数去掉桢头以被高层协议使用.
每一个设备提供了以一套补充的方法来处理 停止, 开始, 控制和物理封装包.
Naming
------
所有的linux网络设备有唯一的名字, 实际上, 网络设备并没有一个对应的文件, 虽然你可以
自己建一个设备文件. 传统的, 名字标明的设备的类型而不是其制造商. 多个设备通过附加
的数字来标明(数字从零开始).
ethn
以太设备
trn
令牌环
sln
SLIP设备
pppn
PPP设备
plipn
PLIP单元
tunln
IPIP封装通道
nrn
NetROM 虚拟设备
isdnn
ISDN设备
dummyn
空设备
lo
本地回绕设备
Registering A Device
--------------------
每一个设备是通过添写一个 struct device 对象, 使用register_netdev(struct device *)
来注册. 它将你的struct device连入内核的网络设备表. 你不可以释放该结构, 直到, 你使
用 unregister_netdev(struct device *) 来释放该设备. 内核不支持多个设备使用同一个名
字, 如果你的程序是一个模块, 你应该使用 struct device * dev_get(const char * name)
来确定这个名字是否被使用了, 如果是, 你应该另选一个名字或是失败.
一个典型的注册程序如下:
int register_my_device(void)
{
int i = 0;
for(i = 0;i < 100;i++)
{
sprintf(mydevice.name, "mydev%d",i);
if(dev_get(mydevice.name) == NULL)
{
if(register_netdev(&mydevice) != 0)
return -EIO;
return 0;
}
}
printk("100 mydevs loaded. Unable to load more.\n");
return -ENFILE;
}
The Device Structure
--------------------
所有的一般性信息都放在 struct device 中, 为了创建一个设备你需要初始化它. 下面讨论
它的数据.
Naming
------
首先, name 包含了设备的名字, 这是一个字符串指针, 也可以是四个空格, 这样内核会自动
分配一个ethn的名字, 但最好不要使用这一功能.
Bus Interface Parameters
------------------------
这一部分用于维护网络设备在物理上的一些参数.
Protocol Layer Variables
------------------------
关于协议层的一些数据. mtu等.
二. 报文是如何从网卡传递到相应协议的
------------------------------------
* 我个人的一些心得(仅供参考)
先来看一看网络部分是如何被初始化的. 下面是函数被调用的过程:
start_kernel -> init -> do_basic_setup -> sock_init -> proto-init
inet_proto_init -> ip_init -> dev_add_pack
下面的结构定义了网络协议的初始化入口:
struct net_proto
{
const char *name; /* Protocol name */
void (*init_func)(struct net_proto *); /* Bootstrap */
} protocols[];
每一个协议提供了一个自己的init_func. 如IP提供了ip_init.
dev_add_pack完成了实际的协议添加过程. 系统维护了两个协议表. 一个是单向链表,
另一个是hash表(使用了桶形的hash表).
每一个协议用一个struct packet_type来描述, 其中的func是其入口函数. 当系统从
读到一个报文就会调用相应协议的func来完成实际的处理工作.
struct packet_type
{
unsigned short type; /* This is really htons(ether_type). */
struct device *dev; /* NULL is wildcarded here */
int (*func) (struct sk_buff *, struct device *,
struct packet_type *);
void *data; /* Private to the packet type */
struct packet_type *next;
};
// 的hash表
struct packet_type *ptype_base[16]; /* Hashed types */
// 协议的单向链表的头指针
struct packet_type *ptype_all = NULL; /* Taps */
dev_add_pack完成的工作其实很简单, 它将一个struct packet_type指针加入到相应
的链表中. 请看源码:
void dev_add_pack(struct packet_type *pt)
{
int hash;
//由pt->type来判断加到那一个链表中
if(pt->type == htons(ETH_P_ALL))
{
netdev_nit++;
// 添加到单向链表中
pt->next = ptype_all;
ptype_all = pt;
}
else
{
// 添加到hash表中
hash = ntohs(pt->type)&15;
~~~~~~~~~~~~~~~~~~~~~~~~~~
//这是它的hash算法, 简单的只取低4位
pt->next = ptype_base[hash];
ptype_base[hash] = pt;
}
}
ip_init使用dev_add_pack完成了实际的添加过程.
下面是核心数据结构sk_buff
struct sk_buff {
struct sk_buff * next; /* Next buffer in list */
struct sk_buff * prev; /* Previous buffer in list */
struct sk_buff_head * list; /* List we are on */
struct sock *sk; /*
Socket we are owned by */
struct timeval stamp; /* Time we arrived */
struct device *dev; /* Device we arrived on/are leaving by */
/* Transport layer header */
union
{
struct tcphdr *th;
struct udphdr *uh;
struct icmphdr *icmph;
struct igmphdr *igmph;
struct iphdr *ipiph;
struct spxhdr *spxh;
unsigned char *raw;
} h;
/* Network layer header */
union
{
struct iphdr *iph;
struct ipv6hdr *ipv6h;
struct arphdr *arph;
struct ipxhdr *ipxh;
unsigned char *raw;
} nh;
/* Link layer header */
union
{
struct ethhdr *ethernet;
unsigned char *raw;
} mac;
struct dst_entry *dst;
char cb[48];
unsigned int len; /* Length of actual data */
unsigned int csum; /* Checksum */
volatile char used; /* Data moved to user and not MSG_PEEK */
unsigned char is_clone, /* We are a clone */
cloned, /* head may be cloned(check refcnt to be sure)*/
pkt_type, /* Packet class */
pkt_bridged, /* Tracker for bridging */
ip_summed; /* Driver fed us an IP checksum */
__u32 priority; /* Packet queueing priority */
atomic_t users; /* User count - see datagram.c,tcp.c */
unsigned short protocol; /* Packet protocol from driver. */
unsigned short security; /* Security level of packet */
unsigned int truesize; /* Buffer size */
unsigned char *head; /* Head of buffer */
unsigned char *data; /* Data head pointer */
unsigned char *tail; /* Tail pointer */
unsigned char *end; /* End pointer */
void (*destructor)(struct sk_buff *); /* Destruct function */
#ifdef CONFIG_IP_FIREWALL
__u32 fwmark; /* Label made by fwchains, used by pktsched */
#endif
#if defined(CONFIG_SHAPER) || defined(CONFIG_SHAPER_MODULE)
__u32 shapelatency; /* Latency on frame */
__u32 shapeclock; /* Time it should go out */
__u32 shapelen; /* Frame length in clocks */
__u32 shapestamp; /* Stamp for shaper */
__u16 shapepend; /* Pending */
#endif
#if defined(CONFIG_HIPPI)
union{
__u32 ifield;
} private;
#endif
};
* arch/i386/kernel/irq.c
void do_IRQ(unsigned int irq, struct pt_regs *regs)
{
int cpu = smp_processor_id();
kstat.irqs[cpu][irq]++;
irq_desc[irq].handler->handle(irq, regs);
/*
* This should be conditional: we should really get
* a return code from the irq handler to tell us
* whether the handler wants us to do software bottom
* half handling or not..
*/
//每次处理中断, 都会进行底半处理. 现在还不符合底半的原意.
if (1) {
if (bh_active & bh_mask)
do_bottom_half();
}
__sti();/*VY*/
}
* net/core/dev.c
/*
* Device drivers call our routines to queue packets here. We empty the
* queue in the bottom half handler.
*/
//所有的报文先存放在这里, 随后由再net_bh发给相应的网络层模块, 如 ip_input.
static struct sk_buff_head backlog;
/*
* Receive a packet from a device driver and queue it for the upper
* (protocol) levels. It always succeeds.
*/
/*
* 简单的将新到的包, 插到backlog中.
* 网卡驱动程序会调用这个函数
*/
void netif_rx(struct sk_buff *skb)
{
if(skb->stamp.tv_sec==0)
get_fast_time(&skb->stamp); //取当前的时间
/* The code is rearranged so that the path is the most
short when is congested, but is still operating.
*/
if (backlog.qlen <= netdev_max_backlog) {
if (backlog.qlen) {
if (netdev_dropping == 0) {
skb_queue_tail(&backlog, skb);
mark_bh(NET_BH);
return;
}
atomic_inc(&netdev_rx_dropped);
kfree_skb(skb);
return;
}
netdev_dropping = 0;
skb_queue_tail(&backlog, skb);
mark_bh(NET_BH);
return;
}
netdev_dropping = 1;
atomic_inc(&netdev_rx_dropped);
kfree_skb(skb);
}
/*
* 网络的底半处理程序, 将报文由backlog中取出, 发给相应的网络层模块,
* 如 ip_input. 有一点请大家注意一个报文可以被多个层所处理. 这
* 一点很重要.
*/
void net_bh(void)
{
struct packet_type *ptype;
struct packet_type *pt_prev;
unsigned short type;
unsigned long start_time = jiffies;
NET_PROFILE_ENTER(net_bh);
/*
* Can we send anything now? We want to clear the
* decks for any more sends that get done as we
* process the input. This also minimises the
* latency on a transmit interrupt bh.
*/
if (qdisc_head.forw != &qdisc_head)
qdisc_run_queues();
/*
* Any data left to process. This may occur because a
* mark_bh() is done after we empty the queue including
* that from the device which does a mark_bh() just after
*/
/*
* While the queue is not empty..
*
* Note that the queue never shrinks due to
* an interrupt, so we can do this test without
* disabling interrupts.
*/
while (!skb_queue_empty(&backlog))
{
struct sk_buff * skb;
/* Give chance to other bottom halves to run */
if (jiffies - start_time > 1)
goto net_bh_break;
/* We have a packet. Therefore the queue has shrunk */
skb = skb_dequeue(&backlog); //从backlog中取出一个包
/*
* Bump the pointer to the next structure.
*
* On entry to the protocol layer. skb->data and
* skb->nh.raw point to the MAC and encapsulated data
*/
/* XXX until we figure out every place to modify.. */
skb->h.raw = skb->nh.raw = skb->data;
if (skb->mac.raw < skb->head || skb->mac.raw > skb->data) {
printk(KERN_CRIT "%s: wrong mac.raw ptr, proto=%04x\n",
skb->dev->name, skb->protocol);
kfree_skb(skb);
continue;
}
/* Fetch the packet protocol ID. */
type = skb->protocol; //这里是指网络层
/*
* We got a packet ID. Now loop over the "known protocols"
* list. There are two lists. The ptype_all list of taps (normally empty)
* and the main protocol list which is hashed perfectly for normal protocols.
*/
pt_prev = NULL;
for (ptype = ptype_all; ptype!=NULL; ptype=ptype->next)
{
if (!ptype->dev || ptype->dev == skb->dev) {
if(pt_prev)
{
struct sk_buff *skb2=skb_clone(skb, GFP_ATOMIC);
if(skb2)
pt_prev->func(skb2,skb->dev, pt_prev);
}
pt_prev=ptype;
}
}
for (ptype = ptype_base[ntohs(type)&15]; ptype != NULL; ptype = ptype->next)
{
if (ptype->type == type && (!ptype->dev || ptype->dev==skb->dev))
{
/*
* We already have a match queued. Deliver
* to it and then remember the new match
*/
if(pt_prev)
{
struct sk_buff *skb2;
skb2=skb_clone(skb, GFP_ATOMIC);
/*
* Kick the protocol handler. This should be fast
* and efficient code.
*/
if(skb2)
pt_prev->func(skb2, skb->dev, pt_prev);
}
/* Remember the current last to do */
pt_prev=ptype;
}
} /* End of protocol list loop */
/* Is there a last item to send to ? */
if(pt_prev)
pt_prev->func(skb, skb->dev, pt_prev);
/* Has an unknown packet has been received ? */
else {
kfree_skb(skb);
}
} /* End of queue loop */
/* We have emptied the queue */
/* One last output flush. */
if (qdisc_head.forw != &qdisc_head)
qdisc_run_queues();
netdev_dropping = 0;
NET_PROFILE_LEAVE(net_bh);
return;
net_bh_break:
mark_bh(NET_BH);
NET_PROFILE_LEAVE(net_bh);
return;
}
为了理清头绪, 我重新描述一下上面的过程:
网卡驱动程序调用netif_rx将新收到的报文存在backlog队列中.
在底半处理中, net_bh调用相应的协议模块来处理报文. 而目前
linux的实现中, 每次中断都会调用底半处理.
三. 报文是如何从协议传递到网卡的
--------------------------------
struct Qdisc
{
struct Qdisc_head h;
int (*enqueue)(struct sk_buff *skb, struct Qdisc *dev);
struct sk_buff * (*dequeue)(struct Qdisc *dev);
unsigned flags;
#define TCQ_F_BUILTIN 1
#define TCQ_F_THROTTLED 2
struct Qdisc_ops *ops;
struct Qdisc *next;
u32 handle;
atomic_t refcnt;
struct sk_buff_head q;
struct device *dev;
struct tc_stats stats;
unsigned long tx_timeo;
unsigned long tx_last;
int (*reshape_fail)(struct sk_buff *skb, struct Qdisc *q);
/* This field is deprecated, but it is still used by CBQ
* and it will live until better solution will be invented.
*/
struct Qdisc *__parent;
char data[0];
};
int dev_queue_xmit(struct sk_buff *skb)
{
struct device *dev = skb->dev;
struct Qdisc *q;
#ifdef CONFIG_NET_PROFILE
start_bh_atomic();
NET_PROFILE_ENTER(dev_queue_xmit);
#endif
start_bh_atomic();
q = dev->qdisc;
if (q->enqueue) {
q->enqueue(skb, q);
qdisc_wakeup(dev);
end_bh_atomic();
#ifdef CONFIG_NET_PROFILE
NET_PROFILE_LEAVE(dev_queue_xmit);
end_bh_atomic();
#endif
return 0;
}
/* The device has no queue. Common case for software devices:
loopback, all the sorts of tunnels...
Really, it is unlikely that bh protection is necessary here:
virtual devices do not generate EOI events.
However, it is possible, that they rely on bh protection
made by us here.
*/
if (dev->flags&IFF_UP) {
if (netdev_nit)
dev_queue_xmit_nit(skb,dev);
if (dev->hard_start_xmit(skb, dev) == 0) {
end_bh_atomic();
#ifdef CONFIG_NET_PROFILE
NET_PROFILE_LEAVE(dev_queue_xmit);
end_bh_atomic();
#endif
return 0;
}
if (net_ratelimit())
printk(KERN_DEBUG "Virtual device %s asks to queue packet!\n", dev->name);
}
end_bh_atomic();
kfree_skb(skb);
#ifdef CONFIG_NET_PROFILE
NET_PROFILE_LEAVE(dev_queue_xmit);
end_bh_atomic();
#endif
return 0;
}
