Chinaunix首页 | 论坛 | 认证专区 | 博客
  • 博客访问: 1226901
  • 博文数量: 578
  • 博客积分: 161
  • 博客等级: 入伍新兵
  • 技术积分: 5084
  • 用 户 组: 普通用户
  • 注册时间: 2011-07-01 07:37
个人简介

只有偏执狂才能生存

文章分类

全部博文(578)

文章存档

2016年(10)

2015年(116)

2014年(88)

2013年(335)

2012年(29)

微信关注

IT168企业级官微



微信号:IT168qiye



系统架构师大会



微信号:SACC2013

Docker源码学习3 2015-08-10 10:51:37

分类: LINUX

这篇文章继续来看docker的代码,这里我们重点看下daemon.NewDaemon的实现。

1
2
registryService := registry.NewService(registryCfg)
d, err := daemon.NewDaemon(daemonCfg, registryService)

NewDaemon的代码很长,我们一步一步来看下,首先:

1
2
3
4
// Ensure we have compatible configuration options
iferr := checkConfigOptions(config); err != nil {
    returnnil, err
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
// checkConfigOptions checks for mutually incompatible config options
func checkConfigOptions(config *Config) error {
    // Check for mutually incompatible config options
    ifconfig.Bridge.Iface != ""&& config.Bridge.IP != ""{
        returnfmt.Errorf("You specified -b & --bip, mutually exclusive options. Please specify only one.")
    }
    if!config.Bridge.EnableIPTables && !config.Bridge.InterContainerCommunication {
        returnfmt.Errorf("You specified --iptables=false with --icc=false. ICC uses iptables to function. Please set --icc or --iptables to true.")
    }
    if!config.Bridge.EnableIPTables && config.Bridge.EnableIPMasq {
        config.Bridge.EnableIPMasq = false
    }
    returnnil
}

这里检查了下输入的参数是否有矛盾的地方。接着:

1
2
// Do we have a disabled network?
config.DisableBridge = isBridgeNetworkDisabled(config)
1
2
3
4
5
6
7
8
func isBridgeNetworkDisabled(config *Config) bool{
    returnconfig.Bridge.Iface == disableNetworkBridge
}
......
const(
    defaultNetworkMtu    = 1500
    disableNetworkBridge = "none"
)

这里会设置config.DisableBridge这个参数,如果我们的配置文件中给出的Iface为none则表明我们disable了bridge network。继续看代码:

1
2
3
4
5
6
7
8
9
// Verify the platform is supported as a daemon
ifruntime.GOOS != "linux"&& runtime.GOOS != "windows"{
    returnnil, ErrSystemNotSupported
}
 
// Validate platform-specific requirements
iferr := checkSystem(); err != nil {
    returnnil, err
}

这里会做一些平台相关的检查。继续看代码:

1
2
3
4
5
6
7
8
9
10
11
    // set up SIGUSR1 handler on Unix-like systems, or a Win32 global event
    // on Windows to dump Go routine stacks
    setupDumpStackTrap()
.......
func DumpStacks() {
    buf := make([]byte, 16384)
    buf = buf[:runtime.Stack(buf, true)]
    // Note that if the daemon is started with a less-verbose log-level than "info" (the default), the goroutine
    // traces won't show up in the log.
    logrus.Infof("=== BEGIN goroutine stack dump ===\n%s\n=== END goroutine stack dump ===", buf)
}

可以看到我们可以通过SIGUSR1这个信号量获取go routine的stack信息。后者是通过go提供的runtime模块实现的。继续看代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// get the canonical path to the Docker root directory
var realRoot string
if_, err := os.Stat(config.Root); err != nil && os.IsNotExist(err) {
    realRoot = config.Root
} else{
    realRoot, err = fileutils.ReadSymlinkedDirectory(config.Root)
    iferr != nil {
        returnnil, fmt.Errorf("Unable to get the full path to root (%s): %s", config.Root, err)
    }
}
config.Root = realRoot
// Create the root directory if it doesn't exists
iferr := system.MkdirAll(config.Root, 0700); err != nil && !os.IsExist(err) {
    returnnil, err
}

这里会建立root目录。继续看代码:

1
2
3
4
5
6
7
8
9
10
// set up the tmpDir to use a canonical path
tmp, err := tempDir(config.Root)
iferr != nil {
    returnnil, fmt.Errorf("Unable to get the TempDir under %s: %s", config.Root, err)
}
realTmp, err := fileutils.ReadSymlinkedDirectory(tmp)
iferr != nil {
    returnnil, fmt.Errorf("Unable to get the full path to the TempDir (%s): %s", tmp, err)
}
os.Setenv("TMPDIR", realTmp)

这里建立了一个tmp目录。tempDir做的事情就是拼接出一个/root/tmp的目录,其中root路径由config.Root给出。继续看代码:

1
2
// Set the default driver
graphdriver.DefaultDriver = config.GraphDriver

这里设置了默认的graphdriver。根据http://www.infoq.com/cn/articles/docker-source-code-analysis-part1这篇文章我们可以知道graphdriver主要用于完成容器镜像的管理。driver的概念就是分了一层,熟悉面向对象的同学把其认为是实现某个接口的类就行了,不过docker这里会复杂一些,比如我的driver是基于XXX实现的,那么除了实现与XXX交互的各个方法外,docker在启动的时候还需要保证XXX这个服务正常运行。继续看代码:

1
2
3
4
5
6
7
8
9
// Load storage driver
driver, err := graphdriver.New(config.Root, config.GraphOptions)
iferr != nil {
    returnnil, fmt.Errorf("error initializing graphdriver: %v", err)
}
logrus.Debugf("Using graph driver %s", driver)
 
d := &Daemon{}
d.driver = driver

这里可以看到docker加载了graphdriver的driver。我满来看下这里的New:

1
2
3
4
5
6
7
func New(root string, options []string) (driver Driver, err error) {
    for_, name := range []string{os.Getenv("DOCKER_DRIVER"), DefaultDriver} {
        ifname != ""{
            logrus.Debugf("[graphdriver] trying provided driver %q", name) // so the logs show specified driver
            returnGetDriver(name, root, options)
        }
    }

这里显示获取driver的列表,DefaultDriver现在使用的是配置中的config.GraphDriver。config.GraphDriver默认是””,所以我们不会执行if中的代码而是由docker猜测一个可能的driver:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
// Guess for prior driver
priorDrivers := scanPriorDrivers(root)
for_, name := range priority {
    ifname == "vfs"{
        // don't use vfs even if there is state present.
        continue
    }
    for_, prior := range priorDrivers {
        // of the state found from prior drivers, check in order of our priority
        // which we would prefer
......
    // Check for priority drivers first
for_, name := range priority {
    driver, err = GetDriver(name, root, options)
    iferr != nil {
        iferr == ErrNotSupported || err == ErrPrerequisites || err == ErrIncompatibleFS {
            continue
        }
        returnnil, err
    }
    returndriver, nil
}
 
// Check all registered drivers if no priority driver is found
for_, initFunc := range drivers {
    ifdriver, err = initFunc(root, options); err != nil {
        iferr == ErrNotSupported || err == ErrPrerequisites || err == ErrIncompatibleFS {
            continue
        }
        returnnil, err
    }
    returndriver, nil
}
returnnil, fmt.Errorf("No supported storage backend found")

假设我们的driver选取了aufs,此时会执行GetDriver,看下其实现:

1
2
3
4
5
6
7
func GetDriver(name, home string, options []string) (Driver, error) {
    ifinitFunc, exists := drivers[name]; exists {
        returninitFunc(filepath.Join(home, name), options)
    }
    logrus.Errorf("Failed to GetDriver graph %s %s", name, home)
    returnnil, ErrNotSupported
}

可以看到这里调用了driver注册的initFunc函数来执行初始化。那么drivers中的内容是何时注册的呢?通过Register函数注册的:

1
2
3
4
5
6
7
8
func Register(name string, initFunc InitFunc) error {
    if_, exists := drivers[name]; exists {
        returnfmt.Errorf("Name already registered %s", name)
    }  
    drivers[name] = initFunc
 
    returnnil
}

比如在aufs的代码中可以看到:

1
2
3
func init() {
    graphdriver.Register("aufs", Init)
}

现在问题来了,这里的init是什么时候被调用的呢?其实init方法是go的一个特性,其会的在main函数调用前调用。

继续看代码,这里来看下aufs的Init做了什么事情:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
// New returns a new AUFS driver.
// An error is returned if AUFS is not supported.
func Init(root string, options []string) (graphdriver.Driver, error) {
 
    // Try to load the aufs kernel module
    iferr := supportsAufs(); err != nil {
        returnnil, graphdriver.ErrNotSupported
    }
 
    fsMagic, err := graphdriver.GetFSMagic(root)
    iferr != nil {
        returnnil, err
    }
    iffsName, ok := graphdriver.FsNames[fsMagic]; ok {
        backingFs = fsName
    }
 
    for_, magic := range incompatibleFsMagic {
        iffsMagic == magic {
            returnnil, graphdriver.ErrIncompatibleFS
        }
    }

首先是检查系统是否支持aufs,如果支持则获取fs的magic number然后获取对应的fsName,这里就是aufs。然后检查下兼容性。接着:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
    paths := []string{
        "mnt",
        "diff",
        "layers",
    }
 
    a := &Driver{
        root:   root,
        active: make(map[string]int),
    }
 
    // Create the root aufs driver dir and return
    // if it already exists
    // If not populate the dir structure
    iferr := os.MkdirAll(root, 0755); err != nil {
        ifos.IsExist(err) {
            returna, nil
        }
        returnnil, err
    }
 
    iferr := mountpk.MakePrivate(root); err != nil {
        returnnil, err
    }
 
    for_, p := range paths {
        iferr := os.MkdirAll(path.Join(root, p), 0755); err != nil {
            returnnil, err
        }
    }
    returna, nil
}

可以看到这里建立了aufs的root目录。我们之前将driver比作接口和实现类的关系,这里的&Driver中的Driver就是我们的接口,其需要实现下面几个方法:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
// Driver is the interface for layered/snapshot file system drivers.
type Driver interface {
    ProtoDriver
    // Diff produces an archive of the changes between the specified
    // layer and its parent layer which may be "".
    Diff(id, parent string) (archive.Archive, error)
    // Changes produces a list of changes between the specified layer
    // and its parent layer. If parent is "", then all changes will be ADD changes.
    Changes(id, parent string) ([]archive.Change, error)
    // ApplyDiff extracts the changeset from the given diff into the
    // layer with the specified id and parent, returning the size of the
    // new layer in bytes.
    ApplyDiff(id, parent string, diff archive.ArchiveReader) (size int64, err error)
    // DiffSize calculates the changes between the specified id
    // and its parent and returns the size in bytes of the changes
    // relative to its base filesystem directory.
    DiffSize(id, parent string) (size int64, err error)
}

这几个方法的实现在具体的实现文件中可以找到,比如我们可以在aufs的实现中找到这些方法的实现。换句话说就是aufs实现了这个接口。

我们回到daemon.NewDaemon中来。现在我们 已经看到了graphdriver为aufs时候的初始化实现,主要就是设置了aufs需要的目录。然后我们继续看代码:

1
2
3
4
5
6
7
8
9
10
d := &Daemon{}
d.driver = driver
// Ensure the graph driver is shutdown at a later point
defer func() {
    iferr != nil {
        iferr := d.Shutdown(); err != nil {
            logrus.Error(err)
        }
    }
}()

这里我们建立了一个Daemon的结构体,并将我们的graphdriver做为其driver属性,同时保证代码结束的时候会调用Shutdown。继续看代码:

1
2
3
4
5
6
7
// Verify logging driver type
ifconfig.LogConfig.Type != "none"{
    if_, err := logger.GetLogDriver(config.LogConfig.Type); err != nil {
        returnnil, fmt.Errorf("error finding the logging driver: %v", err)
    }
}
logrus.Debugf("Using default logging driver %s", config.LogConfig.Type)

这里检查了下logging driver。继续看代码:

1
2
3
4
// Configure and validate the kernels security support
iferr := configureKernelSecuritySupport(config, d.driver.String()); err != nil {
    returnnil, err
}

这里检查了下内核的security方面的支持。继续看代码:

1
2
3
4
5
daemonRepo := filepath.Join(config.Root, "containers")
 
iferr := system.MkdirAll(daemonRepo, 0700); err != nil && !os.IsExist(err) {
    returnnil, err
}

这里建立了daemon的repo的目录。继续看代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// Migrate the container if it is aufs and aufs is enabled
iferr := migrateIfDownlevel(d.driver, config.Root); err != nil {
    returnnil, err
}
 
logrus.Debug("Creating images graph")
g, err := graph.NewGraph(filepath.Join(config.Root, "graph"), d.driver)
iferr != nil {
    returnnil, err
}
 
// Configure the volumes driver
iferr := configureVolumes(config); err != nil {
    returnnil, err
}

我们看下configureVolumes,根据注释这里应该是建立了一个volume的driver:

1
2
3
4
5
6
7
8
func configureVolumes(config *Config) error {
    volumesDriver, err := local.New(config.Root)
    if err != nil {
        return err
    }
    volumedrivers.Register(volumesDriver, volumesDriver.Name())
    return nil
}

这里的实现其实和上面的graphdriver类似。我们继续看代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
trustKey, err := api.LoadOrCreateTrustKey(config.TrustKeyPath)
iferr != nil {
    returnnil, err
}
 
trustDir := filepath.Join(config.Root, "trust")
 
iferr := system.MkdirAll(trustDir, 0700); err != nil && !os.IsExist(err) {
    returnnil, err
}
trustService, err := trust.NewTrustStore(trustDir)
iferr != nil {
    returnnil, fmt.Errorf("could not create trust store: %s", err)
}

这里初始化了和秘钥相关的目录。继续看代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
eventsService := events.New()
logrus.Debug("Creating repository list")
tagCfg := &graph.TagStoreConfig{
    Graph:    g,
    Key:      trustKey,
    Registry: registryService,
    Events:   eventsService,
    Trust:    trustService,
}
repositories, err := graph.NewTagStore(filepath.Join(config.Root, "repositories-"+d.driver.String()), tagCfg)
iferr != nil {
    returnnil, fmt.Errorf("Couldn't create Tag store: %s", err)
}

这里简历了一个events,接着是建立了一个TagStore。目前我们都把他们看成是结构体就行了。继续看代码:

1
2
3
4
d.netController, err = initNetworkController(config)
iferr != nil {
    returnnil, fmt.Errorf("Error initializing network controller: %v", err)
}

可以看到这里涉及到了d.netController,我们知道d就是我们的daemon,所以d的属性肯定是以后会被用到的东西。这里初始化了网络controller,来看下其实现:

1
2
3
4
5
6
7
8
9
10
func initNetworkController(config *Config) (libnetwork.NetworkController, error) {
    netOptions, err := networkOptions(config)
    iferr != nil {
        returnnil, err
    }
 
    controller, err := libnetwork.New(netOptions...)
    iferr != nil {
        returnnil, fmt.Errorf("error obtaining controller instance: %v", err)
    }

这里调用了libnetwork,libnetwork我们以后会重点研究。总之这里的代码负责初始化我们的网络环境。继续看代码:

1
2
3
4
5
6
7
graphdbPath := filepath.Join(config.Root, "linkgraph.db")
graph, err := graphdb.NewSqliteConn(graphdbPath)
iferr != nil {
    returnnil, err
}
 
d.containerGraph = graph

这里简历了一个sqlite的db,并且也赋值给了daemon。继续看代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
var sysInitPath string
ifconfig.ExecDriver == "lxc"{
    initPath, err := configureSysInit(config)
    iferr != nil {
        returnnil, err
    }
    sysInitPath = initPath
}
 
sysInfo := sysinfo.New(false)
// Check if Devices cgroup is mounted, it is hard requirement for container security,
// on Linux/FreeBSD.
ifruntime.GOOS != "windows"&& !sysInfo.CgroupDevicesEnabled {
    returnnil, fmt.Errorf("Devices cgroup isn't mounted")
}

这里会判断我们的系统是否支持cgroup。继续看代码:

1
2
3
4
ed, err := execdrivers.NewDriver(config.ExecDriver, config.ExecOptions, config.ExecRoot, config.Root, sysInitPath, sysInfo)
iferr != nil {
    returnnil, err
}

这里其实和我们上面说的graph的driver初始化类似,也是根据参数调用对应的NewDriver函数,后者会在系统上初始化一些东西后返回一个driver,并且这个driver也实现了接口要求的方法。对于native来说这里使用的是libcontainer来获取driver的factory熟悉。相比以后的事情都是交给libcontainer来做吧。libcontainer和libnetwork我们以后会单独分析。继续看代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
    d.ID = trustKey.PublicKey().KeyID()
    d.repository = daemonRepo
    d.containers = &contStore{s: make(map[string]*Container)}
    d.execCommands = newExecStore()
    d.graph = g
    d.repositories = repositories
    d.idIndex = truncindex.NewTruncIndex([]string{})
    d.sysInfo = sysInfo
    d.config = config
    d.sysInitPath = sysInitPath
    d.execDriver = ed
    d.statsCollector = newStatsCollector(1 * time.Second)
    d.defaultLogConfig = config.LogConfig
    d.RegistryService = registryService
    d.EventsService = eventsService
    d.root = config.Root
    go d.execCommandGC()
 
    iferr := d.restore(); err != nil {
        returnnil, err
    }
 
    returnd, nil
}

这里首先是将一大堆我们上面建立的结构体都扔给我们的daemon,然后调用d.execCommandGC()后台执行一些清理工作:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
// execCommandGC runs a ticker to clean up the daemon references
// of exec configs that are no longer part of the container.
func (d *Daemon) execCommandGC() {
    forrange time.Tick(5 * time.Minute) {
        var (
            cleaned          int
            liveExecCommands = d.containerExecIds()
        )
        forid, config := range d.execCommands.s {
            ifconfig.canRemove {
                cleaned++
                d.execCommands.Delete(id)
            } else{
                if_, exists := liveExecCommands[id]; !exists {
                    config.canRemove = true
                }
            }
        }
        logrus.Debugf("clean %d unused exec commands", cleaned)
    }  
}

然后系统会调用下restore加载现有的container到内存中。接着我们的daemon就完成了。

我们来总结下daemon.NewDaemon。其做的事情就是初始化各个子模块,比如graph,net,exec等等。这里的初始化有两种,一种是直接得到一个子模块的struct,还有一种是获取一个driver,这里的driver可以认为是一个接口,然后初始化的时候会获取这个接口的一个实现。另外这里的初始化不仅仅是代码上的获取类这么简单,还会做很多物理上的操作,比如设置aufs的目录、建立sqlite db等等。在这些初始化完成后我们的daemon就拥有了强大的功能,从它入手就可以做很多东西了(它的每个属性都是我们初始化得到的重要结构体)。接着这里会调用下daemon的restore方法恢复系统上已有的container。

阅读(928) | 评论(0) | 转发(0) |
0

上一篇:Docker源码学习2

下一篇:阿里云被DDOS攻击

给主人留下些什么吧!~~
评论热议
请登录后评论。

登录 注册