How to create a repo mirror

Introduction

Creating a repo mirror is useful in many cases, especially when you want to save time and bandwidth to synchronize a remote repo on multiple local stations.

As you can guess, it requires two major steps to create a repo mirror which can be accessed remotely:

• Set up a remotely accessible git server, which provides git services via ssh/git, or furthermore http;
• Create a repo mirror from upstream, and change the manifest file accordingly;

These two steps will be discussed in following sections.

Publish git service via git-daemon

The first thing is to set up a server hosting git repositories. The following two articles can be referenced for more details:

2. Hosting git repositories

1. Step 1, add a linux user named “git” in the system and do some preparation work:
   
   1	$sudo adduser --system --shell /bin/sh --gecos 'GIT Version Control User' --group --disabled-password --home /home/git git

   
   You may need to set a password for “git” user:
   
   1 2	$sudo passwd git #Enter your password for git user

   
   Switch to git user and create a folder to hold all repositories:
   
   1 2	$su - git $mkdir -p ~git/repositories

   
   Now /home/git/repositories should have been created.
2. Step 2, configure git-daemon to run as a system service. Of course, to do this you need to have git-core package installed already.
   
   Create a init.d script for git-daemon:
   
   1	$sudo vim /etc/init.d/git-daemon

   
   Copy the following into git-daemon script:
   
   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63

   PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin     NAME=git-daemon     PIDFILE=/var/run/$NAME.pid     DESC="the git daemon"     DAEMON=/usr/lib/git-core/git-daemon     DAEMON_OPTS="--base-path=/home/git/repositories --export-all --verbose --syslog --detach --pid-file=$PIDFILE --user=git --group=nogroup"       test -x $DAEMON || exit 0       [ -r /etc/default/git-daemon ] && . /etc/default/git-daemon       . /lib/lsb/init-functions       start_git() {       start-stop-daemon --start --quiet --pidfile $PIDFILE \         --startas $DAEMON -- $DAEMON_OPTS     }       stop_git() {       start-stop-daemon --stop --quiet --pidfile $PIDFILE       rm -f $PIDFILE     }       status_git() {       start-stop-daemon --stop --test --quiet --pidfile $PIDFILE >/dev/null 2>&1     }       case "$1" in       start)       log_begin_msg "Starting $DESC"       start_git       log_end_msg 0       ;;       stop)       log_begin_msg "Stopping $DESC"       stop_git       log_end_msg 0       ;;       status)       log_begin_msg "Testing $DESC: "       if status_git       then         log_success_msg "Running"         exit 0       else         log_failure_msg "Not running"         exit 1       fi       ;;       restart|force-reload)       log_begin_msg "Restarting $DESC"       stop_git       sleep 1       start_git       log_end_msg 0       ;;       *)       echo "Usage: $0 {start|stop|restart|force-reload|status}" >&2       exit 1       ;;     esac       exit 0

   
   Start git-daemon service:
   
   1 2 3

   $ sudo chmod a+x /etc/init.d/git-daemon     $ sudo update-rc.d git-daemon defaults     $ sudo service git-daemon start

   
   Check git-daemon is running correctly:
   
   1 2 3 4 5 6

   $ service git-daemon status      * Testing the git daemon:      * Running     $ sudo netstat -an | grep 9418      tcp        0      0 0.0.0.0:9418            0.0.0.0:*               LISTEN      tcp6       0      0 :::9418                 :::*                    LISTEN

Now git-daemon is running and monitoring /home/git/repositories folder, any git repositories created in this folder or its sub-folder can be accessed via git://yourhost/path/to/your/gitrepo.

Create repo mirror and update manifest file

Once git-daemon is running, we can start to create a repo mirror.

1. Step 1, create a mirror repo use the following command:
   
   1 2 3 4 5 6

   $su - git $cd ~git/repositories $mkdir foo $cd foo $repo init -u git://url_to_your_uprestream_repo/foo/manifest --mirror $repo sync

2. Step 2, update the foo repo manifest on a client macine, this is important:
   
   1 2 3

   #On a client machine, clone foo manifest repo to local machine via ssh. #We need to commit change to server $git clone git@your_repo_mirror_server:repositories/foo/manifest

   
   Edit the manifest xml file as you needed, add your repo mirror server as the default remote target.
   For example, in the modification showed below, 10.21.0.101 is added as the default remote target.
   
   1 2 3 4 5 6 7 8 9 10 11 12 13 14

   diff --git a/dev.xml b/dev.xml index e160af1..b82f284 100644 --- a/dev.xml +++ b/dev.xml @@ -6,8 +6,10 @@    <remote  name="git-xxx"             fetch="git://xxx/"             review="xxx" /> +  <remote name="repo_mirror" +         fetch="git://your_repo_mirror_server/foo/" />    <default revision="xxxx" -           remote="git-xxx" /> +           remote="repo_mirror" />     ...

   
   Commit this change and push it back to repo mirror server:
   
   1 2	$git commit -a -m "xxxx" $git push origin

Now a READ-ONLY repo mirror is configured. You can init and sync it from it:

1 2	$repo init -u git://your_repo_mirror_server/foo/manifest $repo sync

This entry was posted in linux, Technical on June 27, 2013.