Basic component of data race detection on M5

 

Please refer to

thank you!

 

I have been working on translating the Chinese articles into English, it will be complete soon...

For overview of M5, please refer to M5 simulator overview

The implementaion of Race
   In order to detect data race, I need a memory to record the collected memory access information and synchronization. As there is no effective way to distinguish the synchronization and from the low level hardware, I adopt the traditional way to instrument the program on synchronizations, and when they occur, pass down to the hardware. Then these informaiton can be used to aid data race detection combined with the memory access information which can be collected during run time in the hardware pipeline level.
   The most effective way to collect these information is to directly pass the synchronization information to the hardware. This work is divided into 2 steps: implement a small memory to attach in the M5 system and make it directly accessible by the application.
   I attach it to the I/O bus, implement it as an I/O device (in dev/, inherit from BasicPioDevice). In M5, to add a new class, I need to follow a procedure as followed: define the class using C++, as there is no need to use template, I declare it in .hh file and implement it in .cc file, there is no _impl.cc needed. Instantiate it in a .py file Race.py.

from m5.params import *
from Device import BasicPioDevice

class Race(BasicPioDevice):
    type = 'Race'
    devicename = Param.String("Race");

and add it to SConstript file to make it visible to the SCons system.

if env['FULL_SYSTEM']:
    SimObject('BadDevice.py')
    SimObject('Race.py')
    SimObject('Device.py')
    SimObject('DiskImage.py')
    SimObject('Ethernet.py')
    SimObject('Ide.py')
    SimObject('Pci.py')
    SimObject('Platform.py')
    SimObject('SimpleDisk.py')
    SimObject('Terminal.py')
    SimObject('Uart.py')
   ...... other classes......

The above steps finish the construction of a new class, then I add it the system by modifying the config/common/FSConfig.py file.

def makeLinuxAlphaSystem(mem_mode, mdesc = None):
    class BaseTsunami(Tsunami):
        ethernet = NSGigE(pci_bus=0, pci_dev=1, pci_func=0)
        ide = IdeController(disks=[Parent.disk0, Parent.disk2],
                            pci_func=0, pci_dev=0, pci_bus=0)

    self = LinuxAlphaSystem()
    if not mdesc:
        # generic system
        mdesc = SysConfig()
    self.readfile = mdesc.script()
    self.iobus = Bus(bus_id=0)
    self.membus = Bus(bus_id=1)
    self.bridge = Bridge(delay='50ns', nack_delay='4ns')
    self.physmem = PhysicalMemory(range = AddrRange(mdesc.mem()))
    self.race_v = Race(pio_addr=0x80140000000,devicename = "Race")
    self.race_v.pio = self.iobus.port
    self.bridge.side_a = self.iobus.port
    self.bridge.side_b = self.membus.port
    self.physmem.port = self.membus.port
    self.disk0 = CowIdeDisk(driveID='master')
    self.disk2 = CowIdeDisk(driveID='master')
    self.disk0.childImage(mdesc.disk())
    self.disk2.childImage(disk('linux-bigswap2.img'))
    self.tsunami = BaseTsunami()
    self.tsunami.attachIO(self.iobus)
    self.tsunami.ide.pio = self.iobus.port
    self.tsunami.ethernet.pio = self.iobus.port

The declaration of the Race class is as followed:

#include "dev/io_device.hh"
#include "params/Race.hh"
class Race : public BasicPioDevice
{
  private:
    std::string devname;

   
    /*
     * the address of shared variable
     */
    unsigned long long var;
    /*
     * the kind of synchronization operation
     */
    int target;
    /*
     * whether the synchronization operation addresses have been recorded
     */
    int init;
    /*
     * the tid of the read thread
     */
    unsigned long long tid;
    /*
     * the join address
     */
    Addr joinAddr;
    /*
     * the PC of the race instruction
     */
    Addr writeInst;
    Addr readInst;
 
  public:
    typedef RaceParams Params;
   
  protected:
    const Params* params() const
    {
    return dynamic_cast(_params);
    }

  public:
    Race(Params *p);
    int  readInit()  { return init; }
    Addr readVar()   { return var; }
    Addr readRInst() { return readInst; }
    Addr readWInst() { return writeInst;}
    void writeTid(int Tid) { printf("write %d\n",Tid);tid = Tid; }
    int  readTid() { return tid; }
    void writeJoinAddr(Addr addr) { printf("write %llx\n",addr); joinAddr = addr; }
    Addr readJoinAddr() { return joinAddr; }
    Tick read(PacketPtr pkt);
    Tick write(PacketPtr pkt);
 
  private:
//    int computeIndex(Addr var);
};

Most of the fields here are related to record the synchronizaton function. I inherit the BasicPioDevice to declare the Race class. The read(PacketPtr pkt) and write(PacketPtr pkt) functions are used to communicat with the applications. The PacketPtr is a pointer to the communication packet. In M5, typical constructor of a class is like Race(Params *p); The Params represents the parameters passed to the class. In compilation, the scons will automatically construct a param class for every class. (this is another difference between src/ and build/).

build/ALPHA_FS/params/Race.hh (generated automatically)
#ifndef __PARAMS__Race
#define __PARAMS__Race

#include
class Race;

#include "params/BasicPioDevice.hh"

struct RaceParams : public BasicPioDeviceParams
{
    Race* create();
    std::string devicename;
};

#endif

when a class is actually instantiated, the create here is invoked. This create is implemented in race.cc
Race* RaceParams::create()
{
  return new Race(this);
}
where the constructor is invoked
Race::Race(Params *p) : BasicPioDevice(p), devname(p->devicename)
{
  pioSize = 0x50;
  var = 0;
  readInst = 0;
  writeInst = 0;
  init = 0;
}
I assign the memory size of 0x50 bytes at first for this basic test.
    The direct communication between hardware and software is through a device driver. This is an architecture dependent program which has the priority of kernel while accessible from application. Before writing a kernel device driver, I need to compile the linux kernel for alpha first. The Makefile of compiling a device driver is as followed:
KERNELDIR=/media/BkTmp/linux-source/linux-alpha/linux-2.6.13
#KERNELDIR=/media/BkTmp/linux-source/linux-source-2.6.27
obj-m=race.o
PWD=$(shell pwd)
CC=$(CROSS_COMPILE)gcc
LD=$(CROSS_COMPILE)ld                                     
PWD=$(shell pwd)
all:
    $(MAKE) -C $(KERNELDIR) M=$(PWD)   #invoke the kernel kbuild
#    $(MAKE) -C $(KERNELDIR) SUBDIRS=$(PWD)
clean:
    rm -rf *.o *~ core .depend .*.cmd *.ko *.mod.c .tmp_versions Module* module*
and the make command is
make ARCH=alpha CROSS_COMPILE=/media/Study/CPU/CPUsource/M5/opt/crosstool/gcc-3.4.3-glibc-2.3.5/alpha-unknown-linux-gnu/bin/alpha-unknown-linux-gnu- 
the race.o will automatically find the race.c source file to compile. As I compile the driver for ALPHA on my X86 machine, so it requires a cross compiler and designating the targe architecture explicitly, and the full compiled version of ALPHA linux source code. In linux2.6, the kernel Kbuild takes up the job of compiling a kernel driver, so the Makefile above just navigate to the source file and invoke the Kbuild to build the driver for me. ARCH= designate the arch/ directory in the linux source code to aid compiling the driver, the CROSS_COMPILE designate the compileer to use. The kernel directory is already specified in the Makefile. There is subtle problem here, I can not put the ARCH= and CROSS_COMPILE= in the Makefile, or the compilation would fail, I don't know why...