Running a Twisted Perspective Broker example with twistd

I've been using Twisted's Perspective Broker to manage networking for my Python program. Perspective Broker allows me to run a Python program on a remote machine and perform remote method calls on an object in the Python program. It also allows me to serialize objects and transfer them over TCP.

Once I got a Perspective Broker server and client running, I wanted to create a "Twisted Application" and run it using twistd, the Twisted Daemon. Two major options are: creating a .tac file and creating a twistd plugin. Below, I show the steps I took to create a .tac application script and a twistd plugin using a simple Perspective Broker example.

Basic Perspective Broker server and client

Here is a simple Perspective Broker server and client example which I adapted from Twisted's examples page. The PB server code lives in pbsimpleserver.py:

from twisted.spread import pb
from twisted.internet import reactor

class Echoer(pb.Root):
    def remote_echo(self, st):
        print 'echoing:', st
        return st

if __name__ == '__main__':
    serverfactory = pb.PBServerFactory(Echoer())
    reactor.listenTCP(8789, serverfactory)
    reactor.run()

Here is a simple PB client, pbsimpleclient.py:

 

from twisted.spread import pb
from twisted.internet import reactor

class EchoClient(object):
    def connect(self):
        clientfactory = pb.PBClientFactory()
        reactor.connectTCP("localhost", 8789, clientfactory)
        d = clientfactory.getRootObject()
        d.addCallback(self.send_msg)

    def send_msg(self, result):
        d = result.callRemote("echo", "hello network")
        d.addCallback(self.get_msg)

    def get_msg(self, result):
        print "server echoed: ", result

if __name__ == '__main__':
    EchoClient().connect()
    reactor.run()

This code connects the client to the server using port 8789. It sends a message by calling the server's remote_echo method with an argument of "hello network". The server prints the message and returns the same message to the client. The client then prints the message.

To test the code, I ran both client and server on my local machine. I ran python pbsimpleserver.py in one terminal shell, then ran python pbsimpleclient.py in another shell. In the server shell, I got:

 

echoing: hello network

 

and in the client shell I got:

server echoed: hello network

 

To stop the client and server, I hit CTRL-C in both shells

Converting to a Twisted Application (.tac script)

To create a Twisted application, I used the twisted.application.service.Application object. When converting the example code above to an Application, I replaced twisted.internet.reactor.listenTCP with twisted.application.internet.TCPServer and twisted.internet.reactor.connectTCP with twisted.application.internet.TCPClient. Below is my server application, pbsimpleserver_app.py. (Note: the two files below are considered .tac files even though the filename doesn't end in .tac. From the documentation, .tac files are Python files which configure an Application object and assign this object to the top-level variable "application".)

from twisted.spread import pb from twisted.application.internet import TCPServer from twisted.application.service import Application class Echoer(pb.Root): def remote_echo(self, st): print 'echoing:', st return st serverfactory = pb.PBServerFactory(Echoer()) application = Application("echo") echoServerService = TCPServer(8789, serverfactory) echoServerService.setServiceParent(application)

Here is my client application, pbsimpleclient_app.py:

from twisted.spread import pb from twisted.application.internet import TCPClient from twisted.application.service import Application class EchoClient(object): def send_msg(self, result): d = result.callRemote("echo", "hello network") d.addCallback(self.get_msg) def get_msg(self, result): print "server echoed: ", result e = EchoClient() clientfactory = pb.PBClientFactory() d = clientfactory.getRootObject() d.addCallback(e.send_msg) application = Application("echo") echoClientService = TCPClient("localhost", 8789, clientfactory) echoClientService.setServiceParent(application)

To run these as daemons with twistd, I executed:

twistd -l server.log --pidfile server.pid -y pbsimpleserver_app.py

then:

twistd -l client.log --pidfile client.pid -y pbsimpleclient_app.py

This created the log file, server.log:

2008-10-27 00:08:35-0700 [-] Log opened. 2008-10-27 00:08:35-0700 [-] twistd 8.1.0 (/usr/bin/python 2.5.2) starting up 2008-10-27 00:08:35-0700 [-] reactor class: 2008-10-27 00:08:35-0700 [-] twisted.spread.pb.PBServerFactory starting on 8789 2008-10-27 00:08:35-0700 [-] Starting factory 2008-10-27 00:08:53-0700 [Broker,0,127.0.0.1] echoing: hello network

and client.log:

2008-10-27 00:08:53-0700 [-] Log opened. 2008-10-27 00:08:53-0700 [-] twistd 8.1.0 (/usr/bin/python 2.5.2) starting up 2008-10-27 00:08:53-0700 [-] reactor class: 2008-10-27 00:08:53-0700 [-] Starting factory 2008-10-27 00:08:53-0700 [Broker,client] server echoed: hello network

 

Creating a twistd plugin

In order to pass command line arguments to my Twisted application daemon, I need to create a twistd plugin. The following is how I implemented my plugin after reading the Writing a twistd plugin documentation. Here is my directory structure. EchoProj is located in ~/Projects.

EchoProj |-- echoproj | |-- __init__.py | |-- pbsimpleclient.py | `-- pbsimpleserver.py `-- twisted `-- plugins |-- echoclient_plugin.py `-- echoserver_plugin.py

pbsimpleserver.py:

from twisted.spread import pb class EchoServer(pb.Root): def remote_echo(self, st): print 'echoing:', st return st

pbsimpleclient.py:

class EchoClient(object): def send_msg(self, result): d = result.callRemote("echo", "hello network") d.addCallback(self.get_msg) def get_msg(self, result): print "server echoed: ", result

echoserver_plugin.py:

from zope.interface import implements from twisted.python import usage from twisted.plugin import IPlugin from twisted.application.service import IServiceMaker from twisted.application.internet import TCPServer from twisted.spread import pb from echoproj.pbsimpleserver import EchoServer class Options(usage.Options): optParameters = [["port", "p", 8789, "The port number to listen on."]] class MyServiceMaker(object): implements(IServiceMaker, IPlugin) tapname = "echoserver" description = "Echo Server" options = Options def makeService(self, options): serverfactory = pb.PBServerFactory(EchoServer()) return TCPServer(int(options["port"]), serverfactory) serviceMaker = MyServiceMaker()

echoclient_plugin.py:

from zope.interface import implements from twisted.python import usage from twisted.plugin import IPlugin from twisted.application.service import IServiceMaker from twisted.application.internet import TCPClient from twisted.spread import pb from echoproj.pbsimpleclient import EchoClient class Options(usage.Options): optParameters = [["port", "p", 8789, "The port number to connect to."], ["host", "h", "localhost", "The host machine to connect to."] ] class MyServiceMaker(object): implements(IServiceMaker, IPlugin) tapname = "echoclient" description = "Echo Client" options = Options def makeService(self, options): e = EchoClient() clientfactory = pb.PBClientFactory() d = clientfactory.getRootObject() d.addCallback(e.send_msg) return TCPClient(options["host"], int(options["port"]), clientfactory) serviceMaker = MyServiceMaker()

I set the PYTHONPATH to include the top-level project directory:

export PYTHONPATH="$HOME/Projects/EchoProj:$PYTHONPATH"

Running twistd --help now showed "echoserver" and "echoclient" in the list of commands. To run my server and client as daemons using port 8790 on my local machine, I executed:

twistd -l server.log --pidfile server.pid echoserver -p 8790

and

twistd -l client.log --pidfile client.pid echoclient -p 8790

This produced the logfiles, server.log:

2008-10-27 11:49:12-0700 [-] Log opened. 2008-10-27 11:49:12-0700 [-] twistd 8.1.0 (/usr/bin/python 2.5.2) starting up 2008-10-27 11:49:12-0700 [-] reactor class: 2008-10-27 11:49:12-0700 [-] twisted.spread.pb.PBServerFactory starting on 8790 2008-10-27 11:49:12-0700 [-] Starting factory 2008-10-27 11:49:17-0700 [Broker,0,127.0.0.1] echoing: hello network

and client.log:

2008-10-27 11:49:17-0700 [-] Log opened. 2008-10-27 11:49:17-0700 [-] twistd 8.1.0 (/usr/bin/python 2.5.2) starting up 2008-10-27 11:49:17-0700 [-] reactor class: 2008-10-27 11:49:17-0700 [-] Starting factory 2008-10-27 11:49:17-0700 [Broker,client] server echoed: hello network

from:
http://www.saltycrane.com/blog/2008/10/running-twisted-perspective-broker-example-twistd/