Edit on Jan 24, 2012: A new now takes top honors.
I write game libraries first, and then games. It's in my blood, or at least I'm in a rut. One way or another, it's a habit that I find hard to break.
And I've become a major fan of the programming language ; it's fast and small and easy to bind to C or C++. Or so I thought...
Warning; this post gets technical. If you don't like reading about C/C++ esoterica, don't hit "Read More." You've been warned. :)
Binding an individual function in Lua to C or C++ isn't that hard. But when you have an entire library of functions to bind, you want to automate it, at least a bit.
Previously I used some macros and template code to automate binding class members tied to a particular instance of a C++ class, but that doesn't really give you the full power of objects in Lua. For that you want the object itself to be bound to Lua.
So I looked around at some of the automated C++/Lua object binding options. I ended up evaluating a few:
Quite a monster; does everything you want, but at a price. And that price is that it's big, and requires you to include a large portion of the monstrous Boost SDK. I'm trying to write games on Android, and size matters: My executable grew by 203k with just a few bindings in LuaBind, and adding a extra class descriptions tacked on between 24k and 60k.
Android users start complaining when your app gets too big. I didn't want to have to resist adding additional Lua bindings, and my binary was already a good bit larger than I wanted, so I finally (with much regret) decided to step away from LuaBind and consider others. Update: I originally misread some numbers and thought that LuaBind had added more overhead than it had; it still seems to add a lot of code for each additional binding, though, so the same analysis holds.
Simple Lua BinderIt sounds good to start with. "Like LuaBind, but simpler." No Boost dependency was also a plus. It's very elegant and clean; adding member functions to a class is very simple and straightforward.
But it turned out to be too simple; it didn't support enough of the features I was looking for. In particular it doesn't seem to have any way to map member variables to Lua; functions only.
Documentation was also lacking. The "docs" folder contained a todo.txt file with two lines; and that's the entirety of the documentation. You have to dig through the unit tests to read comments about the few things that are demonstrated there. Ironically, SLB has a cool feature to add documentation right to the bindings, so that your Lua bindings can be self-documenting.
SLB also ended up benchmarking slower than the next option, OOLua. So by the end I decided to move on.
OOLuaNext up was OOLua. It supported a lot more features, though it lacks in elegance compared to LuaBind or SLB. This wasn't enough to scare me away, though, since I felt I could tame the issue by layering my own macros on top of OOLua's macros.
OOLua was slightly lighter than LuaBind in my tests, weighing in at about 153k; and my additional class binding test took closer to 20k. And OOLua was a lot faster than SLB or LuaBind; close to twice as fast in my benchmarks and benchmarks on the OOLua site.
But speed isn't everything -- and OOLua doesn't have integral support for smart pointers. I rely heavily on smart pointers, and so worked with the OOLua author to try to get things working, and I did appreciate his help, but we ran into a wall and so I decided to look for another solution.
tolua++I first resisted tolua++, since it's a preprocessor that creates C++ binding files, and so requires an extra build step, but given the above problems I finally decided to give it a try.
And to start with, it was great. I built the preprocessor with no issues and was well on my way to getting my first tolua++ code running, when I had a problem. And to track down the problem I needed to modify the Lua code built into tolua++, which required SCons.
Hate is a strong word, but let's just say that I used a lot of strong words in reference to the tolua++ authors' decision to use the SCons build system. I burned way too much time trying to get SCons to work on Windows under Cygwin, but due to a comedy of errors of Python versions, installers that only install to Windows Python and not Cygwin Python, and SCons insisting on a gcc build instead of an MSVC build, I ended up giving up and digging through the configuration files by hand to figure out how the Lua code with tolua++ was built and integrated into the executable. And figure it out I did; one shell script later and I was building tolua++ with custom Lua code.
The tolua++ parser is very primitive. Error messages tend to be limited to "Parse error", and don't include line numbers. They paste a string into the error message that happens somewhere near the parse error. Well, OK, there technically are several line numbers printed out, but they're all referring to that code that's been baked into the tolua++ parser, not the code they're parsing. Sigh.
So after a bunch of flailing until I got the code Just Right, I tested it. And...it can handle smart pointers (if you lie and tell tolua++ that they're derived from the object), but if you use a typedef to refer to them it won't recognize them. Even though it knows about the typedef, and could therefore translate it.
So I removed the typedefs from the spec and got the code to work. Mostly. It still can't recognize a smart pointer to a derived class can be passed to the base class; I'll have to dig in to see how it normally handles inheritance to see if I can handle that case. It may be fixable.
But what blew me away was my executable size. It weighs in at only 35k of overhead. Even better, adding my test binding took about 8k. That same binding takes about 50k in LuaBind. That's less than 1/6 the overhead per class, and less than 1/5 of the initial hit. Now we're talking.
tolua++ almost convinced me to stop the search, but I was on a roll.
MLuaBindI've heard good things about MLuaBind. Given my adventures with template-based libraries above, I wasn't sure I wanted to spend any more time evaluating another template-based Lua binding option.
But on paper at least it looked nice. A lot of the same functionality as LuaBind, but in only 40 source files instead of 97 files, and without a dependency on Boost's 5000+ files.
What made me shy away from it is the fact that it's dormant. The last update was over a year and a half ago, and it's listed as version 0.06. I know some projects start with low numbers and never get to 1.0, despite being widely used, but I only have anecdotal evidence that MLuaBind has been used enough to consider it reliable.
The killer was that it turns out MLuaBind doesn't build with GCC 4.4, which I need to build for Android. I'm not sure whether it's using features that are too new, or whether the standard has changed over time, but I don't have time to try to rewrite MLuaBind, so it's out as an option.
SWIGThe old standby for binding C/C++ to whatever language you want to use, SWIG seemed like it would be too big and complicated to use. It requires a preprocessing step, which I didn't like. But since I finally tried tolua++, I figured I could at least give SWIG a quick test, with the caveat that I'd give up if it started to fight me like tolua++ did.
Well, it didn't, at least at first. I had my first bindings functioning with a working SWIG line in my makefile in about 2 hours, start to finish. I'd never tried to actually use SWIG before, so it amazed me how fast it was to get it working.
Part of the ease of integration is that there's no library to link in; it generates all the code into a single C++ source file. Link that and call the registration function and you're done.
Runtime error messages seem to be strong -- instead of just telling you ONE parameter option for a function, it has pre-baked lists of all the possible parameter combinations ready to print out.
SWIG claims to natively supports smart pointers, but that turns out to be an exaggeration. It turns out that the smart pointer support was only for a subset of the languages SWIG supports; and that subset didn't include Lua. Luckily the support was in a single file that wasn't too hard to hack to work with Lua from the Python version. Still, it took time. It's frustrating when something so basic doesn't work out of the box.
SWIG's wrappers are also a bit clunky when it comes to static members of a class. Typically, if you have a class Game in namespace g with a static function get(), you can access it in Lua like this:
g.Game.get()
But static functions in SWIG are not so nice. Instead, you need to say:
g.Game_get()
Not in itself a deal-killer, but very un-Lua-like. It turns out to be trivial to add a table and function yourself if you really care:
g.Game = {}
g.Game.get = g.Game_get
The final question was size: And it starts out remarkably similar to tolua++ for a single "module"; that's what will compile to be a single .cpp file. My first test showed about 44k of overhead for the library module. So there was much rejoicing. It seemed I'd found the perfect (almost) solution, and I was ready to run with it.
But then I realized that stashing all the code into a .cpp file means that it will end up duplicating code for other modules. Oh oh. So I did the last part of the test and was shocked to see the second module added another 70k to the executable. The module has about 24 functions and 12 member variables spread around 6 classes. A bit more experimentation seems to indicate there's an overhead of about 1k per function bound and 2k per read-write member variable, which isn't too bad, plus only a few dozen bytes for a class, with a fixed overhead of about 32k/module, though nowhere near as thrifty as tolua++.
Requirements SNAFUAt about the point that I got all of the SWIG pieces functioning I realized that I'd missed an entire feature requirement: The ability to call Lua functions with user-defined data structures in parameters. SWIG offers no header files for your C++ code; it intends to be a one-way wrapper, to allow Lua to call your C++ functions.
C++ can return objects to Lua, but there's no mechanism for calling Lua functions from C++ with objects wrapped with the SWIG magic -- which is rather obscure, and tangled in static functions within the .cpp files it generates.
tolua++ has a clean interface that could be used pretty simply to call Lua functions with arbitrary user types as parameters. LuaBind has a full template-driven system to allow you to call arbitrary Lua functions with whatever objects types you've registered with LuaBind with no hassle.
ConclusionThe SWIG parser is more robust and has more features than tolua++, and the size is reasonable. tolua++ takes the prize for size, and at least leaves open the option of calling into Lua functions with objects.
LuaBind is also a strong contender if you don't care about code size; it's also feature rich, it's a bit faster than SWIG's bindings, and it lets you keep your code in C++.
OOLua is somewhat awkward to use, but if speed is the most important, it beats the rest -- in large part, I think, because it doesn't support function overloading, so it can skip a step on a function call.
I'm probably going to move forward with tolua++ at this point in my library. Its size is key, and I'll be able to make C++ call into Lua without much hassle. I know of a template library that just specializes in allowing you to call arbitrary Lua functions from C++, though that's the topic for another post.
Measurement SNAFUPartway through writing the original post my measurements started coming out smaller, and I didn't notice. I've gone through and built with each of the variants and calculated new and correct numbers. Sorry for the confusion.
Update 1:Added some information on the baseline size without any bindings.
Update 2:MLuaBind doesn't build, so it loses.
Update 3:Added SWIG & requirements SNAFU; rewrote conclusion; toned it down a bit. Developers have feelings too.
Update 4:Fixed some numbers that I'd read wrong because I'm an idiot sometimes. Sig