FFI Library

+ +The FFI library allows calling external C functions and +using C data structures from pure Lua code. + +

+ +The FFI library largely obviates the need to write tedious manual +Lua/C bindings in C. No need to learn a separate binding language +— it parses plain C declarations! These can be +cut-n-pasted from C header files or reference manuals. It's up to +the task of binding large libraries without the need for dealing with +fragile binding generators. + +

+The FFI library is tightly integrated into LuaJIT (it's not available +as a separate module). The code generated by the JIT-compiler for +accesses to C data structures from Lua code is on par with the +code a C compiler would generate. Calls to C functions can +be inlined in JIT-compiled code, unlike calls to functions bound via +the classic Lua/C API. +

+This page gives a short introduction to the usage of the FFI library. +Please use the FFI sub-topics in the navigation bar to learn more. +

+ +

Motivating Example: Calling External C Functions

+It's really easy to call an external C library function: +

+①
+②
+
+
+③local ffi = require("ffi")
+ffi.cdef[[
+int printf(const char *fmt, ...);
+]]
+ffi.C.printf("Hello %s!", "world")
+

+So, let's pick that apart: +

+① Load the FFI library. +

+② Add a C declaration +for the function. The part inside the double-brackets (in green) is +just standard C syntax. +

+③ Call the named +C function — Yes, it's that simple! +

+Actually, what goes on behind the scenes is far from simple: ③ makes use of the standard +C library namespace ffi.C. Indexing this namespace with +a symbol name ("printf") automatically binds it to the +standard C library. The result is a special kind of object which, +when called, runs the printf function. The arguments passed +to this function are automatically converted from Lua objects to the +corresponding C types. +

+Ok, so maybe the use of printf() wasn't such a spectacular +example. You could have done that with io.write() and +string.format(), too. But you get the idea ... +

+So here's something to pop up a message box on Windows: +

+local ffi = require("ffi")
+ffi.cdef[[
+int MessageBoxA(void *w, const char *txt, const char *cap, int type);
+]]
+ffi.C.MessageBoxA(nil, "Hello world!", "Test", 0)
+

+Bing! Again, that was far too easy, no? +

+Compare this with the effort required to bind that function using the +classic Lua/C API: create an extra C file, add a C function +that retrieves and checks the argument types passed from Lua and calls +the actual C function, add a list of module functions and their +names, add a luaopen_* function and register all module +functions, compile and link it into a shared library (DLL), move it to +the proper path, add Lua code that loads the module aaaand ... finally +call the binding function. Phew! +

+ +

Motivating Example: Using C Data Structures

+The FFI library allows you to create and access C data +structures. Of course, the main use for this is for interfacing with +C functions. But they can be used stand-alone, too. +

+Lua is built upon high-level data types. They are flexible, extensible +and dynamic. That's why we all love Lua so much. Alas, this can be +inefficient for certain tasks, where you'd really want a low-level +data type. E.g. a large array of a fixed structure needs to be +implemented with a big table holding lots of tiny tables. This imposes +both a substantial memory overhead as well as a performance overhead. +

+Here's a sketch of a library that operates on color images, plus a +simple benchmark. First, the plain Lua version: +

+local floor = math.floor
+
+local function image_ramp_green(n)
+  local img = {}
+  local f = 255/(n-1)
+  for i=1,n do
+    img[i] = { red = 0, green = floor((i-1)*f), blue = 0, alpha = 255 }
+  end
+  return img
+end
+
+local function image_to_gray(img, n)
+  for i=1,n do
+    local y = floor(0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue)
+    img[i].red = y; img[i].green = y; img[i].blue = y
+  end
+end
+
+local N = 400*400
+local img = image_ramp_green(N)
+for i=1,1000 do
+  image_to_gray(img, N)
+end
+

+This creates a table with 160.000 pixels, each of which is a table +holding four number values in the range of 0-255. First, an image with +a green ramp is created (1D for simplicity), then the image is +converted to grayscale 1000 times. Yes, that's silly, but I was in +need of a simple example ... +

+And here's the FFI version. The modified parts have been marked in +bold: +

+①
+
+
+
+
+
+②
+
+③
+④
+
+
+
+
+
+
+③
+⑤local ffi = require("ffi")
+ffi.cdef[[
+typedef struct { uint8_t red, green, blue, alpha; } rgba_pixel;
+]]
+
+local function image_ramp_green(n)
+  local img = ffi.new("rgba_pixel[?]", n)
+  local f = 255/(n-1)
+  for i=0,n-1 do
+    img[i].green = i*f
+    img[i].alpha = 255
+  end
+  return img
+end
+
+local function image_to_grey(img, n)
+  for i=0,n-1 do
+    local y = 0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue
+    img[i].red = y; img[i].green = y; img[i].blue = y
+  end
+end
+
+local N = 400*400
+local img = image_ramp_green(N)
+for i=1,1000 do
+  image_to_grey(img, N)
+end
+

+Ok, so that wasn't too difficult: +

+① First, load the FFI +library and declare the low-level data type. Here we choose a +struct which holds four byte fields, one for each component +of a 4x8 bit RGBA pixel. +

+② Creating the data +structure with ffi.new() is straightforward — the +'?' is a placeholder for the number of elements of a +variable-length array. +

+③ C arrays are +zero-based, so the indexes have to run from 0 to +n-1. One might want to allocate one more element instead to +simplify converting legacy code. +

+④ Since ffi.new() +zero-fills the array by default, we only need to set the green and the +alpha fields. +

+⑤ The calls to +math.floor() can be omitted here, because floating-point +numbers are already truncated towards zero when converting them to an +integer. This happens implicitly when the number is stored in the +fields of each pixel. +

+Now let's have a look at the impact of the changes: first, memory +consumption for the image is down from 22 Megabytes to +640 Kilobytes (400*400*4 bytes). That's a factor of 35x less! So, +yes, tables do have a noticeable overhead. BTW: The original program +would consume 40 Megabytes in plain Lua (on x64). +

+Next, performance: the pure Lua version runs in 9.57 seconds (52.9 +seconds with the Lua interpreter) and the FFI version runs in 0.48 +seconds on my machine (YMMV). That's a factor of 20x faster (110x +faster than the Lua interpreter). +

+The avid reader may notice that converting the pure Lua version over +to use array indexes for the colors ([1] instead of +.red, [2] instead of .green etc.) ought to +be more compact and faster. This is certainly true (by a factor of +~1.7x). Switching to a struct-of-arrays would help, too. +

+However, the resulting code would be less idiomatic and rather +error-prone. And it still doesn't get even close to the performance of +the FFI version of the code. Also, high-level data structures cannot +be easily passed to other C functions, especially I/O functions, +without undue conversion penalties. +

+
+