Generating bindings
tools/gen-bindings.js reads a C or C++ header with clang and writes a
JavaScript module that binds it: one CFunction per function,
a constant per enum value, and, on request, a class per struct, union and C++
class. The module is plain JavaScript that imports ffi; the generator is only
needed to make it.
This page follows a run of the generator from the command line to the output,
one section per processing step, with a small C and a small C++ library as the
running examples. The listings come from real runs of the tool (paths
shortened, long bodies elided with ...), and the usage examples ran against
the built libraries.
- Synopsis
- The examples
- Pipeline
- Structs and variables (
--structs) - C++ classes
- Wrappers for C conventions
- Output options
- The IR on its own
- Using the result
- Limitations
Synopsis
qjs-ffi-genbindings [options] <source>...
qjs-ffi-genbindings [options] --from-ir=<ir.json>qjs-ffi-genbindings is the installed name of qjsm tools/gen-bindings.js;
both take the same options. Run from a checkout:
qjsm tools/gen-bindings.js --library=libcairo.so.2 /usr/include/cairo/cairo.h -o lib/bindings/cairo.jsThe generator needs clang and qjsm with the json module of qjs-modules.
The generated module needs only ffi, and one with read (see
Pointers and memory).
| Option | Meaning |
|---|---|
<source>... | headers or sources to bind. Several are merged into one module. |
--library=<path> | dlopen() this shared library; without it the symbols are looked up in the running process (RTLD_DEFAULT). |
-o, --output=<path> | write the module here instead of to standard output. |
-I<dir>, -D<name[=value]> | passed on to clang; repeatable. --include= and --define= are the long forms. |
--follow-includes | also bind headers that a source includes from under its own directory (SDL.h brings in SDL_video.h). |
--exclude=<name> | do not bind this function or class; repeatable. |
--c++ | parse the sources as C++. Implied by .cc, .cpp, .cxx, .hh, .hpp, .hxx; a .h needs it. |
--std=<std> | the C++ standard for clang, e.g. c++17. |
--namespace=<name> | drop the C++ namespace prefix name:: from the names; repeatable. |
--structs | also wrap every struct and union as an ArrayBuffer class, and every extern variable. |
--target=<runtime> | qjs (default), bun, deno or node: the module runs under that runtime, see Bun, Deno and Node. |
--class-types | with the classes of --structs or --c++: write the class itself as the type of a T * argument or return (args: [Point], returns: Point) instead of "Point *", and give each such class a static size. Needs a qjs-ffi that takes a constructor as a type (classes as types); off by default. |
--cstring=<function> | the unsigned char * function returns is a string, see wrappers; repeatable. |
--handle=<type>:<dtor> | a type * is an object closed by dtor; repeatable. |
--out=<function>:<arg> | arg is an out-parameter the wrapper fills and returns; repeatable. |
--owned-return=<function>[:<dtor>] | the char * function returns is copied, then freed with dtor (default libc free); repeatable. |
--out-auto | also treat as --out every T ** of a --handle type and every pointer parameter named out* or pp<Name>, of functions returning void or an integer. |
--finalize | destroy a C++ object made with new when it is garbage collected, too. |
--ffitype | write types as FFIType.i32 instead of 'i32'. |
--describe | give each function its parameter names, and describe its C types for describeObject(). |
--jsdoc | a JSDoc comment on each function, method and class. |
--emit-ir=<file> | write the intermediate JSON and stop. |
--emit-specs=<file> | write the dlopen() symbol specs as JSON and stop, see Specs from the IR. Not with --emit-ir. |
--emit-specs | the same without a file name: the specs go to stdout (or -o) as a JS module, unless --json. |
--js | write the --emit-ir or --emit-specs file as a JS module, export default { ... };, instead of JSON. |
--json | write the --emit-ir or --emit-specs file as JSON. The default, except for a bare --emit-specs. Not with --js. |
--from-ir=<file> | generate from an IR instead of running clang. |
--clang=<path> | the clang binary (default clang). |
--cache-dir=<dir> | where the condensed AST is cached (default .tmp/gen-bindings). |
--no-cache | do not read or write that cache. |
--no-auto-include | do not add -I<dir> for a header clang could not find in a parent of the source; the missing header is only a warning. |
-h, --help | the usage text. |
Companions in tools/: gen-structs.js (qjs-ffi-genstructs) makes the struct
classes, or a C header, from an IR on its own, and describe.sh lists
what a module, object or class has. Both are covered
below.
The examples
A small C library, geom.h:
#define GEOM_VERSION 3
typedef enum { GEOM_OK = 0, GEOM_BAD_ARG = -1, GEOM_OVERFLOW = 2 } geom_status;
typedef struct vec2 {
float x, y;
} vec2;
typedef struct shape {
int id;
const char *name;
vec2 origin;
unsigned visible : 1;
} shape;
typedef int (*geom_visit_fn)(const shape *s, void *user);
extern int geom_count;
int geom_abs(int v);
double geom_length(vec2 v);
vec2 geom_add(vec2 a, vec2 b);
const char *geom_version(void);
geom_status geom_move(shape *s, double dx, double dy);
shape *geom_find(const char *name);
void geom_each(geom_visit_fn fn, void *user);
int geom_log(const char *fmt, ...);and a C++ one, shapes.hpp:
namespace geo {
class Shape {
public:
Shape(double scale = 1.0);
virtual ~Shape();
virtual double area() const = 0;
double scale() const;
void setScale(double s);
static int count();
int id;
private:
double s_;
};
class Circle : public Shape {
public:
Circle(double radius);
double area() const override;
double radius;
};
double lerp(double a, double b, double t);
}Both are built into shared libraries (libgeom.so, libshapes.so) and bound
with
qjs-ffi-genbindings --library=./libgeom.so --structs -o geom.js geom.h
qjs-ffi-genbindings --c++ --std=c++17 --namespace=geo --library=./libshapes.so -o shapes.js shapes.hppPipeline
source ──► 1 command line ──► 2 clang AST ──► 3 layout probes
│
JavaScript ◄── 8 emit ◄── 7 names ◄── 6 by value ◄─┤
▲ ▼
└──── 4 collect ──► 5 merge ──► IR (JSON)Steps 2 to 6 turn the sources into the IR, a JSON description of the C API;
steps 7 and 8 turn the IR into JavaScript without looking at clang or the
sources again. --emit-ir stops after step 6 and --from-ir starts at step 7,
see The IR on its own.
1. Command line
args.js reads the options from one table, which also makes the --help
text. It refuses an unknown option, --from-ir together
with sources or with --emit-ir, and no source at all. A problem prints the
message and the usage text and exits with status 1. The generated file's header records the options
it was made with, so that it can say how to regenerate it:
/* Auto-generated by qjs-ffi-genbindings from geom.h -- do not edit by hand.
* Regenerate with:
* qjs-ffi-genbindings --library=./libgeom.so --structs geom.h
* (clang invocation used to build the AST, once per source: clang -Xclang -ast-dump=json -fsyntax-only <source>)
*/2. Running clang
For each source, clang.js runs
clang -Xclang -ast-dump=json -fsyntax-only [-x c++ -std=<std>] -I<dir>... -D<def>... <source>and reads clang's JSON AST from the pipe and its errors from stderr, which
goes to a file in the cache directory during the same run. An error leaves
the AST wrong from there on: a declaration whose macros (XMLPUBFUN) are
undefined turns into a variable. So the generator acts on what clang said:
- a header it could not find (
<libxml/xmlversion.h>) is looked for in the source's own directory and its parents; the nearest one that has it is added as-I<dir>(a note says so,/usr/include/libxml2forlibxml/tree.h), for the other sources and the layout probe too, and clang runs again, at most four times (--no-auto-includeskips this) - anything left over is a warning with the error count and the first message,
and, for a header still not found, a hint to add
-I<dir>yourself
That AST is huge (megabytes for
cairo.h), almost all of it source ranges, function bodies and attributes the
generator never uses. So it is not parsed as one value: json.JsonParser
yields tokens and condense.js keeps only what is needed, deciding from a
node's kind, as soon as it arrives, which children survive:
- at the top level: functions, variables, enums, records, typedefs, C++
classes and namespaces (
stdand__*namespaces are dropped); - below a function: its parameters, nothing else;
- below an enum: its constants; below a record: its fields, nested records and enums, alignment and packing attributes; below a class also its methods, constructors, destructor and access specifiers;
- constant expressions are kept as
kind/value/opcode/inneronly.
The condensed AST is cached in .tmp/gen-bindings/<name>.<hash>.ast.json
(--cache-dir), the hash covering the clang command line, the source path and
the cache format. A later run reuses it as long as no file it was built from is
newer, which makes a repeat run take a fraction of a second. --no-cache turns
the cache off.
An unreadable source, or a parse that fails, ends the run with clang's own diagnostics:
gen-bindings.js: failed to parse clang AST JSON output: unexpected end of clang AST JSON; stderr was:
clang: error: no such file or directory: 'nothere.h'3. Layout probes
clang's AST has no sizes and no field offsets, and a binding needs both: to
read a struct member, to pass a struct by value, to find a virtual method in a
vtable. They only come from a record layout dump, and clang prints one only for
a record that code needs. So runLayoutDump() writes a probe, a translation
unit that includes the source and asks for the numbers, and compiles it with
-Xclang -fdump-record-layouts-simple:
#include "/path/to/geom.h"
enum { __probe0 = sizeof(struct vec2) };
enum { __probe1 = sizeof(struct shape) };
struct __gb_f0 { char b[sizeof(((struct vec2 *)0)->x)]; };
enum { __gb_fu0 = sizeof(struct __gb_f0) };
...The dump gives each record's size, alignment and the bit offset of each field.
A field's own size is read from the dump of a tiny struct holding
char[sizeof(field)], and a typedef's from the same trick. For a polymorphic
C++ class the probe also calls the destructor and takes the address of a
virtual method, and the compile is then run with `-fdump-vtable-layouts -S
-emit-llvm`, which prints
VTable indices for 'geo::Shape' (3 entries).
0 | geo::Shape::~Shape() [complete]
1 | geo::Shape::~Shape() [deleting]
2 | double geo::Shape::area() constvtable.js parses these lines into slot numbers. They are stored with the
condensed AST, so the cache covers the probes too. A field with no size (a
bitfield, a flexible array, an incomplete type) simply has none.
4. Collecting the IR
collectIR() walks the declarations in the files it is to bind (the source, or
with --follow-includes anything under its directory), keeping track of
clang's "current file", and fills the IR. static functions are left out, and
static variables unless they are constants with a known value. --emit-ir
shows it:
qjs-ffi-genbindings --emit-ir=geom.ir.json geom.hA function becomes an entry of methods, its C types mapped to ffi type
names:
{ name: 'geom_abs', kind: 'function',
args: ['v: i32'], returns: 'i32', enums: [] }
{ name: 'geom_move', kind: 'function',
args: ['s: shape *', 'dx: f64', 'dy: f64'], returns: 'i32', enums: ['0:0x5e1031dee5d0'], ... }
{ name: 'geom_find', kind: 'function',
args: ['name: cstring'], returns: 'shape *', ... }types.js does the mapping, one C type at a time (typedefs are resolved
first, const and volatile dropped):
| C type | ffi type |
|---|---|
int, unsigned, long, size_t, float, double, _Bool, ... | i32, u32, i64, u64, f32, f64, bool |
char * | cstring |
T *, T **, void * | "T *", "T **", "void *": a pointer, the text before * is documentation |
T &, T && (C++) | "T *" |
| function pointer | function |
enum E, a typedef of an enum | i32, remembering which enum, see below |
struct S by value | struct S, settled in step 6 |
array parameter int a[4] | "int *" |
union by value, an unknown name | skipped |
Everything else the IR holds:
enums: every enum with its constants and their values. Only an enum that a bound function's signature uses is emitted later, in order of first use.fields: variables, and constants with a known value:{ "name": "geom_count", "type": "i32", "cType": "int" }. Aconstvariable with a literal initializer carries itsvalueand becomes anexport const.defines: object-like#defineconstants scanned from the header text (GEOM_VERSION), as{ "name": "GEOM_VERSION", "value": 3 }; see the limitations.structsandclasses: each struct, union and class with its size, alignment, and its fields, and each field with its type, byteoffset, bytesizeandffitype. A bitfield also hasbitsandbitOffsetand the offset and size of its storage unit. The numbers come from the probes, or else from the type alone.{ name: 'shape', type: 'struct', size: 32, align: 8, line: 12, fields: [ { name: 'id', type: 'int', offset: 0, size: 4, ffi: 'i32' }, { name: 'name', type: 'const char *', offset: 8, size: 8, ffi: 'cstring' }, { name: 'origin', type: 'vec2', offset: 16, size: 8, ffi: 'vec2' }, { name: 'visible', type: 'unsigned int', offset: 24, size: 4, bits: 1, bitOffset: 0, ffi: 'u32' } ], typedefs: ['shape'] }typedefs: everytypedefandusingalias, with the type it stands for and the record it names. One declared inside a namespace or class is known by its qualified name and by the bare one clang spells where it is in scope (stk::StkFloatandStkFloat), so a method returning it binds.warnings: bound, but with a caveat:{ name, reason }for a virtual method of a class that is notfinaland got no vtable slot, so it is bound to its own symbol and a call through a base class wrapper is not virtual. They are written as// Warnings:at the end of the module and counted on stderr. An IR file from before this key has none.skipped: what could not be bound and why. Operators are skipped here, an unknown type or a union by value by the mapping (a variadic function, C or C++, is bound, withvariadic: true):ir.skipped = [{ name: 'geom_ops', reason: 'operators are not supported' }];
A C++ class is the same, plus its bases, its public constructors, methods and
destructor with their mangledName (what to dlsym()), and for each virtual
method and the destructor its vtableSlot:
{ name: 'geo::Shape', type: 'class', size: 24, align: 8, abstract: true, polymorphic: true,
bases: [], constructors: [],
fields: [{ name: 'id', type: 'int', offset: 8, size: 4, ffi: 'i32' }],
methods: [
{ name: 'area', mangledName: '_ZNK3geo5Shape4areaEv', virtual: true, pure: true, const: true,
args: [], returns: 'f64', vtableSlot: 2 },
{ name: 'setScale', mangledName: '_ZN3geo5Shape8setScaleEd', args: ['s: f64'], returns: 'void' },
{ name: 'count', mangledName: '_ZN3geo5Shape5countEv', static: true, args: [], returns: 'i32', ... } ],
destructor: { mangledName: '_ZN3geo5ShapeD1Ev', virtual: true, vtableSlot: 0 } }A parameter with a literal default (int b = 3, -1.5, true, "hi",
nullptr, an enum constant, integer arithmetic of these) is recorded as
defaults, the values of the trailing run of such parameters
(mix(int a, int b = 3, int c = -1) has defaults: [3, -1]), and the call may
leave those arguments out: mix(1), mix(1, 5). --describe puts them in the signature as defaults, --jsdoc writes [b=3]. A default that is a call or a
name (Size(), kDefault) ends the run, so that parameter stays required.
Overloads are separate entries. A constructor carries the mangled name of the
complete-object constructor (C1); an abstract class has none, because the
compiler emits only the base-object one. Only public members are collected, and
a struct with no methods and no bases stays a plain structs entry even in C++.
5. Merging sources
With several sources, each one's IR is merged into one. The first declaration of a name wins, so a function or enum in two headers is bound once, and a variable declared without a value is upgraded by a later declaration with one. A source with nothing to bind prints `warning: no bindable functions found in <source>` and the run continues.
6. Structs by value
A function that takes or returns a struct by value (vec2 geom_add(vec2, vec2))
is bound through a struct type that CFunction takes as an array of member
types (Types and ABI). libffi works out the size,
the alignment and the registers from that list alone, so a list that does not
reproduce the compiler's layout is silently wrong. by-value.js therefore lays
the list out the way libffi does and compares it with the IR: every member must
land on the offset the IR has, and the size must match. If it does, the list
goes into ir.byValue:
ir.byValue = { vec2: ['f32', 'f32'] };and the function's types become struct vec2. If not, the function is skipped
with the reason it would have had without by-value support:
struct __attribute__((packed)) pk { char c; int i; };
union u { int a; float b; };
int takes_packed(struct pk p);
int takes_union(union u v);
int takes_array(int a[4]);export const takes_array = __bind('takes_array', () => CFunction({ptr:__sym('takes_array'),args:['int *'],returns:'i32'}));
// Skipped (unsupported):
// - takes_union: parameter v (union u): union passed by value is not supported
// - takes_packed: parameter p: struct pk passed by value: libffi would put i at 4, not 1An anonymous struct or union member is described in the IR (members of the
field), so its layout is known: a struct is laid out like any other, a union
when all its alternatives give the same element list. nanovg's colour
typedef struct NVGcolor { union { float rgba[4]; struct { float r, g, b, a; }; }; } NVGcolor;is [['f32','f32','f32','f32']] and travels in SSE registers, and a struct that
holds two of them stays 76 bytes, not 80. A union whose alternatives differ
(union { int i; float f; }) or a member whose layout is not known is covered
by integers of the right size. That is sound only for a struct of more than 16
bytes, which is passed in memory whatever it holds; a smaller one, whose members
decide which registers it travels in, is skipped. Named unions are never bound.
C++ types by value are bound when they pass like a C struct:
- a class that clang says can be passed in registers (
canPassInRegisters: trivial copy and destructor, not polymorphic) with no base class, such asImVec2with its constructors, is laid out like a struct. A class with a copy constructor or destructor of its own is passed by hidden reference and stays unbound; - an instantiation of a class template that the header names,
Size_<int>orcv::Size_<float>, is a struct of its data members with the parameters replaced (T widthisint width), no methods; the sizes come from clang. A nested instantiation (W<Size_<int>>, which needsSize_<Size_<int>>) is made too, even when clang never listed it; - a trivially copyable class with one non-virtual base is by value as well: the base's members come first;
- a template with non-type parameters works the same (
Vec<float, 3>is a struct withfloat v[3]); - a name clang spells unqualified inside its namespace (
Size_<float>forcv::Size_<float>) is the record when that short name is unique.
7. Names
C++ names are qualified (geo::Shape) and become identifiers by replacing ::
with _ (geo_Shape). --namespace=geo drops that prefix instead
(Shape); with several --namespace options all are dropped, so with a and
b, a::b::Foo is Foo. Dropping what told two names apart can make them
collide, so names.js checks every name the module would export, and refuses to
write anything if two things would share one:
$ qjs-ffi-genbindings --c++ --namespace=a --namespace=b ns.hpp
gen-bindings.js: after dropping a::, b::, "size" would be exported by: function a::size; function b::size
gen-bindings.js: name collision, nothing written; drop fewer --namespace valuesOnly a clash that the stripping caused is reported. A struct and a function
share a name in C (struct lws_xos and lws_xos()) but not in JavaScript, where
both are exports; the class then gets _t (lws_xos_t, and _ is added
while that is taken too). A function whose name is a JavaScript
reserved word is exported with an underscore in front (delete as _delete),
with a comment beside the export.
8. Emitting JavaScript
emit/functions.js writes the module, in this order: the header comment, the
import of what the module needs from ffi, the library, __sym(), the
enums, the support code (only if there are structs, classes or variables), the
struct types for by-value functions, the functions, and the list of skipped
declarations.
The import adds names as they are needed: CFunction, dlsym, FFIType with
--ffitype, read as __rd, ptr as __ptr and the others for struct and class
wrappers. With --library the module opens that library and resolves symbols in
it, without it in the process:
import { CFunction, dlsym, dlopen, RTLD_NOW } from 'ffi';
const __lib = dlopen('libgeom.so', RTLD_NOW);
if (__lib == null) throw new Error('gen-bindings: dlopen(libgeom.so) failed');
function __sym(name) {
const p = dlsym(__lib, name);
if(p == null) throw Object.assign(new Error('gen-bindings: symbol not found: ' + name), { missing: name });
return p;
}A function whose symbol the library lacks (declared under an #ifdef, or for
another platform) does not stop the import: __bind() makes it a stub that
throws gen-bindings: symbol not found: NAME when called. That holds for every
target: bun and deno look each symbol up on its own, node's dlsym throws, and
each tags the error as a missing symbol (a library that does not open still
fails the import). tests/test-gen-bindings-names.js checks bun, deno and node.
An enum constant used by a function signature is exported, with the
enum's name (or (anonymous)) as a comment:
// enum (anonymous)
export const GEOM_OK = 0;
export const GEOM_BAD_ARG = -1;
export const GEOM_OVERFLOW = 2;A function is a CFunction. The result for geom.h (without --structs):
const __s_vec2 = ['f32','f32'];
export const geom_abs = __bind('geom_abs', () => CFunction({ptr:__sym('geom_abs'),args:['i32'],returns:'i32'}));
export const geom_length = __bind('geom_length', () => CFunction({ptr:__sym('geom_length'),args:[__s_vec2],returns:'f64'}));
export const geom_add = __bind('geom_add', () => CFunction({ptr:__sym('geom_add'),args:[__s_vec2,__s_vec2],returns:__s_vec2}));
export const geom_version = __bind('geom_version', () => CFunction({ptr:__sym('geom_version'),args:[],returns:'cstring'}));
export const geom_move = __bind('geom_move', () => CFunction({ptr:__sym('geom_move'),args:['shape *','f64','f64'],returns:'i32'}));
export const geom_find = __bind('geom_find', () => CFunction({ptr:__sym('geom_find'),args:['cstring'],returns:'shape *'}));
export const geom_each = __bind('geom_each', () => CFunction({ptr:__sym('geom_each'),args:['function','void *'],returns:'void'}));
export const geom_log = __bind('geom_log', () => CFunction({ptr:__sym('geom_log'),args:['cstring'],returns:'i32',variadic:true}));A variadic C function such as geom_log is bound with variadic: true and its
fixed arguments, and called with (type, value) pairs for the rest, see
Variadic functions. A variadic C++
function or method is bound the same way; among a class's overloads it takes
any call with at least its fixed arguments, so acc.add(2, "i32", 1, "i32", 2)
and acc.add("label") pick different ones (tests/test-gen-bindings-variadic.js).
| C | JavaScript |
|---|---|
int geom_abs(int) | geom_abs(-5) is 5 |
const char *geom_version() | a string: 'geom 3' |
shape *geom_find(const char *) | a pointer: null, a number or a bigint |
geom_status geom_move(shape *, double, double) | pass a pointer, or a struct class; returns the status as a number |
void geom_each(geom_visit_fn, void *) | pass a JSCallback's .ptr |
double geom_length(vec2) | pass an ArrayBuffer of at least 8 bytes, or a vec2 instance |
vec2 geom_add(vec2, vec2) | returns an ArrayBuffer of 8 bytes (a vec2 with --structs) |
The wrappers are used like any CFunction:
import * as geom from './geom.js';
import { JSCallback } from 'ffi';
console.log(geom.geom_abs(-5)); // 5
console.log(geom.geom_version()); // geom 3
const seen = [];
const cb = new JSCallback((p, user) => { seen.push(geom.shape.at(p).id); return 0; }, { args: ['pointer', 'pointer'], returns: 'i32' });
geom.geom_each(cb.ptr, null);
console.log(seen.join()); // 1,2
cb.close();Structs and variables (--structs)
With --structs every struct and union of the sources gets a class, each
typedef name of it an alias, and every extern variable an accessor. The classes
extend ArrayBuffer, so an instance can be passed to a pointer parameter as it
is. For vec2 and shape the generator writes
/* struct vec2, 8 bytes (line 8) */
export class vec2 extends ArrayBuffer {
constructor(init = 8) { super(typeof init === 'number' ? init : init.byteLength); if(typeof init !== 'number') new Uint8Array(this).set(__bytes(init)); }
static at(p, owner, size = 8) { return __view(vec2, p, size, owner); }
get ptr() { return __ptr(this); }
get x() { return __rd.f32(this, 0); }
set x(v) { __wr.f32(this, 0, v); }
get y() { return __rd.f32(this, 4); }
set y(v) { __wr.f32(this, 4, v); }
}
vec2.size = 8;
vec2.align = 4;
vec2.fields = {x:{type:'f32',offset:0},y:{type:'f32',offset:4}};
/* struct shape, 32 bytes (line 12) */
export class shape extends ArrayBuffer {
constructor(init = 32) { ... }
static at(p, owner, size = 32) { return __view(shape, p, size, owner); }
get ptr() { return __ptr(this); }
get id() { return __rd.i32(this, 0); }
set id(v) { __wr.i32(this, 0, v); }
get name() { return __ptrOut(__rd.u64(this, 8)); }
set name(v) { __wr.u64(this, 8, __ptrIn(v)); }
get origin() { return vec2.at(__ptr(this, 16), this); }
set origin(v) { new Uint8Array(this, 16, 8).set(__bytes(v)); }
get visible() { return (__rd.u32(this, 24) >>> 0) & 0x1; }
set visible(v) { __wr.u32(this, 24, ((__rd.u32(this, 24) & ~0x1) | ((v << 0) & 0x1)) >>> 0); }
}
shape.size = 32;
shape.align = 8;
shape.fields = {id:{type:'i32',offset:0},name:{type:'cstring',offset:8},origin:{type:'vec2',offset:16},visible:{type:'u32',bits:1,bitOffset:192}};- A member is an accessor at the byte offset of the IR: it reads with
readand writes withwrite. A nested struct or an array member is assigned through a typed array instead, whoseset()throws aRangeErrorfor a source that is too large, wherewrite.byteswould write past the member. 64-bit integers arebigint. new shape()allocates zeroed memory of the struct's size; `new shape(bytes)copies anArrayBufferor view.shape.at(ptr)` makes a view of native memory (not copied, not owned), ands.ptris the address.- A pointer member gives
null, anumberor abigintand takes those, a class instance or a buffer. A pointer to a struct of a known class gives a live instance of that class (nullfor NULL).const char *is a plain pointer here: decode it withCString(s.name). - A bitfield is read and written through its storage unit,
bitOffsetbits up. - A nested struct is a view of the same memory (
s.originabove), an array member a typed-array view, an array of pointers aProxyover aBigUint64Array, and anything else aUint8Arrayover its bytes. Name.size,Name.alignandName.fieldsdescribe the layout.- A member with an unknown offset is left out. Members that would hide
something an
ArrayBufferhas (byteLength,slice, ...) get an underscore.
A function that returns a struct by value then returns an instance of its class
(__ret(vec2, CFunction(...))), and a class can be passed for such an
argument. An extern variable is wrapped in an accessor with the address and,
for a scalar or pointer, the value:
export const geom_count = __variable('geom_count','i32');Used on top of the library of the example:
const a = new geom.vec2();
a.x = 3; a.y = 4;
console.log(geom.geom_length(a)); // 5
const sum = geom.geom_add(a, a);
console.log(sum instanceof geom.vec2, sum.x, sum.y); // true 6 8
const s = geom.shape.at(geom.geom_find('circle'));
console.log(s.id, CString(s.name), s.origin.x, s.visible); // 1 circle 0 1
console.log(geom.geom_move(s, 2, 3), s.origin.x, s.origin.y); // 0 2 3
console.log(geom.geom_count.value); // 2 (geom_count.ptr is its address)
console.log(geom.GEOM_BAD_ARG, geom.shape.size); // -1 32C++ classes
A C++ source (--c++, or a .cpp/.hpp name) also gets classes, with or
without --structs. For shapes.hpp the generator writes, after the shared
support code, a lazily bound overload list per method and the class:
const __Shape_m_area = [
{n:0,t:[],f:__lazy(()=>__virtual(2,{args:['geo::Shape *'],returns:'f64'}))},
];
const __Shape_m_scale = [
{n:0,t:[],f:__lazy(()=>CFunction({ptr:__sym('_ZNK3geo5Shape5scaleEv'),args:['geo::Shape *'],returns:'f64'}))},
];
const __Shape_m_setScale = [
{n:1,t:['f64'],f:__lazy(()=>CFunction({ptr:__sym('_ZN3geo5Shape8setScaleEd'),args:['geo::Shape *','f64'],returns:'void'}))},
];
const __Shape_s_count = [
{n:0,t:[],f:__lazy(()=>CFunction({ptr:__sym('_ZN3geo5Shape5countEv'),args:[],returns:'i32'}))},
];
/* class geo::Shape, 24 bytes (line 5) */
export class Shape extends __CxxObject {
get id() { return __rd.i32(this, 8); }
set id(v) { __wr.i32(this, 8, v); }
area(...args) { return __invoke(__Shape_m_area, this, args); }
scale(...args) { return __invoke(__Shape_m_scale, this, args); }
setScale(...args) { return __invoke(__Shape_m_setScale, this, args); }
static count(...args) { return __invoke(__Shape_s_count, undefined, args); }
}
Shape.__info = { size: 24, ctors: null, dtor: __lazy(()=>__virtual(0,{args:['geo::Shape *'],returns:'void'})) };
const __Circle_ctor = [
{n:1,t:['f64'],f:__lazy(()=>CFunction({ptr:__sym('_ZN3geo6CircleC1Ed'),args:['geo::Circle *','f64'],returns:'void'}))},
];
const __Circle_m_area = [
{n:0,t:[],f:__lazy(()=>__virtual(2,{args:['geo::Circle *'],returns:'f64'}))},
];
/* class geo::Circle, 32 bytes (line 18) */
export class Circle extends Shape {
get radius() { return __rd.f64(this, 24); }
set radius(v) { __wr.f64(this, 24, v); }
area(...args) { return __invoke(__Circle_m_area, this, args); }
}
Circle.__info = { size: 32, ctors: __Circle_ctor, dtor: __lazy(()=>__virtual(0,{args:['geo::Circle *'],returns:'void'})) };
// C++ functions
const __fn_lerp = [
{n:3,t:['f64','f64','f64'],f:__lazy(()=>CFunction({ptr:__sym('_ZN3geo4lerpEddd'),args:['f64','f64','f64'],returns:'f64'}))},
];
export const lerp = (...args) => __invoke(__fn_lerp, undefined, args);- Objects: a class extends its first base class when that is bound, public,
non-virtual and sits at offset 0 (a polymorphic class has its vptr there, a
non-polymorphic base does not), else the common root
__CxxObject, itself anArrayBuffer. `new Circle(2)allocatessizebytes and runs the complete constructor (C1`);c.delete()runs the destructor, after which any call on the object throws.Shape.at(ptr)(alsoShape.from(ptr)) wraps an existing object without owning it, andc.ptris its address. A class that is abstract or has no constructor symbol cannot be constructed: `TypeError: Shape cannot be constructed`. - Calls: a method takes the object as its first argument (
this), a static method and a free function none. Everything is bound on first call, since an inline member often has no exported symbol. Overloads are told apart by the number of arguments and then by their types. - Names: the symbol is the mangled name the IR carries. A class keeps its
qualified name (
geo_Shape), orShapewith--namespace=geo. - Virtual methods are not called through their symbol but through the
object's vtable,
__virtual(slot, spec): the slot is the number from the layout probe, read from the vtable that the object's first word points to. So an object wrapped as the base class runs the override of its real class, a pure virtual method is callable, anddelete()through a base runs the derived destructor. - Fields are accessors, like those of a struct; static fields are class properties.
import { Circle, Shape, lerp } from './shapes.js';
const c = new Circle(2);
console.log(c.radius, c.id, c.scale()); // 2 1 1
c.setScale(1.5);
console.log(c.area()); // 18 (Circle::area: 3 * 2 * 2 * 1.5)
console.log(Shape.count(), lerp(0, 10, 0.25)); // 1 2.5
const base = Shape.at(c.ptr); // the same object, wrapped as the base class
console.log(base.area()); // 18 still Circle::area, through the vtable
c.radius = 3;
console.log(base.area()); // 40.5
try { new Shape(); } catch(e) { console.log(e.message); } // Shape cannot be constructed
c.delete();
try { c.area(); } catch(e) { console.log(e.message); } // object was deleted--finalize also destroys an object made with new when it is garbage
collected. The object then lives in memory from calloc, which the wrapper
only views, and a FinalizationRegistry runs the destructor (through the
vtable for a virtual one) and frees it. delete() still destroys at once. The
callback runs on a later turn of the event loop, not inside the collection, and
an object still alive at exit is not destroyed.
Not covered: multiple and virtual inheritance (the this adjustment needs base
offsets), default arguments, templates other than explicit instantiations,
exceptions across the boundary, operators, and by-value class parameters and
returns. See TODO.md and the
design note on virtual dispatch.
Wrappers for C conventions
A C signature does not say that unsigned char * is text, that a sqlite3 **
is where the new handle goes, or that the string a function returns must be
freed. Four options say it, by function name; the generator then wraps those
functions (qjs target). They apply to the named functions only, nothing is
guessed, unless --out-auto is given.
| Option | Effect |
|---|---|
--cstring=sqlite3_column_text | the return is a string, as for char * |
--handle=sqlite3:sqlite3_close | a sqlite3 * returned, or stored in a sqlite3 ** out-parameter, is a closable object; one --handle per type |
--out=sqlite3_open:ppDb | ppDb leaves the parameter list; the wrapper makes the cell and returns what C stored in it. Repeat for several: the result is then an array in parameter order |
--owned-return=sqlite3_mprintf:sqlite3_free | the returned char * is copied to a string, then passed to the destructor; without :dtor libc free |
qjs-ffi-genbindings --library=libsqlite3.so.0 --cstring=sqlite3_column_text \
--handle=sqlite3:sqlite3_close --handle=sqlite3_stmt:sqlite3_finalize \
--out=sqlite3_open:ppDb --out=sqlite3_prepare_v2:ppStmt --out=sqlite3_prepare_v2:pzTail \
/usr/include/sqlite3.h -o sqlite3.jsconst db = sqlite3_open(':memory:'); // a handle, not a number
const [stmt, tail] = sqlite3_prepare_v2(db, 'select 1', -1);
while(sqlite3_step(stmt) === SQLITE_ROW) console.log(sqlite3_column_text(stmt, 0));
sqlite3_finalize(stmt); // closes the object
sqlite3_close(db);- Status returns. A function with
--outand an integer return treats0as success; any other value throws anErrorwithcodeset to it, after closing the handles the call made (sqlite3_openhands back a handle even when it fails). Avoidreturn gives the out values only. Any other return type is refused. - Out cells. A
T *out-parameter is read as the scalarT(int *gives a number,double *a number,long long *a BigInt);T **gives a pointer number ornull, achar **a string (copied, not freed: use a destructor yourself if C allocated it), a handle type its object. - Handles. The object has
ptr,closed,valueOf()(the pointer) andclose(), which runs the destructor once and returns its result. Passing it where the C function takes thetype *unwraps it; a number is still accepted. Calling the destructor function on it (sqlite3_close(db)) closes it. A handle nobody closed is closed by the garbage collector, on a later turn of the event loop and in no particular order: a destructor that needs the others first (sqlite3_closefails withSQLITE_BUSYwhile statements are open) must be closed by hand. - Owned returns. The memory is freed even if copying throws. The destructor may take the string type: it is called with the raw pointer.
- Errors. An unknown function, a destructor that is not bound, an
--outparameter that is not a pointer or points to a struct, or a name that has no parameter, stop the run with nothing written.--describeand--jsdocdescribe the wrapper's parameters, without the out-parameters. --out-autosaves listing the out-parameters: aT **of a--handletype, and a pointer to a scalar or pointer namedout…orpp<Name>(ppDb,outValue), count as--out. A buffer namedout(uint8_t *out) is taken for an out-parameter too, so name those functions in--excludeor leave the option off. A name likelo/hiis not guessed.- C++ functions and methods take
--out,--out-autoand--owned-returnunder their qualified names:--out=ns::fn:arg,--out=ns::Class::method:arg. The wrapper keepsthis; an overload without the named parameter is left alone, and the overloads are told apart by what is left after the out-parameters go (count(int)andcount(int, int, int *out)becomecount(a)andcount(a, b); two with the same remaining arguments are not told apart).--handledoes not reach C++ entries, and a destructor named for--owned-returnmust be a C function (extern "C"). - Not with
--target=bun|deno|node;--cstringworks everywhere. tests/test-gen-bindings-wrappers.jsruns all four againsttests/cxx/wrappers.{h,c}.
Output options
--ffitype writes the types as members of FFIType:
export const geom_abs = __bind('geom_abs', () => CFunction({ptr:__sym('geom_abs'),args:[FFIType.i32],returns:FFIType.i32}));--describe gives every function, method and constructor its parameter names
in its signature and attaches its C types and overloads as
fn[Symbol.for('describe')], which describeObject() and describeClass()
(qjs-modules) report. A plain function becomes a named wrapper:
const __f_geom_abs = __bind('geom_abs', () => CFunction({ptr:__sym('geom_abs'),args:['i32'],returns:'i32'}));
export function geom_abs(v) { return __f_geom_abs(v); }
__sig(geom_abs, [{params:['v: i32'],returnType:'i32',arity:1}]);--jsdoc puts a JSDoc block before every function, method and class: a
@param per parameter with its JavaScript type and C type, a @returns, and an
@overload group per overload. A pointer is object|number|bigint|null, plus
the class when it points to a bound one.
/**
* geom_move
* @param {object|number|bigint|null} s - shape *
* @param {number} dx - f64
* @param {number} dy - f64
* @returns {number} - i32
*/
export const geom_move = __bind('geom_move', () => CFunction({ptr:__sym('geom_move'),args:['shape *','f64','f64'],returns:'i32'}));The IR on its own
The two halves of the generator are separate runs. The IR is JSON: methods and fields for the
functions and variables, enums, structs, classes, typedefs, skipped,
byValue, and source, which records how it was made (files, includes,
defines, language) for the header of the output. version is the IR format
(2); --from-ir refuses an IR of another version and says to make it again
with --emit-ir.
qjs-ffi-genbindings --emit-ir=api.json geom.h # clang -> IR
qjs-ffi-genbindings --from-ir=api.json -o geom.js # IR -> JavaScript, no clang
qjs-ffi-genbindings --from-ir=api.json --library=libgeom.so \
--emit-specs=geom.specs.json # IR -> dlopen() specs--from-ir reads the IR as JSON, or as the JS module --emit-ir --js writes
(export default { ... };).
--from-ir takes no sources and takes the options of the emitting phase
(--structs, --describe, --target, --namespace, ...) as before. The IR can
be edited or checked in between, and a binding can be regenerated on a machine
without clang.
gen-structs.js makes just the struct, union and class wrappers from an IR:
qjs-ffi-genstructs [--struct=<name>]... [--describe] [--jsdoc] [--format=js|c] [-o out] api.json...--format=js is the same class code --structs emits. --format=c writes a C
header that defines the structs (fields only) and checks the layout the IR
claims with _Static_asserts, so that a layout that does not match the real
compiler fails to compile:
struct vec2 {
float x;
float y;
};
struct shape {
int id;
const char *name;
vec2 origin;
unsigned int visible : 1;
};
_Static_assert(sizeof(struct vec2) == 8, "vec2 size");
_Static_assert(offsetof(struct vec2, x) == 0, "vec2.x offset");
_Static_assert(offsetof(struct vec2, y) == 4, "vec2.y offset");
_Static_assert(sizeof(struct shape) == 32, "shape size");
_Static_assert(offsetof(struct shape, id) == 0, "shape.id offset");
...describe.sh [-r <runtime>] [--json] [--js] [--dts] [--flat] [--class] <module> [export...]
lists what a module, an object or a class has: the exports of a generated
module or one from lib/ with the C types when it was generated with
--describe, but also ffi.so itself, node:fs, or bun:ffi. It runs
describe.js in QuickJS (qjsm, the default, or qjs), Node.js, Bun or
Deno, so the same API can be dumped in two runtimes and compared:
describe.sh -r qjsm build/ffi.so # what ffi exports
describe.sh -r bun bun:ffi # what bun:ffi exports
describe.sh -r bun --global Bun.FFI # an object by its path
describe.sh -r node --class --global Buffer # a native constructor as a class
describe.sh -r node node:fs readFile promises.readFileWith nothing but a module name it does not dump anything: it asks each installed
runtime whether it can load the module and prints the command that dumps it for
every one that can (./tools/describe.sh -r node node:fs, with the script
named the way it was started), the others noted on stderr. A module is a path or a specifier the runtime imports; an export may be a dotted
path; --global takes properties of globalThis instead of a module; --json
prints the raw results; --js prints a skeleton of the module as JavaScript
(export function f(a, b) {}, classes with empty method bodies); --dts prints
TypeScript declarations, with the C types of a binding generated with
--describe as number, bigint, string and Pointer; --flat prints one
sorted line per export and member, so that two runtimes' dumps diff line by
line. -r browser runs it in a headless Chrome or Firefox (DESCRIBE_BROWSER
names the executable; tools/describe-browser.js drives it over the DevTools
protocol or WebDriver BiDi), for --global navigator or a module URL. Importing a module loads its libraries, so
QUICKJS_MODULE_PATH must find the ffi module.
Specs from the IR
--emit-specs=<file> turns the IR into the symbol specs that dlopen() takes
(Libraries and symbols), so a script can bind a library without a
generated .js file. It works from sources or from an IR file with
--from-ir, which needs no clang:
qjs-ffi-genbindings --library=/usr/lib/libgeom.so --emit-specs=geom.specs.json geom.h
qjs-ffi-genbindings --from-ir=geom.ir.json --library=/usr/lib/libgeom.so \
--emit-specs=geom.specs.jsonconst { library, symbols } = JSON.parse(std.loadFile('geom.specs.json'));
const lib = dlopen(library, symbols); // lib.symbols.geom_abs(-3), lib.symbols.counterWithout a file name, --emit-specs writes to stdout, as a JS module unless
--json is given, so it pipes straight into a file or another tool:
qjs-ffi-genbindings --library=/usr/lib/libgeom.so --emit-specs geom.h > geom.specs.js
qjs-ffi-genbindings --library=/usr/lib/libgeom.so --emit-specs --json geom.h | jq .symbolsWith --js the file is an ES module with the same data, written by inspect()
of qjs-modules, so a script can import it (--emit-ir takes --js too):
qjs-ffi-genbindings --library=/usr/lib/libgeom.so --emit-specs=geom.specs.js --js geom.himport specs from './geom.specs.js';
const lib = dlopen(specs.library, specs.symbols);The file is { library, symbols, constants, types, omitted }:
| Key | Holds |
|---|---|
library | the --library path, else null |
symbols | { name: spec }: { args, returns } for a function, { type, readonly? } for a variable |
constants | the values of constants that have no symbol (static const int N = 5), to be JS constants |
types | the names of the structs and unions that have a size: pass the classes of those names as dlopen(specs.library, specs.symbols, { pt }) and a "struct pt *" in symbols is a pointer typed by the class pt (naming a class) |
omitted | { name, reason } for everything that has no spec |
From the IR entry to the spec:
| IR | Spec |
|---|---|
args: ["dx: f64"] | args: ["f64"], the names dropped |
"struct vec3" | the member types from byValue: ["f32", "f32", "f32"] |
type: "int[3][2]" | nested array types: { array: { array: "i32", length: 2 }, length: 3 } |
const: true | readonly: true |
value on a constant | not a symbol: goes to constants |
Left out, with the reason in omitted:
- what the IR already
skipped(unions and unplaceable structs by value); - C++ functions (the symbol is mangled) and classes (
this, virtual dispatch and constructors are not part of a spec: use the generated classes); - an array of more than 2^20 elements or of unknown size (
int tail[]); - a struct that is not in
byValue, which would otherwise reach theffimodule as an unknown type name, which it rejects with aTypeError.
Enums, typedefs and structs are not part of the specs.
Using the result
- Run the module with
qjsm, withffionQUICKJS_MODULE_PATH, and import it:import * as cairo from './lib/bindings/cairo.js'. - Ready-made bindings are in
lib/bindings/(cairo, SDL2, freetype, libcurl, libusb, libarchive, zlib);examples/andtests/test-cairo.jsuse them. They are output of an earlier version of the generator, so regenerate them with a currentffimodule before depending on newer features. - Installed, the tools live in
share/qjs-ffi/toolsandbin/hasqjs-ffi-genbindingsandqjs-ffi-genstructswrappers:qjsmresolves a script's imports against the path it is run as, not a symlink's target. - A generated module imports
read, so it needs anffiwithreadand the pointer form oftoArrayBuffer.
Limitations
- Operators and arrays by value are skipped, as are unions and C++ classes by value that are not trivially copyable.
- A struct by value needs a layout libffi can reproduce: not a packed struct.
- Preprocessor macros are not in the AST. An object-like
#definewhose value is a number, a string, a character or integer arithmetic over other such defines (#define B (A << 2)) is bound as a constant, scanned from the header text. Function-like macros, casts,sizeofand anything else are not bound. - The tests:
tests/test-gen-bindings-cxx.jschecks every IRmangledNameagainstnm -Dof a compiled fixture and runs the module,test-gen-bindings-virtual.jsand-finalize.jsthe vtable and finalizer paths,test-gen-structs.jsandtest-struct-by-value.jsthe structs,test-gen-bindings-wrappers.jsthe wrappers,test-gen-bindings-names.jsthe class/function name clash, missing symbols (also under bun and deno) and unions by value (tests/cxx/anonunion.{h,c}),test-gen-bindings-defaults.jsdefault arguments (tests/cxx/defaults.*),test-gen-bindings-cxxvalue.jsclasses and template instances by value (tests/cxx/cxxvalue.*),test-gen-bindings-cxxwrap.jsthe wrappers of C++ functions and methods (tests/cxx/cxxwrap.*).
Bun
--target=bun writes a module for bun's bun:ffi instead of this one's ffi:
qjs-ffi-genbindings --target=bun --library=./libgeom.so --structs -o geom.mjs geom.h
bun run-it.mjs # import * as geom from "./geom.mjs"The functions, enums, constants, --structs classes, extern variables and
C++ classes of the default output are the same code; what differs:
- it needs
--library: bun:ffi has noRTLD_DEFAULT, so a symbol's address is the.ptrof a one-symboldlopen().--finalizeand--class-typesare refused (they needcalloc/freelookup and a constructor as a type). - every typed pointer (
"struct pt *") is"pointer"; a function returning a pointer to a generated class gives an instance of that class (Class.at()), and an instance is passed back as a pointer argument as it is. - a struct passed or returned by value, and a variadic function, are not in bun:ffi: they are skipped, with the reason in the comment at the end.
readandwriteof the module are aDataViewover the buffer (or overtoArrayBuffer()of an address), since bun has nowrite().
tests/test-gen-bindings-bun.js runs the same probes through this target
under bun and through the default one under qjsm and compares the output.
Deno
--target=deno writes a module for Deno; it needs --allow-ffi (and
--allow-read for the library):
qjs-ffi-genbindings --target=deno --library=./libgeom.so --structs -o geom.mjs geom.h
deno run --allow-ffi --allow-read run-it.mjsIt is the bun output on top of a small prelude at the top of the module that
supplies CFunction, toArrayBuffer, ptr and CString through
Deno.UnsafeFnPointer and Deno.UnsafePointerView. The same rules as
Bun apply (--library needed, --finalize and --class-types
refused, typed pointers are "pointer" and come back as Numbers), except:
- a symbol's address is the value of a
{ type: "pointer" }static of a one-symbolDeno.dlopen(), one per function, so loading a module costs onedlopen()per function it calls, not one in all. - a struct passed or returned by value works (
{ struct: [...] }, the bytes become the class again); only variadic functions are skipped. - a
"cstring"argument is a string ornull, encoded with a NUL.
tests/test-gen-bindings-deno.js compares the probes with the default output
under qjsm; it skips the Deno half when deno is not installed.
Node
--target=node writes a module for Node 26's experimental node:ffi (no flag
needed, it prints an ExperimentalWarning; --disable-warning=ExperimentalWarning
silences it):
qjs-ffi-genbindings --target=node --library=./libgeom.so --structs -o geom.mjs geom.h
node run-it.mjsAgain the bun output on top of a prelude (CFunction, toArrayBuffer, ptr,
CString, here over DynamicLibrary.getFunction()), with the rules of
Bun (--library needed, --finalize and --class-types refused,
typed pointers are "pointer" and come back as Numbers) and these differences:
node:fficalls only a function it looked up by name, so the module remembers the address of every symbol it resolves with its name. A virtual method is called through an address read from a vtable: the first such call resolves all the C++ symbols the module binds and looks the address up among them. An address that is none of them (an inline or pure virtual method has no symbol) throws aTypeError.- a struct passed or returned by value and a variadic function are not in
node:ffi: skipped, with the reason in the comment at the end. - a
"cstring"argument is a string ornull, encoded with a NUL.
tests/test-gen-bindings-node.js runs the probes under node and compares
them with the default output under qjsm; it looks for a Node with node:ffi
on PATH, then under ~/.nvm/versions/node, and skips the node half if there
is none.