Building a Register VM Interpreter - Ch. 11: Functions and Calls
This guide is about building a register-based VM interpreter from scratch. To evolve from an advanced calculator into a programming language, we must introduce reusable, isolated blocks of code. We need Functions.
This language suffers a from a massive structural limitation: it is a single, monolithic script. Right now, the compiler reads the entire source file and emits every single instruction into on global Chunk. If a user writes a usefule piece of math or logic and wants to use it multiple times, they cannot. To evolve from an advanced calculator into a programming language, we must introduce reusable, isolated blocks of code. We need Functions.
But in a dynamically typed scripting language, a function is not just a static memory address to jump to (like it is in C). We are building a language with First-Class Functions. This means a function is just another piece of data. It can be assigned to a variable, passed as an argument to another function, returned from a function, and stored in a data structure.
To achieve this, the Virtual Machine needs a way to package a function’s bytecode, its metadata, and its identity into a single, heap-allocated entity.
The Function Object
We have created the Object base struct, which allowed us to allocate complex data like strings and hash tables on the heap and pass the pointers around inside the VM. Because the function is the first-class citizens, they will utilize this exact same heap-allocation strategy.
In core/object.h, we define the ObjectFunc structure:
1
2
3
4
5
6
7
8
9
10
11
typedef struct ObjectFunc{
Object obj;
int arity;
// int upvalueCnt;
Chunk chunk;
ObjectString* name;
// ObjectString* srcName;
// FuncType type;
// struct ObjectClass* fieldOwner;
int maxRegSlots;
}ObjectFunc;
Let’s just understand this five basic fields. The integer arity tracks the number of parameters the function expects to receive. As we discussed in previous parts, this compiler acts as a traffic controller, keeping track of the highest register index it ever uses via compiler->maxRegSlots. The most important architectural shift lays on the Chunk chunk maintained by ObjectFunc. Up until new, the VM relied on a single global Chunk to hold bytecode and constants. Now instead, every single function owns its own isolated chunk.
By encapsulating the bytecode and metadata inside an ObjectFunc, the compiler is no longer just generating a sequence of operations; it is generating a self-contained executable package.
Compiling the Function
To support functions, we should completely overhaul how the compiler emits bytecode. As we established with the ObjectFunc struct, every function must own its own isolated bytecode array.
This intoduces a compilation paradox: What happens when the user defines a function inside another block of code?
1
2
3
4
5
func add(a, b) {
return a + b;
}
print add(1, 2);
The instructions inside add should not be emitted into the main script chunk. If they were, the VM would execute the function body immediately when the file is loaded. That is not what a function declaration means. A function declaration should create a function value. The function body should only run later, when the function is called.
So the compiler needs to temporarily stop writing to the current chunk, create a new function object, redirect bytecode emission into that function’s chunk, compile the body, then return to the outer compiler and emit an instruction that creates the function value at runtime.
In this interpreter, that is done by creating a nested Compiler.
The Nested Compiler
In compiler/compiler.h, we update the Compiler struct to include a pointer to an enclosing compiler, as well as a pointer to the ObjectFunc it is actively building:
1
2
3
4
5
6
7
typedef struct Compiler{
struct Compiler* enclosing;
// Enclosing compiler for nested functions
// ...
FuncType type;
ObjectFunc* func;
}Compiler;
FuncType is defined in core/object.h:
1
2
3
4
5
6
7
typedef enum{
TYPE_FUNC,
TYPE_SCRIPT,
TYPE_MODULE,
TYPE_METHOD,
TYPE_INITIALIZER,
}FuncType;
When the VM boots up, it initializes a root Compiler (of TYPE_SCRIPT) to wrap the top-level code. When the parser encounters a func keyword, it does not allocate a new compiler on the heap. Instead, it simply declares a new Compiler struct as a local C variable, linking it back to the parent:
1
2
3
4
5
static void initCompiler(Compiler* compiler, VM* vm, Compiler* enclosing, FuncType type, ObjectString* srcName){
// ...
compiler->func = newFunction(vm);
// ...
}
Because all the helper functions like emitABC take a Compiler* as the first argument, simply passing the &compiler pointer instantly redirects all bytecode generation, register allocation, and scope tracking to the new function’s environment.
Parameters
Once the nested compiler is initialized, it must parse the function’s parameters.
In a dynamically typed language, parameters are fundamentally just local variables that happen to be pre-filled with data by the caller. Because of how we built lexical scoping and name erasure, the compiler can treat parameters exactly like standard locals.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
static void compileFunc(Compiler* compiler, FuncType type, int destReg, Token* funcName){
Compiler* funcCompiler = (Compiler*)reallocate(compiler->vm, NULL, 0, sizeof(Compiler));
if(funcCompiler == NULL){
errorAt(compiler, &compiler->parser.pre, "Not enough memory to compile function.");
return;
}
funcCompiler->parser = compiler->parser;
funcCompiler->enclosing = compiler;
funcCompiler->vm = compiler->vm;
funcCompiler->func = NULL;
compiler->vm->compiler = funcCompiler;
initCompiler(funcCompiler, compiler->vm, compiler, type, compiler->func->srcName);
if(funcName != NULL){
funcCompiler->func->name = copyString(
compiler->vm,
funcName->head,
funcName->len
);
}
beginScope(funcCompiler);
consume(funcCompiler, TOKEN_LEFT_PAREN, "Expect '(' after function name.");
if(!checkType(funcCompiler, TOKEN_RIGHT_PAREN)){
do{
funcCompiler->func->arity++;
if(funcCompiler->func->arity > 255){
errorAt(funcCompiler, &funcCompiler->parser.cur, "Too many function args.");
}
int constant = parseVar(funcCompiler, "Expect param name.");
defineVar(funcCompiler, constant);
}while(match(funcCompiler, TOKEN_COMMA));
}
consume(funcCompiler, TOKEN_RIGHT_PAREN, "Expect ')' after parameters.");
// compile the body...
}
Let’s look closely at what happens inside the do-while loop. For every parameter, we increment the arity count and call defineVar.
Function Body
This is how we complete the compileFunc function:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// ...
consume(funcCompiler, TOKEN_RIGHT_PAREN, "Expect ')' after parameters.");
consume(funcCompiler, TOKEN_LEFT_BRACE, "Expect '{' before function body.");
block(funcCompiler);
ObjectFunc* func = stopCompiler(funcCompiler);
int constIndex = makeConstant(compiler, OBJECT_VAL(func));
emitClosure(compiler, destReg, constIndex, funcCompiler);
compiler->parser = funcCompiler->parser;
compiler->vm->compiler = compiler;
reallocate(compiler->vm, funcCompiler, sizeof(Compiler), 0);
}
Calling the block parsing function hands control back to the statement parser:
1
2
3
4
5
6
static void block(Compiler* compiler){
while(!checkType(compiler, TOKEN_RIGHT_BRACE) && !checkType(compiler, TOKEN_EOF)){
decl(compiler);
}
consume(compiler, TOKEN_RIGHT_BRACE, "Expect '}' after block.");
}
The important detail is that the function passes funcCompiler is not the outer compiler. That one pointer decides where the entire function body goes. Every declaration, statement, expression, loop, and return inside the body emits bytecode into funcCompiler->func->chunk. The outer compiler is paused. Its chunk receives nothing while the body is being compiled.
This gives the true isolation. The function body is not inline code. It is stored inside the ObjectFunc.
After the body is compiled, compileFunc() receives the finished function object:
1
ObjectFunc* func = stopCompiler(funcCompiler);
At this point, the function’s bytecode is complete, but the outer code still needs a way to refer to it.
A function is a runtime value, so the outer chunk stores the function object as a constant:
1
int constIndex = makeConstant(compiler, OBJECT_VAL(func));
Finally, the parser state is restored and the compiler is freed:
1
2
3
4
compiler->parser = funcCompiler->parser;
compiler->vm->compiler = compiler;
reallocate(compiler->vm, funcCompiler, sizeof(Compiler), 0);
}
Function Declarations and Expressions
Now that we have a general compileFunc, the next question is where does the resulting function value go.
A function body is compiled into an ObjectFunc, we can have two different ways to create functions in a dynamically typed language.
The first one is function declaration:
1
2
3
func add(a, b) {
return a + b;
}
The second one is function expression:
1
2
3
var add = func(a, b) {
return a + b;
};
They look similar, but they appear in different grammatical positions.
A declaration introduce a name directly into the current scope. An expression produces a value, and that value can be assigned, passed, or even returned.
This difference is important because our language treats functions as first-class values. A function is not only something that can be declared by name. It is also an expression that can produce a runtime value.
So we need two entry points in compiler/compiler.h:
1
2
static void funcDecl(Compiler* compiler);
static void funcExpr(Compiler* compiler, ExprDesc* expr, bool canAssign);
Both of them eventually call the same compileFunc() helper. The difference is not how the function body is compiled. The difference is where the resulting function value is stored.
Function Expressions
A function expression is handled by funcExpr:
1
2
3
4
5
6
static void funcExpr(Compiler* compiler, ExprDesc* expr, bool canAssign){
int destReg = getFreeReg(compiler);
reserveReg(compiler, 1);
compileFunc(compiler, TYPE_FUNC, destReg, NULL);
initExpr(expr, EXPR_REG, destReg);
}
Since an expression must produce a value, the compiler first reserves a register. Then it compiles the function and place the resulting callable value into the register. The last argument is NULL because this function expression does not necessarily have a declaration name.
For example:
1
2
3
var add = func(a, b) {
    return a + b;
};
The func(a, b) { ... } part is an expression. It produces a function value. That value is stored in a register, and then the variable declaration stores that register into add.
After compileFunc() finishes, the compiler marks the expression result as a register expression:
1
initExpr(expr, EXPR_REG, destReg);
This means the rest of the expression compiler can treat the function exactly like any other value. The compiler does not need a special rule that calls the function named add. It only need to compile add as an expression, load the function value from the variable, and then emit a call instruction.
Function Declarations
Function declarations are handled by funcDecl:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
static void funcDecl(Compiler* compiler){
int global = parseVar(compiler, "Expect function name.");
Token funcName = compiler->parser.pre;
if(compiler->scopeDepth > 0){
int reg = compiler->localCnt - 1;
compileFunc(compiler, TYPE_FUNC, reg, &funcName);
defineVar(compiler, global);
}else{
int tmpReg = getFreeReg(compiler);
compileFunc(compiler, TYPE_FUNC, tmpReg, &funcName);
emitABx(compiler, OP_SET_GLOBAL, tmpReg, global);
defineVar(compiler, global);
}
}
The compiler checks whether the declaration is inside a local scope, this gives us two cases:
-
Local Function Declarations
-
Global Function Declarations
If the function is declared inside a block or another function, then it is local variable:
1
2
3
4
5
if(compiler->scopeDepth > 0){
int reg = compiler->localCnt - 1;
compileFunc(compiler, TYPE_FUNC, reg, &funcName);
defineVar(compiler, global);
}
This works because parseVar() has already declared the function name as a local. The newest local is located at compiler->localCnt - 1.
The function declaration creates a function value and stores it in the local slot named add.
The important point is that local function declarations reuse the local variable system from the previous article. There is no separate runtime namespace for local functions. A local function is simply a local variable whose value happens to be callable.
This fits naturally with the register model.
If add is assigned to register R3, then later uses of add simply resolve to R3.
The VM does not know that R3 came from a function declaration. It only sees a value in a register.
If the function is declared at the top level, the compiler uses the global table:
1
2
3
4
5
6
else{
int tmpReg = getFreeReg(compiler);
compileFunc(compiler, TYPE_FUNC, tmpReg, &funcName);
emitABx(compiler, OP_SET_GLOBAL, tmpReg, global);
defineVar(compiler, global);
}
The function value is first created in a temporary register. The compiler emits the function value into the global variable table under the function name. Again, the function declaration does not mean executing now, it means creating a function value and binding it to this name.
The reason both funcDecl() and funcExpr() call compileFunc() is that the hard part is the same in both cases. The only difference is the destination.
For a function expression, the destination is a temporary expression register. For a local function declaration, the destination is the local variable register. For a global function declaration, the destination is a temporary register, followed by a global assignment.
The syntax is different, but the compiled object is the same. A function declaration is not a special runtime creature. It is just a convenient syntax for creating a function value and storing it under a name.
Compiling a Call Expression
At this point, a function can be compiled and stored as a value. But a function value is only useful if the language can call it. The VM does not want to receive a scattered list of argument registers. It wants the callee and its arguments to sit next to each other in the register array. Then the call instruction only needs to know where the callee starts and how many slots belong to this call.
Parse
A call is parsed as a postfix expression. In the parse table, the left parenthesis has two meanings:
1
[TOKEN_LEFT_PAREN] = {handleGrouping, handleCall, PREC_CALL},
But when ( appears after an existing expression, it means a function call, so the infix parser is handleCall.
The compiler does not need a special rule for “calling a named function.” It only needs a rule for “calling the value produced by the expression on the left.”
The call parser is short:
1
2
3
4
5
6
7
static void handleCall(Compiler* compiler, ExprDesc* expr, bool canAssign){
int argCount = argList(compiler, expr);
emitABC(compiler, OP_CALL, expr->data.loc.index, argCount + 1, 2);
// +1 for the function itself
freeRegs(compiler, argCount);
initExpr(expr, EXPR_REG, expr->data.loc.index);
}
The real work is inside the argList:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
static int argList(Compiler* compiler, ExprDesc* func){
int argCnt = 0;
expr2NextReg(compiler, func);
int funcReg = func->data.loc.index;
if(funcReg < compiler->freeReg - 1){
reserveReg(compiler, 1);
int newReg = compiler->freeReg - 1;
emitABC(compiler, OP_MOVE, newReg, funcReg, 0);
funcReg = newReg;
func->data.loc.index = funcReg;
}
if(!match(compiler, TOKEN_RIGHT_PAREN)){
do{
ExprDesc arg;
expression(compiler, &arg);
int targetReg = funcReg + argCnt + 1;
// function is at funcReg, arguments start from funcReg + 1
expr2Reg(compiler, &arg, targetReg);
freeExpr(compiler, &arg);
reserveReg(compiler, 1);
argCnt++;
if(argCnt >= 255){
errorAt(compiler, &compiler->parser.pre, "Cannot have more than 255 arguments.");
}
}while(match(compiler, TOKEN_COMMA));
consume(compiler, TOKEN_RIGHT_PAREN, "Expect ')' after arguments.");
}
return argCnt;
}
The call target may originally be many different kinds of expression. The VM, however, does not care where the function value came from. It only wants the final callable value to be sitting in a register. That is what this line guarantees:
1
expr2NextReg(compiler, func);
It forces the callee expression into the next available register.
After that, the compiler records the callee register:
1
int funcReg = func->data.loc.index;
From this point on, the entire argument list will be arranged relative to funcReg.
After forcing the callee into a register, argList performs an extra check:
1
2
3
4
5
6
7
if(funcReg < compiler->freeReg - 1){
reserveReg(compiler, 1);
int newReg = compiler->freeReg - 1;
emitABC(compiler, OP_MOVE, newReg, funcReg, 0);
funcReg = newReg;
func->data.loc.index = funcReg;
}
This protects the callee layout. Sometimes the callee expression may already live in an older register. For example, imagine the compiler has a local function value in R1, but the next free register is R3. If the compiler used R1 as the callee register and them placed arguments at R2, it would overwrite the existing values. So the callee register is not the most recently reserved register, the compiler moves it to the top:
1
2
3
4
5
reserveReg(compiler, 1);
int newReg = compiler->freeReg - 1;
emitABC(compiler, OP_MOVE, newReg, funcReg, 0);
funcReg = newReg;
func->data.loc.index = funcReg;
This is one of the most important details in the call compiler. A register-based VM is fast because instructions can refer directly to registers, but that also means the compiler must carefully manage register placement. Function calls require a contiguous register window, so the compiler must sometimes move the callee to make room for arguments.
After the callee is in the correct place, the compiler parses the arguments:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
if(!match(compiler, TOKEN_RIGHT_PAREN)){
do{
ExprDesc arg;
expression(compiler, &arg);
int targetReg = funcReg + argCnt + 1;
// function is at funcReg, arguments start from funcReg + 1
expr2Reg(compiler, &arg, targetReg);
freeExpr(compiler, &arg);
reserveReg(compiler, 1);
argCnt++;
if(argCnt >= 255){
errorAt(
compiler,
&compiler->parser.pre,
"Cannot have more than 255 arguments."
);
}
}while(match(compiler, TOKEN_COMMA));
consume(compiler, TOKEN_RIGHT_PAREN, "Expect ')' after arguments.");
}
For non-empty calls, each argument is parsed as a full expression. This means arguments can be arbitrary expressions. Each argument is then forced into a specific target register. The + 1 is important. The callee itself lives at funcReg, so the first argument must go into the next slot. At the end, argList() returns the number of arguments, not including the function itself.
In this implementation, the VM’s OP_CALL handler only reads A and B, so C is not doing meaningful work for ordinary calls yet. Conceptually, it leaves room for a future convention such as controlling return counts, but right now the result is simply placed back into the callee register.
Freeing Argument
After emitting OP_CALL, the compiler releases the argument registers:
1
freeRegs(compiler, argCount);
Only the argument registers are freed, not the callee register. This is because the callee becomes the result register.
Nested Calls
This design also explains why nested calls work naturally. For example:
1
outer(inner(1), 2)
When compiling the first argument to outer, the compiler recursively compiles inner(1). That inner call places its result in a register. Then the outer call moves the result into the correct argument slot for outer.
The compiler does not need a separate “nested call” feature. Calls are expressions, and expressions can appear as arguments.
This is the benefit of representing every expression using ExprDesc. A literal, a variable, a binary expression, and a function call can all eventually be forced into a register with expr2Reg().
Executing a Function Call
Now the compiler can emit a call instruction, and it has already arranged the registers into the shape the VM expects. Then the VM should handle the call instructions in DO_OP_CALL:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
DO_OP_CALL:
{
int a = GET_ARG_A(instruction);
int b = GET_ARG_B(instruction);
Value callee = R(a);
int argCount = b - 1;
vm->stackTop = &R(a + b);
int frameCnt = vm->frameCount;
if(!callValue(vm, callee, argCount)){
return VM_RUNTIME_ERROR;
}
if(vm->frameCount == frameCnt){
// no new frame was pushed
vm->stackTop = frame->base + frame->closure->func->maxRegSlots;
}
frame = &vm->frames[vm->frameCount - 1];
} DISPATCH();
The VM does not care whether the function came from a global variable, a local variable, a function expression, or another call. It simply looks at the value currently stored in register A.
vm->stackTop = &R(a + b); points one slot past the call area, this gives the runtime a clean way to compute where the new function frame begins.
The VM does not immediately assume the callee is a user-defined function. It calls callValue(). This is necessary because the language has multiple kinds of callable values. At this point in the article, we do not need to discuss these. We can simply treat this as unwrapping the function value and delegates to call:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
static bool call(VM* vm, ObjectClosure* closure, int argCnt){
if(argCnt != closure->func->arity){
runtimeError(vm, "Expected %d args but got %d.", closure->func->arity, argCnt);
return false;
}
if(vm->frameCount == FRAMES_MAX){
runtimeError(vm, "Stack overflow.");
return false;
}
Value* newBase = vm->stackTop - argCnt - 1; // -1 to skip func self
if(newBase + closure->func->maxRegSlots >= vm->stack + STACK_MAX){
runtimeError(vm, "Stack overflow.");
return false;
}
CallFrame* frame = &vm->frames[vm->frameCount++]; // 0-indexing++ to actual count
frame->closure = closure;
frame->ip = closure->func->chunk.code;
frame->base = newBase;
frame->deferCnt = 0;
vm->stackTop = frame->base + closure->func->maxRegSlots;
return true;
}
When the call happens, the VM compares the runtime argument count with the compiled function’s expected argument count.
The most important line in call() is this one:
1
Value* newBase = vm->stackTop - argCnt - 1;
Which lands exactly on the callee slot. The VM does not copy arguments into a new array. It does not allocate a separate argument object. It simply points the callee frame’s base at the existing call area. The VM does not copy arguments into a new array. It does not allocate a separate argument object. It simply points the callee frame’s base at the existing call area.
At this stage, we have explained normal function calls. We have not yet explained captured outer variables. Because closures need an additional mechanism for variables that outlive their original stack frame.
A function call does not copy arguments into a new environment. It creates a new frame view over an existing register window. Once the callee finishes, the VM must destroy that frame and place the return value back into the caller’s register. That is the next piece: returning from a function.
Returning from a Function
By entering the function is only half of the story. At this point, the function must finish. When it finishes, the VM has to do three things:
-
Get the return value
-
Remove the call frame
-
Put the return value into the caller’s register
Return Statement
The compiler handles return in returnStmt:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
static void returnStmt(Compiler* compiler){
if(compiler->type == TYPE_SCRIPT){
errorAt(compiler, &compiler->parser.pre, "Cannot return from the top-level.");
}
if(match(compiler, TOKEN_SEMICOLON)){
if(compiler->type == TYPE_INITIALIZER){
emitABC(compiler, OP_RETURN, 0, 2, 0);
}else{
int reg = getFreeReg(compiler);
emitABC(compiler, OP_LOADNULL, reg, 0, 0);
emitABC(compiler, OP_RETURN, reg, 2, 0);
}
}else{
if(compiler->type == TYPE_INITIALIZER){
errorAt(compiler, &compiler->parser.pre, "Cannot return a value from an initializer.");
}
ExprDesc retExpr;
expression(compiler, &retExpr);
expr2NextReg(compiler, &retExpr);
emitABC(compiler, OP_RETURN, retExpr.data.loc.index, 2, 0);
freeExpr(compiler, &retExpr);
consume(compiler, TOKEN_SEMICOLON, "Expect ';' after return value.");
}
}
A return statement only makes sense inside a function. The top-level script is not a callable body, so the compiler rejects this. This is an important distinction between implementation and language design. Internally, the VM may execute the script through a function-like frame, but syntactically, return still belongs to function bodies.
The next case is an empty return:
1
2
3
if(match(compiler, TOKEN_SEMICOLON)){
...
}
For normal functions, the compiler emits:
1
2
3
int reg = getFreeReg(compiler);
emitABC(compiler, OP_LOADNULL, reg, 0, 0);
emitABC(compiler, OP_RETURN, reg, 2, 0);
So a return is actually loading null into a temporary register, and then return the register. Conceptually, the return; means return null;
The more common case is returning a value:
1
2
3
4
5
6
7
8
ExprDesc retExpr;
expression(compiler, &retExpr);
expr2NextReg(compiler, &retExpr);
emitABC(compiler, OP_RETURN, retExpr.data.loc.index, 2, 0);
freeExpr(compiler, &retExpr);
consume(compiler, TOKEN_SEMICOLON, "Expect ';' after return value.");
It treats the retrun value as a normal expression. The VM’s return instruction expects the returned value to live in a register. So before emitting OP_RETURN, the compiler makes sure the expression has a real register location.
1
expr2NextReg(compiler, &retExpr);
Finally, it emits:
1
emitABC(compiler, OP_RETURN, retExpr.data.loc.index, 2, 0);
The A stores the return value register. Again, this follows the same name-erasure principle we have used throughout the compiler. By the time bytecode runs, source-level names and expressions have already been lowered into register operations.
Implicit Return
A function body may finish without an explicit return. The VM still needs a clear signal that the function is done. So the stopCompiler emits an inplicit return:
1
2
3
4
5
6
7
8
9
10
11
12
static ObjectFunc* stopCompiler(Compiler* compiler){
if(compiler->type == TYPE_INITIALIZER){
emitABC(compiler, OP_RETURN, 0, 2, 0);
}else{
int reg = getFreeReg(compiler);
emitABC(compiler, OP_LOADNULL, reg, 0, 0);
emitABC(compiler, OP_RETURN, reg, 2, 0);
}
ObjectFunc* func = compiler->func;
func->maxRegSlots = compiler->maxRegSlots;
return func;
}
Executing OP_RETURN
At runtime, returning is handled by DO_OP_RETURN:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
DO_OP_RETURN:
{
int a = GET_ARG_A(instruction);
int b = GET_ARG_B(instruction);
Value result = (b > 1) ? R(a) : NULL_VAL;
Value* calleeBase = frame->base;
if(frame->closure->func->type == TYPE_MODULE){
popGlobal(vm);
}
vm->frameCount--;
if(vm->frameCount == 0){
pop(vm);
return VM_OK;
}
frame = &vm->frames[vm->frameCount - 1];
vm->stackTop = frame->base + frame->closure->func->maxRegSlots;
calleeBase[0] = result;
} DISPATCH();
In our normal return instructions, B is emitted as 2, so b > 1 is true. That means the result comes from register A. The 2 does not mean returning the number 2. It is the part of the VM’s return convention. It tells the VM that this return instruction carries an actual return value.
Before the VM pops the current frame, it stores the frame base:
1
Value* calleeBase = frame->base;
This is the key to placing the result back into the caller.
The return instruction then removes the current call frame by vm->frameCount--;. Now the current function is no longer active.
If this was the last frame, execution is finished:
1
2
3
4
if(vm->frameCount == 0){
pop(vm);
return VM_OK;
}
When the script frame returns, there is no caller to resume, so the VM returns VM_OK.
For normal nested function calls, however, there is still a caller frame below the current one. So the VM restores the caller frame:
1
frame = &vm->frames[vm->frameCount - 1];
Finally, the VM writes the return value into the original callee slot calleeBase[0].
Wrapping Up
In this article, we added ordinary functions to the register-based interpreter. The main idea is simple: a function is not compiled into the surrounding bytecode directly. Instead, each function owns its own Chunk through an ObjectFunc.
When the compiler sees a function, it creates a nested compiler. That nested compiler emits bytecode into the function’s private chunk. After the function body is compiled, the outer compiler resumes and emits code that creates a function value.
When the function finishes, OP_RETURN removes the current frame and writes the return value back into the original callee register.
With this, the interpreter is no longer limited to one monolithic script. It now supports reusable executable units, argument passing, call frames, and return values.