Technical details
Variable storage
On 6502-based targets, all variables are allocated statically and there is no concept of dynamic heap or stack frames. The virtual target uses the same static-memory model. M68K targets use stack frames for ordinary subroutine locals and parameters; see M68K memory model and calling convention below. Essentially all variables are global (but scoped) and can be accessed and modified anywhere, but care should be taken of course to avoid unexpected side effects.
Especially when you’re dealing with interrupts or re-entrant routines: don’t modify variables that you not own or else you will break stuff.
Variables that are not put into zeropage, will be put into a special ‘BSS’ section for the assembler. This section is usually placed at the end of the resulting program but because it only contains empty space it won’t actually increase the size of the resulting program binary. Prog8 takes care of properly filling this memory area with zeros at program startup and then reinitializes the subset of variables that have a nonzero initialization value.
Arrays with initialization values are not put into BSS but just occupy a sequence of bytes in the program memory: their values are not reinitialized at program start.
It is possible to relocate the BSS section using a compiler option so that more system ram is available for the program code itself.
ROM/RAM bank selection
On certain systems prog8 provides support for managing the ROM or RAM banks that are active.
system |
get banks (returns byte) |
set banks |
|---|---|---|
c64 |
|
|
c128 |
|
|
cx16 |
|
|
other |
N/A |
N/A |
Banked data on the Commander X16
Use the -varshigh <rambank> compiler option to place uninitialized,
non-zero-page variables and memory() slabs in the $A000-$BFFF window
of a chosen HIRAM bank. For example, compile with bank 3:
prog8c -target cx16 -varshigh 3 program.p8
The compiler selects that bank while initializing the relocated BSS at startup,
but it does not switch banks automatically around later variable accesses. The
selected HIRAM bank must match the bank containing the variables whenever the
program reads or writes them. Initialized data remains in the program image and
is not moved by -varshigh. See Compiling a program for the placement options.
A program normally needs no bank switching around accesses to these variables:
leave the configured HIRAM bank selected while using them.
If code must temporarily use another HIRAM bank, save the current bank, switch,
and restore it afterward. For example, when bank 3 is active,
cx16.push_rambank(4) switches to bank 4 and cx16.pop_rambank() restores
bank 3. This lets later accesses to the relocated variables continue to work.
Pair each push with a pop on every control-flow path. Use cx16.rambank(bank)
when deliberately changing the active bank without saving the previous one.
The %varsaddress <address> directive and -varsaddress <address> option
are raw-address alternatives, not bank selectors. When placing BSS in the
$A000-$BFFF window this way, the program must also arrange for the intended
HIRAM bank to be active during initialization and every access. See
Programming in Prog8 for the directive.
Calling a subroutine in another memory bank can be done by using the callfar or callfar2 builtin functions.
When you are using the routines above, you are doing explicit manual banks control. However, Prog8 also provides something more sophisticated than this, when dealing with banked subroutines:
External subroutines defined with extsub can have a non-standard ROM or RAM bank specified as well.
The compiler will then transparently change a call to this routine so that the correct bank is activated
automatically before the normal jump to the subroutine (and switched back on return). The programmer doesn’t
have to bother anymore with setting/resetting the banks manually, or having the program crash because
the routine is called in the wrong bank! You define such a routine by adding @bank <bank>
to the extsub subroutine definition. This specifies the bank number where the subroutine is located in.
You can use a constant bank number 0-255, a ubyte variable, or even the name of a subroutine
(must have a single ubyte parameter, and returning a ubyte) to make it dynamic:
extsub @bank 10 $C09F = audio_init()
extsub @bank banknr $A000 = first_hiram_routine()
extsub @bank get_bank $A000 = second_hiram_routine()
When a subroutine is used as a banking routine, the compiler will call it just before the actual banked subroutine is invoked. The compiler passes a unique “call-site ID” (0-255) as a single ubyte parameter in register A to the banking routine. The return value of the banking routine (in register A) is then used as the bank number for the subsequent call.
Call-site IDs
To support advanced scenarios like dynamic overlay loading, the compiler assigns a unique call-site ID to every banked external subroutine that uses a subroutine for its bank selection.
- What they are
A call-site ID is a unique
ubytevalue (0-255) assigned by the compiler to each unique bankedextsubdeclaration that specifies a bank selector routine.- How they work
When your program calls a banked subroutine, and that subroutine has a bank selector routine (instead of a constant bank number), the compiler:
Loads the unique ID associated with that subroutine into the accumulator (register A).
Calls your bank selector routine.
Your routine uses the ID in A to decide which bank to activate (and potentially load code into it).
Your routine returns the bank number in A.
The compiler then performs the far call to the target subroutine in the returned bank.
- What they are intended for
These IDs are primarily intended for implementing Overlay Managers, dynamic library loading, or any scenario where the bank number isn’t known at compile time (e.g. a library loaded into the first available HiRAM bank, or language-specific versions selected via a global setting). An overlay manager can use the ID as an index into a table to discover:
Which bank the required code should be in.
Which file or offset on disk contains the code for this routine.
Whether the code is already loaded or needs to be fetched.
Without these IDs, the bank selector would only know that some routine is being called, but it wouldn’t know which one unless you had a separate manager routine for every single banked subroutine.
The same banked subroutine called from multiple different places in your code will currently
result in the same ID being passed to the banking routine, as the ID is unique per extsub declaration.
- Viewing assigned IDs
During compilation, the compiler outputs a list of all assigned call-site IDs into a file named
<stem>.bankedcallsin the output directory (the stem is the name of the program, or of the file given with the-ooption). This information is useful for synchronizing your overlay manager with the subroutines it needs to handle.The IDs are determined once before code generation begins, by collecting all
extsubdeclarations that have a bank selector and a bank number assigned, scanning the program in declaration order, and then assigning sequential IDs starting from 0. This means the first bankedextsubin your source code gets ID 0, the second gets ID 1, and so on. The IDs are deterministic and stable across compilations — adding a new banked subroutine at the end of the source will not change the IDs of existing declarations.ID Address Name BankSelector -------------------------------------------------- 0 $ffd2 main.chrout main.selector 1 $a000 main.other_sub main.selector
This might be useful for implementing dynamic overlay loading, where a banking routine can check if the required code is already loaded in a certain bank, load it from disk if necessary, and then return the bank number to the banked subroutine call.
When you then call this routine in your program as usual, the compiler will no longer generate a simple JSR instruction to the routine. Instead it will generate a piece of code that automatically switches the ROM or RAM bank to the correct value, does the call, and switches the bank back. The exact code will be different for different compilation targets, and not all targets even have banking or support this. As an example, on the Commander X16, prog8 will use the JSRFAR kernal routine for this. On the Commodore 128, a similar call exists (but requires a lot more code to prepare, so beware). On the Commodore 64 some custom code is also emitted that toggle the banks, retains some registers, and does the call. Other compilation targets don’t have banking or prog8 doesn’t yet support automatic bank selection on them.
There’s a “banking” example for the Commander X16 that shows a possible application of the extsub with bank support, check out the bank example code .
There’s also a “dynamicbanking” example for the Commander X16 that demonstrates the use of a bank selector subroutine with call-site IDs to manage multiple subroutines across different HiRAM banks, using library blobs loaded from disk: dynamic banking example code .
Notice that the symbol for this routine in the assembly source code will still be defined as usual. The bank number is not translated into assembly (only as a comment):
p8s_audio_init = $c09f ; @bank 10
Caution
Calls with automatic bank switching like this are not safe to use from IRQ handlers. Don’t use them there. Instead change banks in a controlled manual way (or not at all).
Note
On the C64 and C128, the Basic ROM is banked out by default when running a Prog8 program, because
it is not needed. This means on the C64 we get access to another 8Kb of RAM at that
memory area, which actually gives us a 50 Kb contiguous RAM block from $0801 to $d000 (exclusive).
This means you can create programs of up to 50 Kb size with prog8 on the C64.
On the C128, it means programs can use ~41 Kb of contiguous RAM at $1c00 to $c000 (exclusive).
However, if your program uses floats, Prog8 does need the Basic ROM for the floating point routines,
and it won’t be banked out. Such programs are limited to the regular size of about 38 Kb on the C64, and less on the C128.
Be aware: the bank setting is only done if you are not using %option no_sysinit because the
program’s bootstrap code is what initializes the memory bank configuration.
Symbol prefixing in generated Assembly code
All symbols in the prog8 program will be prefixed in the generated assembly code:
Element type |
prefix |
|---|---|
Block |
|
Subroutine |
|
Variable |
|
Constant |
|
Label |
|
Struct |
|
Struct Field |
|
Enum Member |
|
other |
|
This is to avoid naming conflicts with CPU registers, assembly instructions, etc.
So if you’re referencing symbols from the prog8 program in inlined assembly code, you have to take
this into account. Stick the proper prefix in front of every symbol name component that you want to reference that is coming
from a prog8 source file.
All elements in scoped names such as main.routine.var1 are prefixed so this becomes p8b_main.p8s_routine.p8v_var1.
Attention
Symbols from library modules are not prefixed and can be used in assembly code as-is. So you can write:
%asm {{
lda #'a'
jsr cbm.CHROUT
}}
Subroutine Calling Conventions
Calling a subroutine requires three steps:
preparing the arguments (if any) and passing them to the routine. Numeric types are passed by value (bytes, words, longs, booleans, floats), but array types passed by reference which means as
uwordbeing a pointer to their address in memory. Strings are passed as a pointer to a byte:^^ubyte.calling the subroutine
preparing the return value (if any) and returning that from the call.
The exact convention depends on the compilation target. The static-memory convention is described below for 6502-based targets and the virtual target. M68K targets use the frame-based convention described in M68K memory model and calling convention.
M68K memory model and calling convention
On the amiga500, amiga1200 and qemu68k targets, ordinary subroutine locals and
parameters are stored in an activation record on the CPU stack. Each invocation gets its own
frame, so ordinary subroutines can be recursive and re-entrant. Block-level variables, globals,
@shared variables, memory slabs, and objects that must have a permanent externally visible
address remain statically allocated.
The frame pointer is register A5. A6 remains available for AmigaOS library bases.
Locals are addressed at negative offsets from A5. Parameters are in the incoming argument
area at positive offsets. The compiler establishes and removes a frame with link a5,#-N
and unlk a5.
Virtual registers are kept in hardware registers (D2-D6, A2-A4, FP2-FP7) by an
intraprocedural register allocator, with values that live across calls placed in the
callee-saved registers and saved/restored by the subroutine’s movem prologue and
epilogue. Values that cannot stay in a register spill to slots appended to the A5
frame. D7, A5 and A6 are reserved. Ordinary subroutine locals may therefore
never be accessed directly from assembly; go through the frame instead.
Clean (non-@dirty) locals are zero-initialized on every call with a clr prologue, one
store per element. This includes arrays, which on other targets are only cleared once at startup.
A large clean local array is therefore costly; the compiler warns about framed arrays larger
than 8 bytes. Add @dirty when you assign the array before use. A framed @dirty local
has no static storage and is not zeroed at startup either, so its initial value is stack garbage.
Normal Prog8 subroutines use one all-stack argument convention with word-packed slots:
The caller evaluates arguments and pushes them from left to right.
Every argument occupies a slot of its natural width: byte, ubyte, word and uword arguments take a two-byte slot; long, pointer and float arguments take a four-byte slot. Byte values sit at the slot’s base byte (they occupy the slot’s first byte; the second byte is garbage), word values occupy the whole two-byte slot, and wide values fill the four-byte slot.
After
link a5,#-N, the rightmost argument is at8(a5). The offsets accumulate from the rightmost argument according to each argument’s slot width, so parameteri’s slot base depends on the widths of the parameters to its right.The caller removes the complete argument area after the call. The area size is always even, so the machine stack stays word-aligned. The callee’s epilogue does not depend on the number of arguments.
Scalar and pointer results remain register-based, normally in
D0. Float results use the floating-point accumulator.
The other byte of a two-byte slot holding a byte argument is undefined: read byte arguments byte-size-only. Because all slot sizes are even, slot bases stay word-aligned; a byte value lands on an even address, which is fine on the 68000 as a byte access.
asmsub and extsub keep their explicit register-based, annotation-driven convention.
AmigaOS library calls continue to use A6 for the selected library base. Hand-written M68K
assembly that calls a normal Prog8 subroutine must follow the stack convention and preserve
A5.
On M68K targets, the compiler rejects a clobbers(...) list on an asmsub. Instead, the
routine itself must preserve the registers the register allocator relies on:
D2-D6,A2-A4andFP2-FP7are callee-saved: save and restore any of them the routine uses.D7andA5are reserved: the routine may use them, but must save and restore them around the change.D0-D1,A0-A1andFP0-FP1are scratch registers and may be changed freely.A6is outside the allocator’s pool. The compiler reloads the library base into it before every AmigaOS call, so anasmsubneed not preserve it.
AmigaOS library calls preserve only D2-D5, A2-A5 and A6, so D6 is volatile across
them. An inline asmsub runs in its caller’s register context, so it must save and restore D6
itself around any library call.
The compiler relies on this rule when it allocates registers around a call to an asmsub, so a
call destroys only the caller-saved registers. When an asmsub body changes a callee-saved or
reserved register without saving it, the compiler reports a warning. The warning counts an assembly
level call through A6 (jsr/jmp with an (a6) operand) as a D6 change as well, so
a routine that calls the OS through such a vector must save and restore D6 itself.
An extsub may still have an optional clobbers(...) list; external routines are outside
the compiler’s control, so calls to them assume that every register may be changed.
For example, calling sub add2(ubyte a, uword b) -> uword:
; call add2($12, $3456); word-packed slots
move.b #$12,-(sp) ; byte: A7 steps by 2, so this reserves the 2-byte slot
; and the byte lands at the slot's base byte
move.w #$3456,-(sp) ; word: 2-byte predecrement push
jsr p8b_main.p8s_add2
addq.l #4,sp ; caller pops the 4-byte argument area
; result is now in d0
Long, pointer and float arguments fill the whole slot, so they push in one step
(move.l x,-(sp) or fmove.s fp0,-(sp)). Byte and word pushes always reserve their
2-byte slot - do not push them where the callee expects a wide argument, because that
desynchronizes the slot layout. Use lea N(sp),sp instead of addq for the cleanup
when the call returns a status flag: addq sets condition codes and would clobber the
returned flags.
Note
The calling convention of normal subroutines changed with IR format 8 from one padded
4-byte slot per argument to the word-packed layout above. Hand-written assembly written
against the old convention must be updated; older .p8ir files are rejected because
of this change.
Note
The built-in virtual machine does not currently execute frame-based IR. It rejects M68K frame-based programs instead of treating frame slots as static memory. Run M68K programs with the M68K backend and an appropriate emulator.
Caution
A frame address is valid only until its subroutine returns. The compiler emits best-effort warnings for addresses that escape, but does not catch every indirect escape. Taking the address of a parameter, complex string or array initializers, and some externally visible or dynamically dispatched subroutines can prevent frame allocation. Such cases remain static or are rejected when framing them would be unsafe.
A defer body that accesses a subroutine’s locals or parameters is not supported when
that subroutine can be recursive or dynamically re-entered. The compiler rejects that case.
on..goto and on..call jump tables remain static because their symbolic initializers
cannot currently be placed in a frame.
Assembly code that writes to A5 without preserving the caller’s frame pointer would
invalidate every frame slot of its caller, so a subroutine that (directly or indirectly) calls
such assembly is not given a stack frame and keeps the static convention instead. The usual
move.l a5,-(sp) … move.l (sp)+,a5 prologue and epilogue is recognised as balanced.
A single local frame is limited to 16 KiB. The compiler reports oversized frames, but does not detect exhaustion caused by a deep call chain or recursion. Frame-slot reuse is also conservative in routines containing loops, so such routines may use more stack space than necessary.
6502-based and virtual static-memory subroutines
Each subroutine parameter is represented as a variable scoped to the subroutine. These targets do not have call stack frames.
The arguments passed in a subroutine call are evaluated by the caller, and then put into those variables by the caller. The order of evaluation of subroutine call arguments is unspecified and should not be relied upon.
The subroutine is invoked.
The return value is not put into a variable, but the subroutine passes it back to the caller via register(s). See below.
Single arguments will often be passed in registers:
For single byte, word, and pointer arguments (not long or float), the values are simply loaded in cpu registers by the caller before calling the subroutine. The subroutine itself will take care of putting the values into the parameter variables. This saves on code size because otherwise all callers would have to store the values in those variables themselves. This convention is also still used for subroutines that specify parameters to be put into one of the virtual registers R0-R15, as those are in the end just variables too (see Reusing virtual registers R0-R15 for parameters) The rules are as follows:
- Single byte parameter:
sub foo(ubyte bar) { ... } gets bar in the accumulator A, subroutine stores it into parameter variable
- Two byte parameters:
sub foo(ubyte bar, ubyte baz) { ... } gets bar in the accumulator A, and baz in Y, subroutine stores it into parameter variable
- Single word parameter:
sub foo(uword bar) { ... } gets bar in the register pair A + Y (lsb in A, msb in Y), subroutine stores it into parameter variable
- Single pointer parameter:
sub foo(^^ubyte bar) { ... } gets bar in the register pair A + Y (lsb in A, msb in Y), subroutine stores it into parameter variable
- Long or Floating point parameter:
sub foo(long bar) { ... },sub foo(float bar) { ... } value for bar gets stored into the parameter variable by the caller
- Other:
sub foo(ubyte bar, ubyte baz, ubyte zoo) { ... } not using registers; all values get stored in the subroutine’s parameter variables by the caller
Return value
A byte return value will be put in
A.A boolean return value will be put in
Atoo, as 0 or 1.A word return or pointer value will be put in
A+Yregister pair (lsb in A, msb in Y).A long return value will be put into
cx16.r14andcx16.r15(2 word virtual registers, low word and high word, to make up a single 32 bits long)A float return value will be put in the
FAC1float ‘register’.
In case of multiple return values:
for an
asmsuborextsubthe subroutine’s signature specifies the output registers that contain the values explicitly, just as for a single return value.for regular subroutines, the compiler will return the first of the return values via the cpu register
A`(orA + Y`if it’s a word value), just like for subroutines that only return a single value. The remainder of the return values are returned via the “virtual registers” cx16.r15-cx16.r0 (using R15 first and counting down to R0). Long values will take a pair of those “virtual registers” that combined make up a single 32 bits value. A floating point value is passed via FAC1. Multiple float return values are supported on the virtual target, but limited to a single float on 6502 targets (because the ROM float routines use FAC1/FAC2 as operand registers which would clobber earlier return values).
asmsub and extsub routines
These are kernal (ROM) routines or low-level assembly routines, that get their arguments via specific registers.
Sometimes even via a processor status flag such as the Carry flag.
Word values can be put in a “CPU register pair” such as AY (meaning A+Y registers) but also
in one of the 16 ‘virtual’ 16 bit registers introduced by the Commander X16, R0-R15.
Float values can be put in the FAC1 or FAC2 floating point ‘registers’.
The return values also get returned via designated registers, or via processor status flags again.
This means that after calling such a routine you can immediately act on the status
via a special branch instruction such as if_z or if_cs etc.
The register/status flag usage is fully specified in the asmsub or extsub signature definition
for both the parameters and the return values:
extsub $2000 = extfunction(ubyte arg1 @A, uword arg2 @XY, uword arg3 @R0,
float frac @FAC1, bool flag @Pc) -> ubyte @Y, bool @Pz
asmsub function(ubyte arg1 @A, uword arg2 @XY, uword arg3 @R0,
float frac @FAC1, bool flag @Pc) -> ubyte @Y, bool @Pz {
%asm {{
...
...
}}
}
Float support contract (floats.* routines expected by the code generators)
How much of the floats block a target’s library must provide depends on the backend.
The 6502 backends call into floats.* routines by hardcoded name; the m68k and virtual
backends don’t (m68k lowers float operations to FPU instructions, virtual to IR float
operations plus ordinary library calls), so they have no hardcoded contract beyond the
user-visible API. On CBM targets most of the arithmetic entry points are extsub
declarations pointing at the kernal ROM routines (addresses differ per target).
A new CBM-style target only needs the new IR-based codegen’s contract, which comes in tiers — a target can support scalar floats without transcendentals or float arrays:
Tier 0 — scalar floats (23 routines): assignment, arithmetic, comparisons, int/float casts, float subroutine arguments and return values.
FADD FMULT FADDT FMULTT FDIVT FSUBT
(memory-operand add/multiply plus FAC2-operand add/multiply/divide/subtract;
note the memory-operand FSUB/FDIV are not used)
FREADSA FREADUY GIVAYFAY GIVUAYFAY (int to float)
MOVFM MOVMF MOVAF MOVFA CONUPK (FAC1/FAC2 movement)
NEGOP (unary minus) QINT (float to long) SIGN (comparison via
FSUBT + SIGN instead of FCOMP)
cast_as_w_into_ay cast_as_uw_into_ya (float to int)
pushFAC1 popFAC (float call arguments)
FAC_ADDR (const uword: base address of the FAC1/FAC2 working registers).
Tier 1 — transcendental functions (+10 ROM routines, only needed if the program uses
them or the ** operator): ABS ATN COS INT LOG ROUND
SIN SQR TAN FPWRT.
Tier 2 — float arrays (+1 routine, only needed for the in containment operator on
float arrays): containment_floatarray.
The legacy 6502 codegen needs 38 routines on top of tier 0 (33 if the program uses no float arrays): cast/copy/comparison trampolines that the new backend inlines or routes directly to ROM instead:
cast_from_b cast_from_ub cast_from_w cast_from_uw
cast_as_bool_into_a cast_as_long
cast_FAC1_as_bool_into_a cast_FAC1_as_w_into_ay cast_FAC1_as_uw_into_ya
internal_long_AY_to_FAC internal_long_R1_to_float_AY
copy_float copy_float2 swap_floats pushFAC
set_array_float set_array_float_from_fac1 set_0_array_float
inc_var_f dec_var_f
var_fac1_equal_f var_fac1_notequal_f var_fac1_less_f var_fac1_lesseq_f
var_fac1_greater_f var_fac1_greatereq_f
vars_equal_f vars_less_f vars_lesseq_f
func_abs_f_into_FAC1 func_sign_f_into_A func_sqrt_into_FAC1
FSUB FDIV FADDH MUL10 MOVEF floats_temp_var
(5-byte temporary float storage).
So: 23 routines for scalar floats on the new backend (34 with transcendentals, 35 with
float-array containment), versus 62 for the legacy backend. The helpers live in
c64/floats.asm + c64/floats_funcs.asm on CBM targets (included by both
c64/floats.p8 and cx16/floats.p8), with per-target equivalents elsewhere.
Regenerate per backend with:
rg -o "floats\.[A-Za-z0-9_]+" codeGenCpu6502/src --no-filename | sort -u # legacy 6502
rg -o "floats\.[A-Za-z0-9_]+" codeGenNew6502/src --no-filename | sort -u # new 6502
User-visible routines such as sin(), cos(), sqrt(), print() and tostr()
are not part of this contract; they are regular library functions documented in
Library modules and builtin functions that are implemented on top of the above.
Compiler Internals
Here is a diagram of how the compiler translates your program source code into a binary program:
Some notes and references into the compiler’s source code modules:
The
compileProgram()function (in thecompilermodule) does all the coordination and basically drives all of the flow shown in the diagram.ANTLR is a Java parser generator and is used for initial parsing of the source code. (
parsermodule)Most of the compiler and the optimizer operate on the Compiler AST. These are complicated syntax nodes closely representing the Prog8 program structure. (
compilerAstmodule)For code generation, a much simpler AST has been defined that replaces the Compiler AST. Most notably, node type information is now baked in. (
codeCoremodule, Pt- classes)An Intermediate Representation has been defined that is generated from the simplified AST. This IR is more or less a machine code language for a virtual machine - and indeed this is what the built-in prog8 VM will execute if you use the ‘virtual’ compilation target and use
-emuto launch the VM. (intermediateandcodeGenIntermediatemodules, andvirtualmachinemodule for the VM related stuff) This IR is still targeted to one specific compilation target only; various properties and all library code for the selected target machine is encoded into the IR. It is not possible to eventually create a C64 program from an IR file created for the CommanderX16 target.All symbol names in the IR are fully scoped (they include the entire dotted path from the outermost block down to the symbol itself). Additionally, each part of the scoped name carries a type prefix that identifies what kind of symbol it is. Any backend consuming IR files can rely on these already being present and does not need to add them itself:
p8b_prefix for block names (e.g.p8b_main- simple, no dots)p8s_prefix for subroutine names (e.g.p8b_main.p8s_start)p8v_prefix for variables (e.g.p8b_main.p8s_start.p8v_x)p8c_prefix for constants (e.g.p8b_main.p8c_SIZE)p8l_prefix for labels (e.g.p8b_main.p8s_start.p8l_loop)p8t_prefix for struct type names (e.g.p8b_main.p8t_Point)p8i_prefix for struct instance names
Library code that uses
%option no_symbol_prefixingkeeps its names without these prefixes. The built-in Virtual Machine strips these prefixes automatically when loading an IR program.The code generator backends all implement a common interface
ICodeGeneratorBackenddefined in thecodeCoremodule. Currently they get handed the program Ast, Symboltable and several other things. The default 6502 code generator consumes the simplified AST directly. The experimental-newcodegen6502 backend, the M68K backend, and the VM use theIRCodeGenclass fromcodeGenIntermediateto convert the AST into IR first.
ROM-able programs
Normally Prog8 will use some tricks to generate the smallest and most optimized code it can. This includes the following techniques that by default prevent generated program code from running in ROM:
- self-modifying code
This is program code that actually modifies itself during program execution (instructions or operands are modified) When the program is in ROM, such modifications are impossible, so the program will not execute correctly.
- inline variables
These are variables that are located in the same memory region that the program code is in (or even interleaved within the program code). Again, writing to such variables will not work when it is in ROM, so the program will not execute correctly.
(Not all prog8 source code will end up using these techniques but you should not depend on it.)
The directive %option romable changes this behavior.
It tells the compiler to no longer generate code using these two tricks, and instead revert to slightly slower running code (or needing more instructions)
but which is able to run from ROM.
There are a few things to note:
string variables and array variables that are initialized with something other than just zeros, are no longer mutable. This is because both of these will still end up as part of the same memory region the program code is in (which will be ROM). The compiler will try to detect writes to them and give an error if these occur. However it cannot detect all such writes, so beware.
arrays without an initialization literal will be placed into the memory region for variables instead which can and should be placed in RAM, so those arrays are mutable as usual.
the same holds for memory blocks allocated using the
memoryfunction; nothing changes for them.the memory region for variables and memory blocks (BSS sections) should be explicitly placed in RAM memory. You can do this with the
%varsaddressmodule directive (or the-varsaddressCLI option, or thevars_addresstarget config property), which directly specifies the memory address to use for both variables and slabs together, or with the target-aware-varsgoldenor-varshighcommand line options. Do not mix both mechanisms: use either the raw address directive/option or the golden/high RAM shorthand, not both.
Note
The ROMable code generation is still quite experimental. Problems may still surface, and perhaps stuff will change a bit in a future compiler version.
Formal ANTLR4 syntax and grammar definition
/*
Prog8 combined lexer and parser grammar
NOTES:
- whitespace is ignored. (tabs/spaces)
- every position can be empty, be a comment, or contain ONE statement.
*/
// -> java classes Prog8ANTLRParser and Prog8ANTLRLexer,
// both NOT to be used from Kotlin code, but ONLY through Kotlin class Prog8Parser
grammar Prog8ANTLR;
@header {
package prog8.parser;
}
EOL : ('\r'? '\n' | '\r' | '\n')+ ;
LINECOMMENT : EOL [ \t]* COMMENT -> channel(HIDDEN);
COMMENT : ';' ~[\r\n]* -> channel(HIDDEN) ;
BLOCK_COMMENT : '/*' ( BLOCK_COMMENT | ~'*' | '*' ~'/' )*? '*/' -> channel(HIDDEN) ;
WS : [ \t] -> skip ;
// WS2 : '\\' EOL -> skip;
VOID: 'void';
STRUCT: 'struct';
UNION: 'union';
ON: 'on';
GOTO: 'goto';
CALL: 'call';
INLINE: 'inline';
PRIVATE: 'private';
PUBLIC: 'public';
STEP: 'step';
VIA: 'via';
UNTIL: 'until';
ELSE: 'else';
THEN: 'then';
ENUM: 'enum';
POINTERSTR: 'pointer';
UNICODEDNAME : [\p{Letter}]([\p{Letter}\p{Mark}\p{Digit}_] | '::')* ; // match unicode properties
UNDERSCORENAME : '_' UNICODEDNAME ; // match unicode properties
DEC_INTEGER : DEC_DIGIT (DEC_DIGIT | '_')* ;
HEX_INTEGER : '$' HEX_DIGIT (HEX_DIGIT | '_')* ;
BIN_INTEGER : '%' BIN_DIGIT (BIN_DIGIT | '_')* ;
ADDRESS_OF: '&' ;
TYPED_ADDRESS_OF: '&&' ;
ADDRESS_OF_MSB: '&>' ;
ADDRESS_OF_LSB: '&<' ;
POINTER: '^^';
fragment HEX_DIGIT: ('a'..'f') | ('A'..'F') | ('0'..'9') ;
fragment BIN_DIGIT: ('0' | '1') ;
fragment DEC_DIGIT: ('0'..'9') ;
FLOAT_NUMBER : DEC_DIGIT (DEC_DIGIT | '_')* ('.' (DEC_DIGIT | '_')*)? (('E'|'e') ('+'|'-')? DEC_INTEGER)?
| '.' (DEC_DIGIT | '_')+ (('E'|'e') ('+'|'-')? DEC_INTEGER)?
;
STRING_ESCAPE_SEQ : '\\' [\u0021-\u007E] | '\\x' HEX_DIGIT HEX_DIGIT | '\\u' HEX_DIGIT HEX_DIGIT HEX_DIGIT HEX_DIGIT;
STRING :
'"' ( STRING_ESCAPE_SEQ | ~[\\\r\n\f"] )* '"'
;
INLINEASMBLOCK :
'{{' .+? '}}'
;
SINGLECHAR :
'\'' ( STRING_ESCAPE_SEQ | ~[\\\r\n\f'] ) '\''
;
TAG: '@' ('a'..'z' | '0'..'9')+ ;
EMPTYARRAYSIG : '[' [ \t]* ']' ;
NOT_IN: 'not' [ \t]+ 'in' [ \t] ;
// A module (file) consists of zero or more directives or blocks, in any order.
// If there are more than one, then they must be separated by EOL (one or more newlines).
// However, trailing EOL is NOT required.
// Note: the parser may see *several* consecutive EOLs - this happens when EOL and comments are interleaved (see #47)
module: EOL* (module_element (EOL+ module_element)*)? EOL* EOF;
module_element:
directive | block ;
block: identifier integerliteral? EOL? '{' EOL? (block_statement | EOL)* '}';
// Note: enum and alias appear in both block_statement and statement rules.
// This is intentional - they are declaration statements like variabledeclaration,
// and can appear at block level or inside subroutines (local enums/aliases).
block_statement:
directive
| subroutinedeclaration
| variabledeclaration
| structdeclaration
| uniondeclaration
| inlineasm
| labeldef
| alias
| enum
;
statement :
directive
| ongoto
| subroutinedeclaration
| variabledeclaration
| structdeclaration
| uniondeclaration
| assignment
| augassignment
| unconditionaljump
| postincrdecr
| functioncall_stmt
| if_stmt
| branch_stmt
| inlineasm
| returnstmt
| forloop
| whileloop
| untilloop
| repeatloop
| unrollloop
| whenstmt
| breakstmt
| continuestmt
| labeldef
| defer
| alias
| enum
| swap
// Error recovery: match tokens that can NEVER start a valid expression
| 'if' '{'
{ notifyErrorListeners("Missing condition: expected expression after 'if'"); }
| 'while' '{'
{ notifyErrorListeners("Missing condition: expected expression after 'while'"); }
| 'for' identifier 'in' '}'
{ notifyErrorListeners("Missing range: expected expression after 'in'"); }
| 'for' identifier 'in' ('else' | 'return' | 'break' | 'continue' | 'defer' | '}')
{ notifyErrorListeners("Missing range: expected expression after 'in'"); }
| 'return' ('}' | 'else' | 'while' | 'for' | 'if' | 'defer')
{ notifyErrorListeners("Invalid token after 'return'"); }
| 'defer' ('}' | 'else' | 'while' | 'for' | 'if' | 'return' | 'break' | 'continue')
{ notifyErrorListeners("Expected statement after 'defer'"); }
| 'when' '{'
{ notifyErrorListeners("Missing expression: expected condition after 'when'"); }
| ENUM '{'
{ notifyErrorListeners("Expected enum name after 'enum'"); }
| STRUCT '{'
{ notifyErrorListeners("Expected struct name after 'struct'"); }
| UNION '{'
{ notifyErrorListeners("Expected union name after 'union'"); }
| ON (GOTO | CALL)
{ notifyErrorListeners("Missing expression: expected index after 'on'"); }
;
enum : PRIVATE? PUBLIC? ENUM identifier '{' EOL? enum_member? (',' EOL? enum_member)* ','? EOL? '}' ; // you can split the values over several lines, trailing comma allowed
enum_member : identifier ('=' integerliteral)? ;
swap: 'swap' '(' assign_target ',' assign_target ')' ;
variabledeclaration :
varinitializer
| vardecl
| constdecl
| memoryvardecl
;
structdeclaration:
PRIVATE? PUBLIC? STRUCT identifier '{' EOL? (structfielddecl | EOL)+ '}'
;
uniondeclaration:
PRIVATE? PUBLIC? UNION identifier '{' EOL? (structfielddecl | EOL)+ '}'
;
structfielddecl: datatype (arrayindex arrayindex? | EMPTYARRAYSIG)? identifierlist;
// grammar allows [] and [][] so the visitor can give user-friendly error messages instead of cryptic parse errors
subroutinedeclaration :
subroutine
| asmsubroutine
| extsubroutine
;
alias: PRIVATE? PUBLIC? 'alias' identifier '=' (scoped_identifier | basedatatype | pointertype) ;
defer: 'defer' (statement | statement_block) ;
labeldef : identifier ':' ;
unconditionaljump : GOTO expression ;
directive : directivename '!'? (directivenamelist | directivearg (',' directivearg)*)? ;
directivename: '%' UNICODEDNAME;
directivenamelist: '(' EOL? scoped_identifier (',' EOL? scoped_identifier)* ','? EOL?')' ;
directivearg : stringliteral | identifier | integerliteral | expression ;
vardecl: PRIVATE? PUBLIC? datatype (arrayindex arrayindex? | EMPTYARRAYSIG)? TAG* identifierlist ;
// grammar allows [] and [][] so the visitor can give user-friendly error messages for invalid combinations
identifierlist: identifier (',' identifier)* ;
varinitializer :
vardecl '=' (tuple_expression | expression)
;
constdecl: PRIVATE? PUBLIC? 'const' datatype? identifierlist '=' expression ;
// datatype is optional in the grammar so the visitor can give "datatype missing" instead of a cryptic parse error
memoryvardecl: ADDRESS_OF varinitializer;
basedatatype: 'ubyte' | 'byte' | 'uword' | 'word' | 'long' | 'float' | 'str' | 'bool' | 'pointer' ;
datatype: pointertype | basedatatype | structtype=scoped_identifier;
// basedatatype first: 'pointer' is a reserved datatype name that no struct can use,
// so '^^pointer' must parse as pointer-to-pointer, not as pointer to a struct.
pointertype: POINTER (basedatatype | scoped_identifier);
arrayindex:
'[' expression ']' #ArrayIndexNormal
| '[' expression ',' expression ']' #ArrayIndexComma
;
// ArrayIndexComma exists in the grammar so the visitor can give a friendly error for the wrong 2D array syntax
assignment :
assign_target '=' (assignment | expression)
| multi_assign_target '=' (tuple_expression | expression)
;
augassignment :
assign_target operator=('+=' | '-=' | '/=' | '*=' | '&=' | '|=' | '^=' | '%=' | '<<=' | '>>=' ) expression
;
// Note: VOID can be used in multiple ways but a semantic AST check takes care of any mistakes there later.
assign_target:
scoped_identifier #IdentifierTarget
| arrayindexed #ArrayindexedTarget
| directmemory #MemoryTarget
| pointerdereference #PointerDereferenceTarget
| '(' expression ')' ('.' identifier)+ #ParenDerefTarget
| functioncall ('.' identifier)+ #FunctioncallDerefTarget
| arrayindexed ('.' identifier)+ #ArrayindexedDerefTarget
| VOID #VoidTarget
;
multi_assign_target:
assign_target (',' assign_target)+ ;
postincrdecr : assign_target operator = ('++' | '--') ;
expression :
'(' expression ')'
| sizeof_expression = 'sizeof' '(' sizeof_argument ')'
| functioncall
| left = expression EOL? bop = '.' EOL? right = expression // "scope traversal operator"
| <assoc=right> prefix = ('+'|'-'|'~') expression
| left = expression EOL? bop = ('*' | '/' | '%' ) EOL? right = expression
| left = expression EOL? bop = ('+' | '-' ) EOL? right = expression
| left = expression EOL? bop = ('<<' | '>>' ) EOL? right = expression
| left = expression EOL? bop = '&' EOL? right = expression
| left = expression EOL? bop = '^' EOL? right = expression
| left = expression EOL? bop = '|' EOL? right = expression
| left = expression EOL? bop = ('<' | '>' | '<=' | '>=') EOL? right = expression
| left = expression EOL? bop = ('==' | '!=') EOL? right = expression
| rangefrom = expression rto = ('to'|'downto') rangeto = expression (STEP rangestep = expression)? // can't create separate rule due to mutual left-recursion
| left = expression EOL? bop = 'in' EOL? right = expression
| left = expression EOL? bop = NOT_IN EOL? right = expression
| prefix = 'not' expression
| left = expression EOL? bop = 'and' EOL? right = expression
| left = expression EOL? bop = 'or' EOL? right = expression
| left = expression EOL? bop = 'xor' EOL? right = expression
| literalvalue
| scoped_identifier
| arrayindexed
| directmemory
| addressof
| expression typecast
| if_expression
| branchcondition_expression
| pointerdereference
| staticstructinitializer
;
tuple_expression: expression (',' EOL? expression)+ ;
sizeof_argument: basedatatype | scoped_identifier | pointertype | addressof ;
arrayindexed:
scoped_identifier arrayindex+
;
typecast : 'as' datatype;
directmemory : '@' '(' expression ')';
addressof : <assoc=right> (ADDRESS_OF | TYPED_ADDRESS_OF | ADDRESS_OF_LSB | ADDRESS_OF_MSB) scoped_identifier arrayindex? ;
functioncall : scoped_identifier '(' EOL? expression_list? EOL? ')' ;
functioncall_stmt : VOID? scoped_identifier '(' EOL? expression_list? EOL? ')' ;
expression_list :
expression (',' EOL? expression)* // you can split the expression list over several lines
;
returnstmt : 'return' returnvalues? ;
returnvalues: expression (',' expression)* ;
breakstmt : 'break';
continuestmt: 'continue';
identifier : UNICODEDNAME | UNDERSCORENAME | ON | CALL | INLINE | PRIVATE | STEP | POINTERSTR ; // due to the way antlr creates tokens, need to list the tokens here explicitly that we want to allow as identifiers too
scoped_identifier : identifier ('.' identifier)* ;
integerliteral : intpart=(DEC_INTEGER | HEX_INTEGER | BIN_INTEGER) ;
booleanliteral : 'true' | 'false' ;
arrayliteral : EMPTYARRAYSIG | '[' EOL? expression? (',' EOL? expression)* ','? EOL? ']' ; // you can split the values over several lines, trailing comma allowed
stringliteral : (encoding=UNICODEDNAME ':')? STRING ;
charliteral : (encoding=UNICODEDNAME ':')? SINGLECHAR ;
floatliteral : FLOAT_NUMBER ;
literalvalue :
integerliteral
| booleanliteral
| arrayliteral
| stringliteral
| charliteral
| floatliteral
;
inlineasm : directivename EOL? INLINEASMBLOCK; // directive name should be '%asm' or '%ir'
subroutine :
PRIVATE? PUBLIC? INLINE? 'sub' identifier '(' sub_params? ')' sub_return_part? EOL? (statement_block EOL?)
;
sub_return_part : '->' datatype (',' datatype)* ;
statement_block :
'{' EOL?
(statement | EOL) *
'}'
;
sub_params : sub_param (',' EOL? sub_param)* ;
sub_param: vardecl ('@' register=UNICODEDNAME)? ;
asmsubroutine :
PRIVATE? PUBLIC? INLINE? 'asmsub' asmsub_decl EOL? (statement_block EOL?)
;
extsubroutine :
PRIVATE? PUBLIC? 'extsub' (TAG (constbank=integerliteral | varbank=scoped_identifier))? address=expression '=' asmsub_decl
;
asmsub_signature : '(' asmsub_params? ')' asmsub_clobbers? asmsub_returns? ;
asmsub_decl : identifier asmsub_signature ;
asmsub_params : asmsub_param (',' EOL? asmsub_param)* ;
asmsub_param : vardecl '@' register=UNICODEDNAME ; // 6502: A,X,Y,AX,AY,XY,Pc,Pz,Pn,Pv | m68k: D0-D7,A0-A6,FP0-FP7 | cx16: R0-R15,R0R1-R14R15
asmsub_clobbers : 'clobbers' '(' clobber? ')' ;
clobber : UNICODEDNAME (',' UNICODEDNAME)* ; // 6502: A,X,Y | m68k: D0-D7,A0-A6,FP0-FP7
asmsub_returns : '->' asmsub_return (',' EOL? asmsub_return)* ;
asmsub_return : datatype '@' register=UNICODEDNAME ; // 6502: A,X,Y,AX,AY,XY,Pc,Pz,Pn,Pv | m68k: D0-D7,A0-A6,FP0-FP7 | cx16: R0-R15,R0R1-R14R15
if_stmt : 'if' expression EOL? (statement | statement_block) EOL? else_part? ; // statement is constrained later
else_part : ELSE EOL? (statement | statement_block) ; // statement is constrained later
if_expression : 'if' expression EOL? THEN? EOL? expression EOL? ELSE EOL? expression ;
branchcondition_expression: branchcondition THEN? expression EOL? ELSE EOL? expression ;
// This is a cursed mix of IdentifierReference (scoped identifiers) and binary expressions with '.' dereference operators.
// but it is needed for now to not have to rewrite all of Prog8's dependence on how the IdentifierReference now works (fully qualified identifier string inside)
// in the future this probably has to be reworked completely to split up the scoped identifier names and just rely on the '.' operator exclusively,
// but that also requires rewriting al name lookup code. Ah well, we'll cross that bridge when we get there.
pointerdereference: (prefix = scoped_identifier '.')? derefchain ('.' field = identifier fieldindex = arrayindex?)? ; // TODO this doesn't look pretty when dealing with the difference between a final ^^ at the end or not
derefchain : singlederef ('.' singlederef)* ;
singlederef : identifier arrayindex? POINTER ;
branch_stmt : branchcondition EOL? (statement | statement_block) EOL? else_part? ;
branchcondition: 'if_cs' | 'if_cc' | 'if_eq' | 'if_z' | 'if_ne' | 'if_nz' | 'if_pl' | 'if_pos' | 'if_mi' | 'if_neg' | 'if_vs' | 'if_vc' ;
forloop : 'for' datatype? scoped_identifier 'in' iterable=expression
(VIA '.' viaField=identifier)?
(UNTIL untilExpr=expression)?
(STEP stepExpr=expression)?
EOL? (statement | statement_block) ;
whileloop: 'while' expression EOL? (statement | statement_block) ;
untilloop: 'do' (statement | statement_block) EOL? UNTIL expression ;
repeatloop: 'repeat' expression? EOL? (statement | statement_block) ;
unrollloop: 'unroll' expression EOL? (statement | statement_block) ; // note: expression must evaluate to a constant
whenstmt: 'when' expression EOL? '{' EOL? (when_choice | EOL) * '}' EOL? ;
when_choice: (expression_list | ELSE ) '->' (statement | statement_block ) ;
// ON...GOTO/ON...CALL with optional else clause - classic BASIC-style multi-way branch.
// This is intentional retro syntax appropriate for 6502/BASIC target audience.
// The else_part handles out-of-range indices (useful for menu dispatch with error handling).
// Compiles efficiently to 6502 jump tables. NOT a design flaw - fits the retro platform.
ongoto: ON expression kind=(GOTO | CALL) directivenamelist EOL? else_part? ;
staticstructinitializer: POINTER? scoped_identifier ':' (arrayliteral | namedstructinitlist) ;
namedstructinitlist: '[' EOL? namedstructinitfield (',' EOL? namedstructinitfield)* ','? EOL? ']' ;
namedstructinitfield: identifier '=' expression ;