How to add breakpoint and debug app in Basilisk II

Motivation

Scenario 1: I want to patch the Macintosh ROM with new virtual hardware. How can I find the right spot in the ROM and debug the M68k assembly code I wrote? Scenario 2: My app crashed in System 6 or Mac OS 7. I want to fix it. How can I stop the execution of the emulation and patch it manually?

Solution

cxmon is your best hope. I added a breakpoint feature to cxmon and developed an M68k app for System 6 and Mac OS7, which suspends the emulation at the runtime.

How

First, compile BII with cxmon support. Below is a sample build by adding --with-mon

make clean
NO_CONFIGURE=1 ./autogen.sh
./configure CFLAGS="-O3" CXXFLAGS="-O3" --enable-sdl-video --enable-sdl-audio --disable-vosf --disable-jit-compiler --with-x --with-gtk --with-mon --with-sdl2
make -j 32

Secondly, copy /cxmon/utils/suspend.bin to the guest OS and unpack it with the binpack app. Run suspend app. It will suspend the emulation and enter into cxmon god mode.

The commands that start with b are related to breakpoint features. The nice thing about loading and saving breakpoints to a file is that BII will load those saved breakpoints from the file automatically when the emulation restarts. It is handy for debugging Mac ROM patches.

[0000000000000000]-> h
x                        Quit mon
h                        This help text
??                       Show list of commands
ver                      Show version
? expression             Calculate expression
@ [size]                 Reallocate buffer
i [start [end]]          ASCII memory dump
m [start [end]]          Hex/ASCII memory dump
b [start [end]]          Binary memory dump
ba [address]             Add a break point
br [breakpoints#]        Remove a break point. If # is 0, remove all break points.
bd [breakpoints#]        Disable a break point. If # is 0, disable all break points.
be [breakpoints#]        Enable a break point. If # is 0, enable all break points.
bi                       List all break points
bs "file"                Save all break points to a file
bl "file"                Load break points from a file
d [start [end]]          Disassemble PowerPC code
d65 [start [end]]        Disassemble 6502 code
d68 [start [end]]        Disassemble 680x0 code
d80 [start [end]]        Disassemble Z80 code
d86 [start [end]]        Disassemble 80x86 (32-bit) code
d8086 [start [end]]      Disassemble 80x86 (16-bit) code
d8664 [start [end]]      Disassemble x86-64 code
: start string           Modify memory
f start end string       Fill memory
y[b|h|w] start end expr  Apply expression to memory
t start end dest         Transfer memory
c start end dest         Compare memory
h start end string       Search for byte string
\ "command"              Execute shell command
ls [args]                List directory contents
rm [args]                Remove file(s)
cp [args]                Copy file(s)
mv [args]                Move file(s)
cd directory             Change current directory
o ["file"]               Redirect output
[ start "file"           Load data from file
] start size "file"      Save data to file
set [var[=value]]        Set/clear/show variables
cv                       Clear all variables
regs                    Dump m68k emulator registers