Debugger · FlingOS/FlingOS Wiki · GitHub
Skip to content
This repository was archived by the owner on Mar 24, 2024. It is now read-only.

Debugger

Edward Nutting edited this page Jan 27, 2017 · 2 revisions

Setup

First, compile a debug version of the kernel for x86 (debugging other architectures is not supported yet). In your VM, you'll need to set a debuggable virtual machine. This means adding the correct COM (serial) ports to the VM. You will require:

  1. COM1 - Direct this to a file. It contains the screen output of the kernel for later reference (but isn't directly used by the debugger)
  2. COM2 - Direct this to a named pipe: \.\pipe\FlingOSDebug_Msg - It should be a client with the other end as an application. This is used for communication (synchronous message passing) between the kernel's debugger thread and the host debugger application.
  3. COM3 - Direct this to a named pipe: \.\pipe\FlingOSDebug_Notif - It should be a client with the other end as an application. This is used for asynchronous notifications to the debugger. It uses the notification 0xFE to tell the debugger a breakpoint has been hit.

Starting the debugger and VM

At this point, you could connect to serial (COM) port 2 using any valid terminal app on the host. For example, Putty using baud 96200, RTS/CTS and "implicit CR with every LF" selected. However, on Windows, we provide a GUI application for debugging programs which is much more effective.

Build and start the GUI debugger. The root of the named pipe should be "FlingOSDebug", the "_Msg" and "Notif" are appended automatically. The assembly name should be Kernel for debugging the main FlingOS kernel. The Bin path should be the full path to the build directory; You will need to change this according to your setup.

Press the Connect button and then start the virtual machine. If all goes well, the VM should connect the ports and the OS should start executing. Once the OS has loaded and the debugger thread has started, the host debugger (GUI) should detect the connection and allow you to proceed.

Using the debugger

Take a look at the overview video:

YouTube: Overview of GUI Debugger

To start with, press the Refresh button to retrieve the list of processes and threads. Selecting a process or thread allows you to suspend it and inspect its state. The rest should be reasonably obvious.

To add breakpoints, type at least 10 characters in the Filter box of the name of the method you wish to add a breakpoint for. Select the available debug point (if any) and click "Set". Suspend the thread you wish to use the breakpoint with and then Step the thread until it reaches the breakpoint.

Clone this wiki locally