[RFC] lldb-dap: Update server mode to allow multiple connections

Subprojects LLDB
1

Overview

Refactoring lldb-dap server mode to allow multiple DAP connections in order to improve symbol caching to improve launch and attach performance.

Background

lldb-dap supports listening on a port for a DAP connection, however this only accepts the first connection and then the process stops. This means we cannot have an lldb-dap process reused for multiple debug sessions.

When launching or attaching to a process one of the steps that must be taken by the debugger is to load symbols for both the binary itself and any runtime linked libraries that are associated with the binary. On an M1 Max MacBook Pro this can take approximately 12s for a small “Hello World” style iOS application. This majority of the time is spent loading symbols, only around ~0.5s of the time spent was performing the attach.

By refactoring lldb-dap to have a server mode we can take advantage of the persistence of the server to cache symbols and improve debugger performance between debug sessions.

Architecture

lldb is architected to allow multiple debug sessions today. The SBDebugger instance represents a single instance of a debug session. In order to support multiple debug sessions on the same server we will need to refactor lldb-dap to not have a single global lldb_dap::DAP g_dap object and by extension SBDebugger instance. Instead, when a connection is made to the server we should create a new lldb_dap::DAP instance that is associated with that specific connection.

A number of helpers and utilities will need to either be redesigned or perhaps moved to accommodate the removal of the g_dap instance.

lldb/tools/lldb-dap/JSONUtils.{h,cpp} - In general these should be abstracted to have parameters for any fields the access from the g_dap instance.

lldb/tools/lldb-dap/LLDBUtils.{h,cpp} - This set of functions depends on the lldb_dap::DAP instance to execute an lldb operations. In general, these functions should take a lldb_dap::DAP &dap parameter.

lldb/tools/lldb-dap/DAP.{h,cpp} - The lldb_dap::DAP object should be updated to ensure it is destructible and correctly cleans up any open file handles it creates while running.

lldb/tools/lldb-dap/lldb-dap.cpp - Request handles should take a lldb_dap::DAP& value as a parameter, so they can be specific to correct connection.

Threading model

  • The main thread is listening and accepting connections
  • Each connection will have a thread for the lldb_dap::DAP::loop(), a thread for EventThreadFunction and a thread for ProgressEventThreadFunction.

Process Management

With this change, we should be able to start the lldb-dap in a server mode and use the same instance until VS Code is closed. We could add a command to stop the running process, since it can end using a lot of memory for the symbol caches.

1 Like
2

FWIW I just landed a patch to parallelize modules loading for the darwin dynamic loader. So loading should be about 3-4 times faster now for iOS apps.

3

All in all, this sounds like a good idea. It would be great to avoid the potentially large initial loading time for symbols.

I know next to nothing about LLDB’s symbol caches. How is this symbol cache currently managed?

E.g., my usual debugging flow is:

  1. build my_binary
  2. debug it, find a bug, fix the bug
  3. rebuild
  4. restart program in the debugger

During (3), the debug symbols of my_binary are (at least partially) rewritten by the compiler. Usually some libraries / .dwo files of objects which I didn’t touch stay unchanged. Other .dwo files are replaced by the recompilation.

Would lldb evict the no longer existing, outdated .dwo files from its symbol cache? Or would the symbol cache fill up with a lot of outdated .dwo files?

4

Oh, and we should of course also stop and restart the server if the lldb-dap binary itself changed. Otherwise, working on lldb-dap itself would become more cumbersome

5

When a Target is created with the main binary that will be run (Target::SetExecutableModule), lldb looks through the executables dependent libraries, creates Modules for all of those, then looks through them for new dependent libraries and loads all of those. From the problem description, I think this is the codepath that they’re spending time in, and it won’t be improved by your PR.

The behavior here (add a module, get its dependents and try to add all of them) is different enough from how DynamicLoaderDarwin works that trying to centralize this multi-threaded approach is probably going to make the unified multi-threader method a lot more complicated; I believe the solution is to have Target::SetExecutableModule create a ThreadPool and create the modules in parallel too.

Your patch was focused on “attach time”, where we are attaching to an already-running process, and this API wasn’t hit. But anyone who does lldb foo; run will see SetExecutableModule as the bottleneck as lldb pre-loads all of the Modules before execution begins.

1 Like
6

I believe it will be cleared correctly, @jingham was looking at this a while ago. See Add a test for evicting unreachable modules from the global module cache by jimingham · Pull Request #74894 · llvm/llvm-project · GitHub

7

It didn’t occur to me that you can run an iOS app from LLDB. Hopefully, I’ll find some time to parallelize Target::SetExecutableModule as well.

8

oh wait, my bad, I didn’t read the problem description closely enough, you’re right. It is possible to give lldb the executable binary when you create the Target before attaching, and you’ll hit the SetExecutableModule bottleneck, but you’re right, if this debug scenario is simply attaching to a process that’s been launched, your patch will definitely apply.

9

Yeah, but I think your point still stands as we can run iOS apps in macOS since M1 so we can probably run them from the debugger as well. I didn’t test it but if it’s true - target create is probably a valid scenario here.

10

Here’s a debugging scenario - I’m debugging an ARM Android application, and a Hexagon DSP application that the Android app spins off. I debug them in the same vscode instance.

Currently, I hit run on the Android program, then jump through some hoops to attach to the DSP program. In both cases, vscode will launch the same lldb-dap, so there are 2 instances of lldb-dap running.

With your proposal, will lldb-dap only be launched the first time it’s needed, and the existing one then used by vscode on subsequent launches/attaches?

11

We could check that, or have an option in the settings to control whether lldb-dap is launched as an executable or in server mode.

I’d only want the server to restart if it was idle, so we’d also need to track which debug sessions are currently active on the executable in order to know when it can be safely restarted.

Yes, that should work with my proposed changes.

12

If you start lldb in the cli then rebuild the target it will intelligently reload modules the next time the target is run.

As far as evicting, we can have the server mode listen for memory pressure events and trigger lldb::SBDebugger::MemoryPressureDetected() on supported platforms.

13

I’ve got a working prototype.

So far this only covers getting lldb-dap to compile after doing the refactors I mentioned above. This still needs some work on tests and on the vscode-extension side of things as well.

14

I’ve got all the tests passing, plus some new ones for validating the server connection in this pull request.