On Friday, 2 June 2017 at 20:47:31 UTC, Mike B Johnson wrote:
Would you mind, when you get some time, to write up a more
detailed analysis of the problems you had to overcome to get it
to work? Possibly we could get some type of library solution
that just "works" with very little change and restriction?
For sure. I actually want to post the source code at some point,
but the changes I made are very much set up specifically for this
project. I'll sift through at some point and see if I can gather
together something worth posting.
The main problem with hot-reloading DLLs stems from the fact that
every new D DLL comes with its own copy of druntime. This means a
second GC, a second registry of active threads, a second registry
of typeinfos, and so on. If there were a way to load a D DLL that
automatically used the executable's copy of druntime, most of the
hacks I had to make could be removed.
For reference, my general approach is that I have an EXE and a
DLL. We run the EXE, which makes a copy of the DLL and loads the
copy. Once per frame, the EXE calls GameUpdateAndRender() on the
DLL, which does everything needed to step the game forward one
frame. After each call to GameUpdateAndRender(), the EXE checks
to see if the original DLL has changed. If so, we unload the DLL,
make a copy of the new one, and load the copy. We then continue
calling GameUpdateAndRender() as usual. All game state that needs
to survive a hot reload is stored directly (or referenced
indirectly) from a giant struct call the GameState. The EXE keeps
a void* to the GameState, that way the GC doesn't collect
anything inside it.
The main issues I remember off the top of my head:
- I have to use the GC proxy to redirect DLL GC calls over to the
EXE's GC. This is because when I hot reload, the DLL's GC dies,
so it can't be holding on to anything that needs to survive the
hot reload (and I want pretty much everything to survive a hot
reload).
- D has some built-in support for redirecting GC calls. However
when you load a DLL, by the time you get a chance to call
gc_setProxy(), it's too late, and druntime has already done a few
allocations into the DLL GC. I had to change the order of a
couple parts of initialization (in runtime.d I think?), and in my
own code defer calling dll_attach_process() until after
DllMain(). That way I get a chance to set up the GC proxy, then
initialize druntime, which will trigger a few allocations that
now go to the right place.
- I haven't experimented thoroughly with threading, but I think I
have to do something similar. Since we're using the EXE's GC, if
the EXE's druntime doesn't know about the existence of a thread
spawned through the DLL's druntime, then we run into problems. So
I redirect all thread spawning over to the EXE.
- With threading, we have to be careful that we don't tear down
the DLL from underneath a thread that might still be running code
on it. We have two options. 1. Join all threads before unloading
the DLL.
2. The thread needs to be running a procedure that originates in
the EXE, and then calls into the DLL, returning periodically back
to the EXE. While execution is back in the EXE, we take that
moment to reload the DLL.
Currently I'm doing the former, but might move to the latter if I
end up needing long-lived threads for performance reasons. Note
that callbacks originating from C code running on a separate
thread also have this same restriction, and need to be handled
careful.
- The different druntimes have different typeinfos for each
class. This can be problematic for a few reasons, and I don't
have as total a grasp on what situations related to classes are
guaranteed to cause a crash. In particular, finalizers seem to
cause problems, so I have disabled them entirely in druntime. In
my own code, I almost exclusively use structs, and any instances
of classes I do use are never stored long-term, and are never
allowed to survive a hot reload.
I think those are all the D-specific gotchas, but I'll
double-check my notes when I get home. In addition, there are
common gotchas to this approach that would exist in other
languages (C/C++ for example). Examples:
- Static variables will die in a hot reload, so be cautious using
them.
- Storing a char* or string that references string in the data
segment of the DLL will cause problems when we hot reload. To
avoid this, I have a templated function that, on hot reload,
iterates over my entire gamestate and copies each string. Since
hot reloading is a debug-only utility for me, this is acceptable.
- Code changes to the gamestate struct will require a restart.
Adding a new field to the gamestate does not, since I have some
padding room at the end of the struct where I can add new fields
during a hot reload.
- Don't hold on to function pointers through a hot reload, as
functions addresses are liable to be reshuffled in the new DLL.
- Probably some others I'm forgetting.
In terms of making a library solution, I'm not sure what the best
approach would be. But allowing users to compile a "light" D DLL
that auto-patches itself back to the calling EXE's druntime would
drastically reduce the number of hacks needed to make this work.
So I guess that would be the main hurdle to overcome?
