Hi, I'm trying to decide between Racket and Go on writing my onion-routing system inspired by Tor. Basically, a network server, involving lots of long-lived connections that often pass large amounts of data. The thing needs to be super scalable; I often find that these servers, although network servers, often become CPU-bound doing encryption and encapsulation of protocols, so I do have experience that this is not "premature optimization"!
Go seems to be the go-to (pun intended) for scalable network things. It
has super cheap threads, is statically compiled (easy to deploy to lots
of machines), and apparently uses little memory and runs fast. However,
after writing some low-level transports in Go, I found a few things I
gripe about but seem to be unsolvable:
- Go doesn't have exceptions. This means checking errors over and over.
- Go uses lightweight user threads like Racket ("goroutines"), but
isn't fully non-blocking under the hood for I/O. Thus some I/O
operations actually spawn a kernel thread, and then kill it when the
operation is done. This is unacceptable for my use case.
- Go isn't as fast as I imagined. My program is still CPU-bound at
around 40 MB/s, and that's just counting pushing zeroes through the
lowest-level transport protocol.
- Go's garbage collector is naive stop-the-world mark-and-sweep. My
program keeps a large amount of state, and constantly pushes data and
accepts connections. The mark-and-sweep process can take as much as 20
seconds on large heaps, severly impacting user experience. Since the GC
is nonmoving, heap fragmentation is also an issue. I searched around a
bit, and it seems that "buffer reuse" is a thing in Go. Ugh.
So I decided to take Racket, namely Typed Racket, a spin. I like it a
lot, it's my favorite language after all, and especially I like the fact
that I can use macros to simplify verbose patterns of code. DrRacket's
REPL is also very helpful. I also find that Racket is much more
performant than I imagined (I used to think of it as a Python-type slow
scripting language). However, there are a few gripes I have, but seeing
how flexible Racket is, I think they might be solvable:
- Racket threads don't seem to be able to utilize multiple CPU cores.
Is there an idiomatic way to, say, form a pool of places and push
threads to them? I imagine sharing data is very messy?
- The Racket VM is huuuuuuge, both on disk and in memory.
Infuriatingly, using #lang racket seems to import every single #lang
racket library into the memory space, and when I raco dist my program,
every single library gets packaged. I know this sounds like whining, but
why can't some sort of analysis phase be applied where only the
libraries actually required for the program to run are loaded?
- Racket doesn't seem to be able to call raw C code or machine code in
static libraries, instead requiring the code be compiled into a library.
Is this related to the fact that Racket is run in a VM rather than
compiled to machine code?
- Some things in racket are pathologically slow. As an example, try
implementing a cipher with loops and array indices and bytestrings. It
will end up orders of magnitude slower than, say, C or Go or Java, or
sometimes even Python.
Are there solutions to these problems? These aren't showstoppers by any
means, but could finally end my endless dilemma between the two langs :)
Thanks!
Yuhao Dong
signature.asc
Description: This is a digitally signed message part
____________________ Racket Users list: http://lists.racket-lang.org/users
