On May 12, 2025, at 12:48 PM, Jason E. Aten <j.e.a...@gmail.com> wrote:
Robert's suggestion is obviously the most direct and correct route if you can change the IDL of generated type. After all, code generation is there to simplify generating alot of code from a little definition.
I was assuming that one of constraints was not being able to modify the struct received. You should take that simpler route, if at all possible.
> If contention is indeed a problem I can mitigate by sharding the map
You are in luck. sync.Map already shards for you. It is very well engineered, and in my benchmarks, you will be very hard pressed to do better. Less than 50ns for 10% writes, and 12ns for 100% reads. Its documentation does it a bit of a disservice by discouraging its wider use, but it is awesome; one of the reasons that net/rpc itself was so performant. There are some studies here (with benchmark code you can run yourself to reproduce it: https://github.com/glycerine/uart), the most salient part is this benchmark, going from 100% writes at the top, to 100% reads at the bottom, in steps of 10%.
BenchmarkReadWriteSyncMap
BenchmarkReadWriteSyncMap/frac_0
BenchmarkReadWriteSyncMap/frac_0-8 11585511 128.9 ns/op 111 B/op 5 allocs/op
BenchmarkReadWriteSyncMap/frac_1
BenchmarkReadWriteSyncMap/frac_1-8 12977352 131.1 ns/op 101 B/op 4 allocs/op
BenchmarkReadWriteSyncMap/frac_2
BenchmarkReadWriteSyncMap/frac_2-8 14838945 122.2 ns/op 90 B/op 4 allocs/op
BenchmarkReadWriteSyncMap/frac_3
BenchmarkReadWriteSyncMap/frac_3-8 14907528 114.5 ns/op 80 B/op 3 allocs/op
BenchmarkReadWriteSyncMap/frac_4
BenchmarkReadWriteSyncMap/frac_4-8 18078240 100.8 ns/op 70 B/op 3 allocs/op
BenchmarkReadWriteSyncMap/frac_5
BenchmarkReadWriteSyncMap/frac_5-8 18791480 89.52 ns/op 59 B/op 3 allocs/op
BenchmarkReadWriteSyncMap/frac_6
BenchmarkReadWriteSyncMap/frac_6-8 21172922 79.55 ns/op 49 B/op 2 allocs/op
BenchmarkReadWriteSyncMap/frac_7
BenchmarkReadWriteSyncMap/frac_7-8 25711459 73.04 ns/op 38 B/op 2 allocs/op
BenchmarkReadWriteSyncMap/frac_8
BenchmarkReadWriteSyncMap/frac_8-8 29925382 60.51 ns/op 28 B/op 1 allocs/op
BenchmarkReadWriteSyncMap/frac_9
BenchmarkReadWriteSyncMap/frac_9-8 41599728 46.34 ns/op 18 B/op 1 allocs/op
BenchmarkReadWriteSyncMap/frac_10
BenchmarkReadWriteSyncMap/frac_10-8 100000000 11.96 ns/op 8 B/op 1 allocs/op
On Monday, May 12, 2025 at 5:34:08 PM UTC+1 Robert Engels wrote:
Why not just modify the code generator to add a user data pointer to the structure?
I can’t imagine something like that isn’t already there? Either the code generator should be easy to enhance or it should have a side car data field - anything else is hard to rationalize as even rudimentary design.
Now that I think about it again - I guess your suggestion could work for my case with some caveats:
1. There can be contention on the map - but as you point out there is sync.Map as well 2. The state should be removed from the map but that can also be arranged by deferring a delete operation 3. If contention is indeed a problem I can mitigate by sharding the map
It won't be pretty but hey - help beggars can't be choosers!
Kind regards: al_shopov Forgive me if this off base, as I'm still a little fuzzy on the exact constraintsof you problem... but, as stated, if you want to associate additional optional behavior and state with any given response that is constrained to the generated, just use the responses's pointer (to its struct) (if local an ephemeral is all you need, or a cryptographic hash of its contents, if you need to survive reboots) as the key into a separate map/sync.Map/ database. Standard Go does not move pointers; the GC is a non-moving collector by intent to enable sane interfacing with C code, and does very well at avoiding fragmentation (the motivation for compaction/copying GC) by using size classes instead. Since you can rely on the pointer not moving, just use it as a key to a map[*generated.Response]*EnrichedResponse to lookup your additional state on either the caller or responder side.
On Monday, May 12, 2025 at 1:12:25 PM UTC+1 Alexander Shopov wrote:
seems a very common assumption
Go struct types are not supported; use a C struct type. So it may be completely at the decision of the implementer.
The pointer to the entire value stays alive while any pointer to one of its fields is alive That requires some cooperation from the GC. In the case where structs are reasonably small and they contain values rather than pointers this would probably hold.
Thanx for even further details (I am writing down the offset trick) but I'd prefer to not stray too far away from usual conventions.
Kind regards:
al_shopov
Is the equality of pointer to struct and pointer to its first member actually always guaranteed by Go lang? https://go.dev/ref/spec#Package_unsafe says The effect of converting between Pointer and uintptr is implementation-defined but I guess it is not the same thing. The sections on Struct types and Alignment do not give such guarantees.
Perhaps not, though it seems a very common assumption and if an implementation violates it, I would expect that to break a bunch of code. I guess that still makes the assumption, that a pointer to the entire value stays alive while any pointer to one of its field is alive. I'm not sure *that* is guaranteed either, but at least it's a weaker assumption.
In the end it is not wise to use such a technique on a large code base shared by many developers. I will not be going this way but many thanx for the example it taught me something I did not know.
Axel, you may use it for a 2nd part of Advanced Generics Patterns (nice presentation! I liked it) - Advanced Type Patterns (use types wrongly so we can learn how to make the wrong way the right way).
I was hoping that there is a trick that I am missing. Like - a returned function can access state in a closure but I see no way to do it for methods. I definitely do not want to go the direction - since I want to add a method I should change the code generation, types (struct vs interface-s) to suit that small change.
Kind regards: al_shopov
A less evil, but still bad way, would be to store it in a global map: https://go.dev/play/p/ZHBJVCduf25 (note: I'm not sure this use of weak pointers is actually correct, I haven't used them yet).
Okay, I'm pretty sure it's not correct. It should be possible to make this work somehow, but I'm a bit too lazy to try right now, especially as you shouldn't do it anyways.
Otherwise, no. The state has to live somewhere. And really, both of these tricks are still really bad and you shouldn't do them. The best way is to respect the type system and if you want extra methods, you need a new type and if that needs extra state, that should be stored in the type.
-----
Why would I want to do this? What am I trying to achieve?
Basically there is a lot of generated code and I want to keep compatibility with it.
Similar to the way Go embeds wider interfaces into narrower ones I want to be able to add methods to the generated code without having to change it.
Change the code generator, and/or add additional code in an extra file. Hacking the language will cause you more pain, in the long run.
Then - whenever some code calls the Get method on the server - based on the whether the returned value implements the Enriched interface or not and the value it returns - I can dispatch behavior.
Kind regards: al_shopov
--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/bbe6bcd8-e33c-41bf-868a-e498561c3e72n%40googlegroups.com.
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/5gx3CCMfyJg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/e823a08d-11c3-49d0-bd24-89eef57efc0an%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/eeb31b6d-6253-4633-9c9b-006ef2df44e1n%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/EB99A018-2F4E-45A8-BD22-23E7D4EDA8F6%40ix.netcom.com.
|
