Robert's suggestion is obviously the most direct and correct route if you can change the IDL of generated type. After all, code generation is there to simplify generating alot of code from a little definition.
I was assuming that one of constraints was not being able to modify the struct received. You should take that simpler route, if at all possible. > If contention is indeed a problem I can mitigate by sharding the map You are in luck. sync.Map already shards for you. It is very well engineered, and in my benchmarks, you will be very hard pressed to do better. Less than 50ns for 10% writes, and 12ns for 100% reads. Its documentation does it a bit of a disservice by discouraging its wider use, but it is awesome; one of the reasons that net/rpc itself was so performant. There are some studies here (with benchmark code you can run yourself to reproduce it: https://github.com/glycerine/uart), the most salient part is this benchmark, going from 100% writes at the top, to 100% reads at the bottom, in steps of 10%. BenchmarkReadWriteSyncMap BenchmarkReadWriteSyncMap/frac_0 BenchmarkReadWriteSyncMap/frac_0-8 11585511 128.9 ns/op 111 B/op 5 allocs/op BenchmarkReadWriteSyncMap/frac_1 BenchmarkReadWriteSyncMap/frac_1-8 12977352 131.1 ns/op 101 B/op 4 allocs/op BenchmarkReadWriteSyncMap/frac_2 BenchmarkReadWriteSyncMap/frac_2-8 14838945 122.2 ns/op 90 B/op 4 allocs/op BenchmarkReadWriteSyncMap/frac_3 BenchmarkReadWriteSyncMap/frac_3-8 14907528 114.5 ns/op 80 B/op 3 allocs/op BenchmarkReadWriteSyncMap/frac_4 BenchmarkReadWriteSyncMap/frac_4-8 18078240 100.8 ns/op 70 B/op 3 allocs/op BenchmarkReadWriteSyncMap/frac_5 BenchmarkReadWriteSyncMap/frac_5-8 18791480 89.52 ns/op 59 B/op 3 allocs/op BenchmarkReadWriteSyncMap/frac_6 BenchmarkReadWriteSyncMap/frac_6-8 21172922 79.55 ns/op 49 B/op 2 allocs/op BenchmarkReadWriteSyncMap/frac_7 BenchmarkReadWriteSyncMap/frac_7-8 25711459 73.04 ns/op 38 B/op 2 allocs/op BenchmarkReadWriteSyncMap/frac_8 BenchmarkReadWriteSyncMap/frac_8-8 29925382 60.51 ns/op 28 B/op 1 allocs/op BenchmarkReadWriteSyncMap/frac_9 BenchmarkReadWriteSyncMap/frac_9-8 41599728 46.34 ns/op 18 B/op 1 allocs/op BenchmarkReadWriteSyncMap/frac_10 BenchmarkReadWriteSyncMap/frac_10-8 100000000 11.96 ns/op 8 B/op 1 allocs/op On Monday, May 12, 2025 at 5:34:08 PM UTC+1 Robert Engels wrote: > Why not just modify the code generator to add a user data pointer to the > structure? > > I can’t imagine something like that isn’t already there? Either the code > generator should be easy to enhance or it should have a side car data field > - anything else is hard to rationalize as even rudimentary design. > > On May 12, 2025, at 11:13 AM, Alexander Shopov <a...@kambanaria.org> > wrote: > > > > Now that I think about it again - I guess your suggestion could work for > my case with some caveats: > > 1. There can be contention on the map - but as you point out there is > sync.Map as well > 2. The state should be removed from the map but that can also be arranged > by deferring a delete operation > 3. If contention is indeed a problem I can mitigate by sharding the map > 4. If contention continues to be a problem perhaps weak pointers with this > recipe https://github.com/golang/go/issues/67552#issuecomment-2195479919 may > help. > > It won't be pretty but hey - help beggars can't be choosers! > > Kind regards: > al_shopov > > На пн, 12.05.2025 г. в 17:22 Jason E. Aten <j.e....@gmail.com> написа: > >> Forgive me if this off base, as I'm still a little fuzzy on the exact >> constraints >> of you problem... but, as stated, if you want >> to associate additional optional behavior and state with any given >> response >> that is constrained to the generated, just use >> the responses's pointer (to its struct) (if local an ephemeral is all >> you need, or a cryptographic hash of >> its contents, if you need to survive reboots) as the key into a separate >> map/sync.Map/ >> database. Standard Go does not move pointers; the GC is a non-moving >> collector >> by intent to enable sane interfacing with C code, and does very well at >> avoiding >> fragmentation (the motivation for compaction/copying GC) by using >> size classes instead. Since you can rely on the pointer not moving, just >> use it as a key to a map[*generated.Response]*EnrichedResponse to >> lookup your additional state on either the caller or responder side. >> >> On Monday, May 12, 2025 at 1:12:25 PM UTC+1 Alexander Shopov wrote: >> >>> seems a very common assumption >>> >>> >>> I would have guessed it is due to compatibility with C but >>> https://pkg.go.dev/cmd/cgo#hdr-C_references_to_Go explicitly states: >>> *Go struct types are not supported; use a C struct type. * >>> So it may be completely at the decision of the implementer. >>> The caveats in https://go.dev/wiki/cgo#common-pitfalls also sound too >>> tricky. >>> >>> The pointer to the entire value stays alive while any pointer to one of >>> its fields is alive >>> >>> That requires some cooperation from the GC. In the case where structs >>> are reasonably small and they contain values rather than pointers this >>> would probably hold. >>> >>> Thanx for even further details (I am writing down the offset trick) but >>> I'd prefer to not stray too far away from usual conventions. >>> >>> Kind regards: >>> al_shopov >>> >>> >>> На пн, 12.05.2025 г. в 12:14 Axel Wagner <axel.wa...@googlemail.com> >>> написа: >>> >>>> >>>> >>>> On Mon, 12 May 2025 at 11:12, Alexander Shopov <a...@kambanaria.org> >>>> wrote: >>>> >>>>> Is the equality of pointer to struct and pointer to its first member >>>>> actually always guaranteed by Go lang? >>>>> https://go.dev/ref/spec#Package_unsafe says The effect of converting >>>>> between Pointer and uintptr is implementation-defined but I guess it >>>>> is not the same thing. The sections on Struct types and Alignment do not >>>>> give such guarantees. >>>>> >>>> >>>> Perhaps not, though it seems a very common assumption and if an >>>> implementation violates it, I would expect that to break a bunch of code. >>>> You can always make sure, by accounting for a potential offset: >>>> https://go.dev/play/p/whdFALRG20P >>>> I guess that still makes the assumption, that a pointer to the entire >>>> value stays alive while any pointer to one of its field is alive. I'm not >>>> sure *that* is guaranteed either, but at least it's a weaker assumption. >>>> >>>> >>>>> In the end it is not wise to use such a technique on a large code base >>>>> shared by many developers. I will not be going this way but many thanx >>>>> for >>>>> the example it taught me something I did not know. >>>>> >>>>> Axel, you may use it for a 2nd part of Advanced Generics Patterns >>>>> (nice presentation! I liked it) - Advanced Type Patterns (use types >>>>> wrongly >>>>> so we can learn how to make the wrong way the right way). >>>>> >>>>> >>>>> I was hoping that there is a trick that I am missing. Like - a >>>>> returned function can access state in a closure but I see no way to do it >>>>> for methods. I definitely do not want to go the direction - since I want >>>>> to >>>>> add a method I should change the code generation, types (struct vs >>>>> interface-s) to suit that small change. >>>>> >>>>> Kind regards: >>>>> al_shopov >>>>> >>>>> На пн, 12.05.2025 г. в 7:12 Axel Wagner <axel.wa...@googlemail.com> >>>>> написа: >>>>> >>>>>> On Mon, 12 May 2025 at 07:05, Axel Wagner <axel.wa...@googlemail.com> >>>>>> wrote: >>>>>> >>>>>>> A less evil, but still bad way, would be to store it in a global >>>>>>> map: https://go.dev/play/p/ZHBJVCduf25 (note: I'm not sure this use >>>>>>> of weak pointers is actually correct, I haven't used them yet). >>>>>>> >>>>>> >>>>>> Okay, I'm pretty sure it's not correct. It should be possible to make >>>>>> this work somehow, but I'm a bit too lazy to try right now, especially >>>>>> as >>>>>> you shouldn't do it anyways. >>>>>> >>>>>> >>>>>>> Otherwise, no. The state has to live somewhere. And really, both of >>>>>>> these tricks are still really bad and you shouldn't do them. The best >>>>>>> way >>>>>>> is to respect the type system and if you want extra methods, you need a >>>>>>> new >>>>>>> type and if that needs extra state, that should be stored in the type. >>>>>>> >>>>>>> ----- >>>>>>>> Why would I want to do this? What am I trying to achieve? >>>>>>>> >>>>>>>> Basically there is a lot of generated code and I want to keep >>>>>>>> compatibility with it. >>>>>>>> Similar to the way Go embeds wider interfaces into narrower ones I >>>>>>>> want to be able to add methods to the generated code without having to >>>>>>>> change it. >>>>>>> >>>>>>> >>>>>>> Change the code generator, and/or add additional code in an extra >>>>>>> file. Hacking the language will cause you more pain, in the long run. >>>>>>> >>>>>>> >>>>>>>> Then - whenever some code calls the *Get* method on the server - >>>>>>>> based on the whether the returned value implements the *Enriched* >>>>>>>> interface or not and the value it returns - I can dispatch behavior. >>>>>>>> >>>>>>>> Kind regards: >>>>>>>> al_shopov >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "golang-nuts" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to golang-nuts...@googlegroups.com. >>>>>>>> To view this discussion visit >>>>>>>> https://groups.google.com/d/msgid/golang-nuts/bbe6bcd8-e33c-41bf-868a-e498561c3e72n%40googlegroups.com >>>>>>>> >>>>>>>> <https://groups.google.com/d/msgid/golang-nuts/bbe6bcd8-e33c-41bf-868a-e498561c3e72n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "golang-nuts" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/golang-nuts/5gx3CCMfyJg/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> golang-nuts...@googlegroups.com. >> To view this discussion visit >> https://groups.google.com/d/msgid/golang-nuts/e823a08d-11c3-49d0-bd24-89eef57efc0an%40googlegroups.com >> >> <https://groups.google.com/d/msgid/golang-nuts/e823a08d-11c3-49d0-bd24-89eef57efc0an%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts...@googlegroups.com. > > To view this discussion visit > https://groups.google.com/d/msgid/golang-nuts/CAP6f5MmNXt-AuUPA4XGc1ZbbYqaMibtdzcxikx04KWNUpy21nA%40mail.gmail.com > > <https://groups.google.com/d/msgid/golang-nuts/CAP6f5MmNXt-AuUPA4XGc1ZbbYqaMibtdzcxikx04KWNUpy21nA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/eeb31b6d-6253-4633-9c9b-006ef2df44e1n%40googlegroups.com.
