the comment was for libraries without context parameters but which would need to have them added.
On Sun, May 14, 2017 at 11:46 AM, Sameer Ajmani <sam...@golang.org> wrote: > Specifically: don't pass nil Contexts. They are not valid, and most code > that uses Contexts don't check for nil and will panic. > > On Sun, May 14, 2017 at 11:43 AM Sameer Ajmani <sam...@golang.org> wrote: > >> Generally I'd suggest passing context.Background() when calling functions >> that need a context from main or tests. >> On Sat, May 13, 2017 at 8:27 AM Peter Weinberger (温博格) <p...@google.com> >> wrote: >> >>> Hi. I was one of the people who failed in an attempt to auto-insert >>> contexts in all of google3. I no longer remember all the obstacles I failed >>> to overcome, but would encourage others to take on the project. >>> >>> One issue was libraries that were used both in paths from http requests >>> (so they needed the context propagated through them) but were also used in >>> places where their callers (starting at main) didn't have any contexts at >>> all. Sitting by my coffee this morning this no longer seems like an >>> insuperable obstacle (e.g., pass nils and change them to >>> context.Background() inside, or, like appengine's log, split packages into >>> one that requires contexts and one that doesn't). But I had lots of what I >>> thought were plausible ideas that broke on the hard rock of google3, and I >>> suspect these would too. >>> >>> (The project wasn't a total failure. It did plumb contexts through a >>> bunch of packages. But the general case was too hard for me.) >>> >>> On Sat, May 13, 2017 at 3:50 AM, <mhhc...@gmail.com> wrote: >>> >>>> A reference i found about "reactive programming" >>>> https://github.com/dsyer/reactive-notes/blob/master/intro.adoc >>>> >>>> Hope this introduced the concept correctly, thanks for pointing that. >>>> >>>> Following are only some thoughts, >>>> >>>> Two things surprising here, >>>> 1/ a programmer prefers not to solve a problem >>>> 2/ one failed attempt mean the end of all attempts** >>>> >>>> That being said, my little understanding so far is that current context >>>> is used for two purposes, >>>> - cancellation >>>> - data (transmission? not sure) (+/- ts, have not checked i expect so >>>> in regards to previous referenceS cited) >>>> >>>> While cancellation is a feature i really want to take advantage of, >>>> the other one is much more arguable in the sense that >>>> it can remains expressed purely by the end user without bloating >>>> his productivity and quality (you don t really need it everywhere, just >>>> where it is needed), >>>> it seems to be highly software context dependent. >>>> >>>> Whereas cancellation, while it looks likes simple, is maybe more subtle. >>>> After all it is about assigning uuid to a chain of call and >>>> appropriately propagate, disjoint/rejoin new uuids with the previous >>>> one, >>>> so that we can ask to stop execution of a sub selection of a chain of >>>> calls >>>> via an handle. >>>> Am i too simplistic? >>>> >>>> Its difficulty reside in its requirement to be passed absolutely >>>> everywhere, >>>> That reveals an important fact about the nature of cancellation, >>>> it is there, it is everywhere, at all time (...within the program >>>> lifetime), >>>> it is somewhere in the background of every ops since the very beginning >>>> the program began, >>>> but not necessarily enabled, and certainly not a straight line. >>>> >>>> That is where the syntax might help to establish the plots >>>> that the runtime consumes to connect the dots >>>> and support what the developers want to achieve, >>>> in my understanding so far. >>>> >>>> My best comparison to cancellation is >>>> request rays tracing in micro services oriented architecture, >>>> on both end it is a multi-tenant, >>>> it always start with one ray, >>>> the ray always split into multiple rays, >>>> because we do programming, we need ways >>>> to create, control distribute existing/new rays, >>>> and possibly let the userland introduce a new behavior for the rays. >>>> >>>> So yeah basically main has a ray, >>>> if you process an http request, >>>> you to create a new ray >>>> to be able to cancel only that request, >>>> but it needs to be connected to the main ray >>>> because if main should be canceled, >>>> that signals should propagate to all rays connected with it, >>>> probably. >>>> >>>> ** i want to put emphasis because in the description provided >>>> by Sameer, as i tried to summarize, It has been tried to handle >>>> the problem via the type system as if it was >>>> a completely defined set of types. >>>> Which in go is wrong (yeah i m very affirmative here :), >>>> from my understanding, that might be correct in other oop languages. >>>> Technically it is possible to rewrite interfaces, methods signatures, >>>> taken individually, >>>> as a whole, and from the consumer pov, >>>> i d say it is an impossible task in go because it it goes against its >>>> nature. >>>> And i confirm/understand that by reading to Sameer feedback. >>>> >>>> Notes: i m a strong lover of go type system (not talking about values >>>> and pointers ;) >>>> >>>> >>>> On Friday, May 12, 2017 at 4:42:54 PM UTC+2, Henrik Johansson wrote: >>>>> >>>>> With the whole "Reactive" movement thread locals have started to >>>>> vanish at least in the Java ecosystem. >>>>> I agree with Sameer that, while convenient, it comes with a whole set >>>>> of obscure bugs. >>>>> >>>>> On Fri, May 12, 2017, 14:57 Sameer Ajmani <sam...@golang.org> wrote: >>>>> >>>>>> Hmm, I'm not finding good open-source examples of ThreadLocal context >>>>>> propagation in C++ and Java. My experience with this is based on what >>>>>> Google uses internally. Perhaps someone more familiar with context use >>>>>> (tracing?) outside Google can chime in? +Jaana Burcu Dogan >>>>>> >>>>>> On Thu, May 11, 2017 at 7:02 AM <mhh...@gmail.com> wrote: >>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> ..integrating Context into the runtime.. >>>>>>> >>>>>>> >>>>>>> 50% runtime, 50% syntax with explicit contextualization. >>>>>>> >>>>>>> ..The flow of request processing in Go may include multiple >>>>>>> goroutines and may move across channels; >>>>>>> >>>>>>> >>>>>>> yes big ? mark here. might the 50% of an handy and explicit syntax >>>>>>> help with it? >>>>>>> >>>>>>> >>>>>>> C++ and Java use thread-local Contexts, and while this is >>>>>>> convenient, it is often a source of mysterious bugs. >>>>>>> >>>>>>> thanks! I don't know them, >>>>>>> >>>>>>> I quickly checked according to this >>>>>>> https://dzone.com/articles/painless-introduction-javas- >>>>>>> threadlocal-storage >>>>>>> I may have totally wrong, the syntax does not look like, not even >>>>>>> from a 100km, >>>>>>> to how i represent go contexts in my head. >>>>>>> >>>>>>> This is more like the actor pattern dave cheney talks about in >>>>>>> https://dave.cheney.net/2016/11/13/do-not-fear-first-class-functions >>>>>>> (search for Let’s talk about actors) >>>>>>> >>>>>>> Is the dzone link correctly describe >>>>>>> what you mentioned as being go context equivalent in java ? >>>>>>> >>>>>>> sorry my questions are so basic. >>>>>>> >>>>>>> >>>>>>> On Thursday, May 11, 2017 at 12:29:11 PM UTC+2, Sameer Ajmani wrote: >>>>>>> >>>>>>>> I think you are asking whether we considered integrating Context >>>>>>>> into the runtime, so that it does not need to be passed explicitly. >>>>>>>> Yes, we >>>>>>>> discussed this, but decided against it. The flow of request processing >>>>>>>> in >>>>>>>> Go may include multiple goroutines and may move across channels; we >>>>>>>> decided >>>>>>>> an explicit Context made this much easier to get right. C++ and Java >>>>>>>> use >>>>>>>> thread-local Contexts, and while this is convenient, it is often a >>>>>>>> source >>>>>>>> of mysterious bugs. >>>>>>>> >>>>>>> On Thu, May 11, 2017 at 4:08 AM <mhh...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Thanks a lot! >>>>>>>>> >>>>>>>>> Might i guess and try to generalize your explanations into >>>>>>>>> "we tried to write a plumber for all cases possible" >>>>>>>>> >>>>>>>>> Which matters a lot, in my humble opinion. >>>>>>>>> >>>>>>>>> At least for the various reasons you put there, >>>>>>>>> simply put, >>>>>>>>> because it seems not technically achievable. >>>>>>>>> >>>>>>>>> Still i m happy you gave me those details, they are of interest >>>>>>>>> indeed. >>>>>>>>> >>>>>>>>> I rather intent to solve the problem on a smaller surface >>>>>>>>> with more predictability, less impossible solution to find. >>>>>>>>> I hope so. >>>>>>>>> And if it saves 90% of the time, that s already a win, imho. >>>>>>>>> >>>>>>>>> May i ask another question, >>>>>>>>> have you considered to create the plumbing >>>>>>>>> at runtime rather than statically ? >>>>>>>>> With some intents from the syntax, necessarily >>>>>>>>> (and yeah go 1 compat will be a problem, let s forget it for 1 >>>>>>>>> minute). >>>>>>>>> >>>>>>>>> I suspect in some aspects in might be handier, >>>>>>>>> because it might be all about chained func calls and not type >>>>>>>>> system handling, >>>>>>>>> and it might be easier to interleave the ctx.check in the flaw of >>>>>>>>> ops, >>>>>>>>> I don t know enough to realize for sure. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wednesday, May 10, 2017 at 11:40:27 PM UTC+2, Sameer Ajmani >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Our approach was to identify function calls that consume a >>>>>>>>>> Context (indicated by a call to context.TODO in the function body) >>>>>>>>>> and >>>>>>>>>> function calls that provide a Context (such as RPC and HTTP >>>>>>>>>> handlers). Then >>>>>>>>>> we use the guru callgraph tool to find all call paths from context >>>>>>>>>> providers to context consumers. These are the call paths that need >>>>>>>>>> to have >>>>>>>>>> a context plumbed through them. >>>>>>>>>> >>>>>>>>>> Starting from a context consumer, we can work upward to the >>>>>>>>>> context provider, adding a context parameter to reach intervening >>>>>>>>>> function >>>>>>>>>> signature. Replace context.TODO in the consumer function body with >>>>>>>>>> the new >>>>>>>>>> ctx parameter, then update all the places that call the consumer to >>>>>>>>>> pass >>>>>>>>>> context.TODO. Now we have a new set of context consumers. Repeat >>>>>>>>>> until you >>>>>>>>>> reach the context providers (if you reach main or a Test function, >>>>>>>>>> pass >>>>>>>>>> context.Background instead). >>>>>>>>>> >>>>>>>>>> This works OK for static function calls but gets messy for >>>>>>>>>> dynamic calls. If you need to add a context parameter to an interface >>>>>>>>>> method, now you have to update all implementations of that method, >>>>>>>>>> too >>>>>>>>>> (guru can find these for you). And if that interface is outside your >>>>>>>>>> control (like io.Writer), you cannot change its signature, so you >>>>>>>>>> have to >>>>>>>>>> pass the context some other way (such as via the method receiver). >>>>>>>>>> >>>>>>>>>> This gets yet more complicated if you cannot make atomic changes >>>>>>>>>> to all callers of your functions, because callers may be in other >>>>>>>>>> repositories. In this case, you must do an incremental refactoring in >>>>>>>>>> multiple steps: each change to a function signature involves adding >>>>>>>>>> a new >>>>>>>>>> function that has the context parameter, then changing all existing >>>>>>>>>> calls >>>>>>>>>> to use the new function, while preventing new calls to the old >>>>>>>>>> function, so >>>>>>>>>> that you can finally delete it. >>>>>>>>>> >>>>>>>>>> Inside Google, we ended up not needing to build all this: Context >>>>>>>>>> was introduced early enough that Go users could plumb it manually >>>>>>>>>> where >>>>>>>>>> needed. I think a context plumbing tool could still be interesting >>>>>>>>>> and >>>>>>>>>> useful to other Go users. I'd love to see someone build it! >>>>>>>>>> >>>>>>>>>> S >>>>>>>>>> >>>>>>>>>> On Tue, May 9, 2017 at 10:54 AM <mhh...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> > I've done a limited form of this using awk ;-) >>>>>>>>>>> >>>>>>>>>>> if you have a minute, >>>>>>>>>>> >>>>>>>>>>> can you tell more about what limited you >>>>>>>>>>> in your attempts and which trade made you stop (guessing), >>>>>>>>>>> if any ? >>>>>>>>>>> >>>>>>>>>>> Do you still think it be awesome ? >>>>>>>>>>> Or have you made your mind to an opposite position ? >>>>>>>>>>> if so, For which reasons? >>>>>>>>>>> >>>>>>>>>>> My tool is very poor, consider it as on going, a place for >>>>>>>>>>> inspiration to get started from absolutely no idea to lets get a >>>>>>>>>>> dirty >>>>>>>>>>> prototype. >>>>>>>>>>> not sure yet how long is going to be the road, still digging :) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tuesday, May 9, 2017 at 4:25:46 PM UTC+2, Sameer Ajmani wrote: >>>>>>>>>>> >>>>>>>>>>>> The eg tool can execute simple refactoring steps, but >>>>>>>>>>>> automating context plumbing through a chain of calls is an open >>>>>>>>>>>> problem. >>>>>>>>>>>> Alan Donovan put some thought into this a few years ago, and I've >>>>>>>>>>>> done a >>>>>>>>>>>> limited form of this using awk ;-) >>>>>>>>>>>> >>>>>>>>>>> On Tue, May 9, 2017 at 6:10 AM <mhh...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>> I want something similar too. >>>>>>>>>>>>> >>>>>>>>>>>>> Automatic and smart insertion of context args in a chain of >>>>>>>>>>>>> calls. >>>>>>>>>>>>> >>>>>>>>>>>>> Methods signature updates are easy, but how to appropriately >>>>>>>>>>>>> insert context check in the ast ? >>>>>>>>>>>>> I m not sure yet. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >The difficulty here seems to differentiate intra package >>>>>>>>>>>>> calls from calls to standard/3rd party libraries which shouldn't >>>>>>>>>>>>> be having >>>>>>>>>>>>> new param. >>>>>>>>>>>>> >>>>>>>>>>>>> That does not sound too difficult, from the pkg identifier, >>>>>>>>>>>>> lookup for the import path, for every import path, exists in >>>>>>>>>>>>> GOROOT ? >>>>>>>>>>>>> >>>>>>>>>>>>> Please put updates here anything you want to share. >>>>>>>>>>>>> >>>>>>>>>>>>> At that moment i m using this package to help me with ast, >>>>>>>>>>>>> https://github.com/mh-cbon/astutil >>>>>>>>>>>>> >>>>>>>>>>>>> might be a start even though it needs refactoring. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Monday, May 8, 2017 at 1:03:52 AM UTC+2, meir fischer wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'm adding tracing to an existing code base with many >>>>>>>>>>>>>> packages and it seems the best way to have context's passed >>>>>>>>>>>>>> around is to >>>>>>>>>>>>>> just have every method take a context.Context. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Is there any tooling for converting a code base/package to >>>>>>>>>>>>>> have: >>>>>>>>>>>>>> (a) context.Context as the first parameter in each function - >>>>>>>>>>>>>> ctx context.Context >>>>>>>>>>>>>> (b) for any function that has changed, have its callers >>>>>>>>>>>>>> (within that package) pass ctx as the first arg >>>>>>>>>>>>>> >>>>>>>>>>>>>> The difficulty here seems to differentiate intra package >>>>>>>>>>>>>> calls from calls to standard/3rd party libraries which shouldn't >>>>>>>>>>>>>> be having >>>>>>>>>>>>>> new param. >>>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>> Google Groups "golang-nuts" group. >>>>>>>>>>>>> >>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>>> it, send an email to golang-nuts...@googlegroups.com. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "golang-nuts" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to golang-nuts...@googlegroups.com. >>>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "golang-nuts" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to golang-nuts...@googlegroups.com. >>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>> >>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "golang-nuts" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to golang-nuts...@googlegroups.com. >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "golang-nuts" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to golang-nuts...@googlegroups.com. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
