Generally I'd suggest passing context.Background() when calling functions that need a context from main or tests. On Sat, May 13, 2017 at 8:27 AM Peter Weinberger (温博格) <p...@google.com> wrote:
> Hi. I was one of the people who failed in an attempt to auto-insert > contexts in all of google3. I no longer remember all the obstacles I failed > to overcome, but would encourage others to take on the project. > > One issue was libraries that were used both in paths from http requests > (so they needed the context propagated through them) but were also used in > places where their callers (starting at main) didn't have any contexts at > all. Sitting by my coffee this morning this no longer seems like an > insuperable obstacle (e.g., pass nils and change them to > context.Background() inside, or, like appengine's log, split packages into > one that requires contexts and one that doesn't). But I had lots of what I > thought were plausible ideas that broke on the hard rock of google3, and I > suspect these would too. > > (The project wasn't a total failure. It did plumb contexts through a bunch > of packages. But the general case was too hard for me.) > > On Sat, May 13, 2017 at 3:50 AM, <mhhc...@gmail.com> wrote: > >> A reference i found about "reactive programming" >> https://github.com/dsyer/reactive-notes/blob/master/intro.adoc >> >> Hope this introduced the concept correctly, thanks for pointing that. >> >> Following are only some thoughts, >> >> Two things surprising here, >> 1/ a programmer prefers not to solve a problem >> 2/ one failed attempt mean the end of all attempts** >> >> That being said, my little understanding so far is that current context >> is used for two purposes, >> - cancellation >> - data (transmission? not sure) (+/- ts, have not checked i expect so in >> regards to previous referenceS cited) >> >> While cancellation is a feature i really want to take advantage of, >> the other one is much more arguable in the sense that >> it can remains expressed purely by the end user without bloating >> his productivity and quality (you don t really need it everywhere, just >> where it is needed), >> it seems to be highly software context dependent. >> >> Whereas cancellation, while it looks likes simple, is maybe more subtle. >> After all it is about assigning uuid to a chain of call and >> appropriately propagate, disjoint/rejoin new uuids with the previous one, >> so that we can ask to stop execution of a sub selection of a chain of >> calls >> via an handle. >> Am i too simplistic? >> >> Its difficulty reside in its requirement to be passed absolutely >> everywhere, >> That reveals an important fact about the nature of cancellation, >> it is there, it is everywhere, at all time (...within the program >> lifetime), >> it is somewhere in the background of every ops since the very beginning >> the program began, >> but not necessarily enabled, and certainly not a straight line. >> >> That is where the syntax might help to establish the plots >> that the runtime consumes to connect the dots >> and support what the developers want to achieve, >> in my understanding so far. >> >> My best comparison to cancellation is >> request rays tracing in micro services oriented architecture, >> on both end it is a multi-tenant, >> it always start with one ray, >> the ray always split into multiple rays, >> because we do programming, we need ways >> to create, control distribute existing/new rays, >> and possibly let the userland introduce a new behavior for the rays. >> >> So yeah basically main has a ray, >> if you process an http request, >> you to create a new ray >> to be able to cancel only that request, >> but it needs to be connected to the main ray >> because if main should be canceled, >> that signals should propagate to all rays connected with it, >> probably. >> >> ** i want to put emphasis because in the description provided >> by Sameer, as i tried to summarize, It has been tried to handle >> the problem via the type system as if it was >> a completely defined set of types. >> Which in go is wrong (yeah i m very affirmative here :), >> from my understanding, that might be correct in other oop languages. >> Technically it is possible to rewrite interfaces, methods signatures, >> taken individually, >> as a whole, and from the consumer pov, >> i d say it is an impossible task in go because it it goes against its >> nature. >> And i confirm/understand that by reading to Sameer feedback. >> >> Notes: i m a strong lover of go type system (not talking about values and >> pointers ;) >> >> >> On Friday, May 12, 2017 at 4:42:54 PM UTC+2, Henrik Johansson wrote: >>> >>> With the whole "Reactive" movement thread locals have started to vanish >>> at least in the Java ecosystem. >>> I agree with Sameer that, while convenient, it comes with a whole set of >>> obscure bugs. >>> >>> On Fri, May 12, 2017, 14:57 Sameer Ajmani <sam...@golang.org> wrote: >>> >>>> Hmm, I'm not finding good open-source examples of ThreadLocal context >>>> propagation in C++ and Java. My experience with this is based on what >>>> Google uses internally. Perhaps someone more familiar with context use >>>> (tracing?) outside Google can chime in? +Jaana Burcu Dogan >>>> >>>> On Thu, May 11, 2017 at 7:02 AM <mhh...@gmail.com> wrote: >>>> >>>>> thanks, >>>>> >>>>> ..integrating Context into the runtime.. >>>>> >>>>> >>>>> 50% runtime, 50% syntax with explicit contextualization. >>>>> >>>>> ..The flow of request processing in Go may include multiple goroutines >>>>> and may move across channels; >>>>> >>>>> >>>>> yes big ? mark here. might the 50% of an handy and explicit syntax >>>>> help with it? >>>>> >>>>> >>>>> C++ and Java use thread-local Contexts, and while this is convenient, >>>>> it is often a source of mysterious bugs. >>>>> >>>>> thanks! I don't know them, >>>>> >>>>> I quickly checked according to this >>>>> >>>>> https://dzone.com/articles/painless-introduction-javas-threadlocal-storage >>>>> I may have totally wrong, the syntax does not look like, not even from >>>>> a 100km, >>>>> to how i represent go contexts in my head. >>>>> >>>>> This is more like the actor pattern dave cheney talks about in >>>>> https://dave.cheney.net/2016/11/13/do-not-fear-first-class-functions >>>>> (search for Let’s talk about actors) >>>>> >>>>> Is the dzone link correctly describe >>>>> what you mentioned as being go context equivalent in java ? >>>>> >>>>> sorry my questions are so basic. >>>>> >>>>> >>>>> On Thursday, May 11, 2017 at 12:29:11 PM UTC+2, Sameer Ajmani wrote: >>>>> >>>>>> I think you are asking whether we considered integrating Context into >>>>>> the runtime, so that it does not need to be passed explicitly. Yes, we >>>>>> discussed this, but decided against it. The flow of request processing in >>>>>> Go may include multiple goroutines and may move across channels; we >>>>>> decided >>>>>> an explicit Context made this much easier to get right. C++ and Java use >>>>>> thread-local Contexts, and while this is convenient, it is often a source >>>>>> of mysterious bugs. >>>>>> >>>>> On Thu, May 11, 2017 at 4:08 AM <mhh...@gmail.com> wrote: >>>>>> >>>>>>> Thanks a lot! >>>>>>> >>>>>>> Might i guess and try to generalize your explanations into >>>>>>> "we tried to write a plumber for all cases possible" >>>>>>> >>>>>>> Which matters a lot, in my humble opinion. >>>>>>> >>>>>>> At least for the various reasons you put there, >>>>>>> simply put, >>>>>>> because it seems not technically achievable. >>>>>>> >>>>>>> Still i m happy you gave me those details, they are of interest >>>>>>> indeed. >>>>>>> >>>>>>> I rather intent to solve the problem on a smaller surface >>>>>>> with more predictability, less impossible solution to find. >>>>>>> I hope so. >>>>>>> And if it saves 90% of the time, that s already a win, imho. >>>>>>> >>>>>>> May i ask another question, >>>>>>> have you considered to create the plumbing >>>>>>> at runtime rather than statically ? >>>>>>> With some intents from the syntax, necessarily >>>>>>> (and yeah go 1 compat will be a problem, let s forget it for 1 >>>>>>> minute). >>>>>>> >>>>>>> I suspect in some aspects in might be handier, >>>>>>> because it might be all about chained func calls and not type system >>>>>>> handling, >>>>>>> and it might be easier to interleave the ctx.check in the flaw of >>>>>>> ops, >>>>>>> I don t know enough to realize for sure. >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wednesday, May 10, 2017 at 11:40:27 PM UTC+2, Sameer Ajmani wrote: >>>>>>> >>>>>>>> Our approach was to identify function calls that consume a Context >>>>>>>> (indicated by a call to context.TODO in the function body) and function >>>>>>>> calls that provide a Context (such as RPC and HTTP handlers). Then we >>>>>>>> use >>>>>>>> the guru callgraph tool to find all call paths from context providers >>>>>>>> to >>>>>>>> context consumers. These are the call paths that need to have a context >>>>>>>> plumbed through them. >>>>>>>> >>>>>>>> Starting from a context consumer, we can work upward to the context >>>>>>>> provider, adding a context parameter to reach intervening function >>>>>>>> signature. Replace context.TODO in the consumer function body with the >>>>>>>> new >>>>>>>> ctx parameter, then update all the places that call the consumer to >>>>>>>> pass >>>>>>>> context.TODO. Now we have a new set of context consumers. Repeat until >>>>>>>> you >>>>>>>> reach the context providers (if you reach main or a Test function, pass >>>>>>>> context.Background instead). >>>>>>>> >>>>>>>> This works OK for static function calls but gets messy for dynamic >>>>>>>> calls. If you need to add a context parameter to an interface method, >>>>>>>> now >>>>>>>> you have to update all implementations of that method, too (guru can >>>>>>>> find >>>>>>>> these for you). And if that interface is outside your control (like >>>>>>>> io.Writer), you cannot change its signature, so you have to pass the >>>>>>>> context some other way (such as via the method receiver). >>>>>>>> >>>>>>>> This gets yet more complicated if you cannot make atomic changes to >>>>>>>> all callers of your functions, because callers may be in other >>>>>>>> repositories. In this case, you must do an incremental refactoring in >>>>>>>> multiple steps: each change to a function signature involves adding a >>>>>>>> new >>>>>>>> function that has the context parameter, then changing all existing >>>>>>>> calls >>>>>>>> to use the new function, while preventing new calls to the old >>>>>>>> function, so >>>>>>>> that you can finally delete it. >>>>>>>> >>>>>>>> Inside Google, we ended up not needing to build all this: Context >>>>>>>> was introduced early enough that Go users could plumb it manually where >>>>>>>> needed. I think a context plumbing tool could still be interesting and >>>>>>>> useful to other Go users. I'd love to see someone build it! >>>>>>>> >>>>>>>> S >>>>>>>> >>>>>>>> On Tue, May 9, 2017 at 10:54 AM <mhh...@gmail.com> wrote: >>>>>>>> >>>>>>>>> > I've done a limited form of this using awk ;-) >>>>>>>>> >>>>>>>>> if you have a minute, >>>>>>>>> >>>>>>>>> can you tell more about what limited you >>>>>>>>> in your attempts and which trade made you stop (guessing), >>>>>>>>> if any ? >>>>>>>>> >>>>>>>>> Do you still think it be awesome ? >>>>>>>>> Or have you made your mind to an opposite position ? >>>>>>>>> if so, For which reasons? >>>>>>>>> >>>>>>>>> My tool is very poor, consider it as on going, a place for >>>>>>>>> inspiration to get started from absolutely no idea to lets get a dirty >>>>>>>>> prototype. >>>>>>>>> not sure yet how long is going to be the road, still digging :) >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tuesday, May 9, 2017 at 4:25:46 PM UTC+2, Sameer Ajmani wrote: >>>>>>>>> >>>>>>>>>> The eg tool can execute simple refactoring steps, but automating >>>>>>>>>> context plumbing through a chain of calls is an open problem. Alan >>>>>>>>>> Donovan >>>>>>>>>> put some thought into this a few years ago, and I've done a limited >>>>>>>>>> form of >>>>>>>>>> this using awk ;-) >>>>>>>>>> >>>>>>>>> On Tue, May 9, 2017 at 6:10 AM <mhh...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>> I want something similar too. >>>>>>>>>>> >>>>>>>>>>> Automatic and smart insertion of context args in a chain of >>>>>>>>>>> calls. >>>>>>>>>>> >>>>>>>>>>> Methods signature updates are easy, but how to appropriately >>>>>>>>>>> insert context check in the ast ? >>>>>>>>>>> I m not sure yet. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >The difficulty here seems to differentiate intra package calls >>>>>>>>>>> from calls to standard/3rd party libraries which shouldn't be >>>>>>>>>>> having new >>>>>>>>>>> param. >>>>>>>>>>> >>>>>>>>>>> That does not sound too difficult, from the pkg identifier, >>>>>>>>>>> lookup for the import path, for every import path, exists in GOROOT >>>>>>>>>>> ? >>>>>>>>>>> >>>>>>>>>>> Please put updates here anything you want to share. >>>>>>>>>>> >>>>>>>>>>> At that moment i m using this package to help me with ast, >>>>>>>>>>> https://github.com/mh-cbon/astutil >>>>>>>>>>> >>>>>>>>>>> might be a start even though it needs refactoring. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Monday, May 8, 2017 at 1:03:52 AM UTC+2, meir fischer wrote: >>>>>>>>>>>> >>>>>>>>>>>> I'm adding tracing to an existing code base with many packages >>>>>>>>>>>> and it seems the best way to have context's passed around is to >>>>>>>>>>>> just have >>>>>>>>>>>> every method take a context.Context. >>>>>>>>>>>> >>>>>>>>>>>> Is there any tooling for converting a code base/package to have: >>>>>>>>>>>> (a) context.Context as the first parameter in each function - >>>>>>>>>>>> ctx context.Context >>>>>>>>>>>> (b) for any function that has changed, have its callers (within >>>>>>>>>>>> that package) pass ctx as the first arg >>>>>>>>>>>> >>>>>>>>>>>> The difficulty here seems to differentiate intra package calls >>>>>>>>>>>> from calls to standard/3rd party libraries which shouldn't be >>>>>>>>>>>> having new >>>>>>>>>>>> param. >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "golang-nuts" group. >>>>>>>>>>> >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>>> send an email to golang-nuts...@googlegroups.com. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "golang-nuts" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to golang-nuts...@googlegroups.com. >>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>> >>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "golang-nuts" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to golang-nuts...@googlegroups.com. >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "golang-nuts" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to golang-nuts...@googlegroups.com. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "golang-nuts" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to golang-nuts...@googlegroups.com. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
