On Wednesday, 31 May 2017 20:34:56 UTC+3, Michael Brown wrote:
>
> This is almost certainly going to be a case where you have the correct
> solution and I don't have the go experience to properly understand it. I
> wasn't quite understanding how your code was "patching" the results and how
> the template package knows to wait for it.
>
Let's say you have a template
{{ sleep }} xxx {{ sleep }}
Once you render with that approach you get something like this...
<<token1>> xxx <<token2>>
So instead of returning the actual value you replace them with a token. And
spawn a go routine to fetch the answers.
You wait for the results or just some, doesn't matter.
Once you get a results, you replace it in the buffer...
token1 -> "alpha"
token2 -> "beta"
"alpha" xxx "beta"
Or whatever the result is.
However this approach is pretty limited when you want to range over the
results.
Doing two passes over the template, might work better, but needs somewhat
better logic to handle lookup; but that could work for handling multiple
results...
{{ sleep }}
In the first pass you start the goroutines and each output goes into an
array of channels
var rpcs = []func() interface{}
var funcMap = template.FuncMap { "sleep": func() { rpcs = append(rpcs,
sleep) }, }
// now you schedule goroutines in whatever way you need for the rpcs and
write the results to a channel
var results = []chan interface{}
the second pass would do it as
current := 0
var funcMap = template.FuncMap { "sleep": func() interface{} {
result := <-results[current]
current++
return result
// ...
Without knowing the actual template, it is hard to recommend something
better simpler or easier.
Let me describe my problem.
>
> I have a REST interface running on an embedded system. The REST api is
> mandated by another entity and I have zero control over the output, I have
> to produce specific conformant output. The data that I need to build the
> interface is on the system in a variety of other processes, and I can get
> that data via a couple of different RPC mechanisms, however sometimes these
> RPC mechanisms can be slow. (I'm in progress building go bindings for them).
>
> The current code which creates the REST interface is a huge morass of C
> code that generates the JSON output, so the exact structure of the JSON
> output is hardcoded in the code.
>
> I have an opportunity here, and that is that it appears to me that, with a
> little work, I can completely templatize the rest output. That is, I can
> have zero page-specific code and render almost all of the output by
> providing a small number of generic data access functions. The issue I ran
> into is the serial nature of the substitution kills the performance in my
> prototype.
>
> Yes, I can make a specific go function per page that gathers all the data
> I need and provides it to the template. But that would mean that I'd need
> to maintain both the templates and the data access functions. This is still
> an improvement on the old way of doing things, but I was hoping to jump
> straight to a fully templatized system, which according to my initial
> analysis, would be considerably less code and would not really be abusing
> the template system (most of the function calls are very straightforward
> and there is no heavy processing of the output).
>
> --
> Michael
>
> On Wednesday, May 31, 2017 at 10:56:57 AM UTC-5, Egon wrote:
>>
>> Both of my described approaches run the funcs serially however it does
>> not wait for the response and later patxhes the results.
>>
>> Can you describe the whole thing you are building? Piecing the
>> requirements and purpose together from comments is difficult.
>>
>> I.e. how much memory, how big is request latency, who gets the output,
>> where do the templates come from...
>>
>> On May 31, 2017 6:40 PM, "Michael Brown" <michael...@gmail.com> wrote:
>>
>> The best thing I can think of is to modify text/template to add futures
>> support, and then do multi-pass rendering. The place to add this looks
>> relatively simple, however the implementation looks complicated (and I just
>> wrote my first program in go a couple weeks ago...)
>>
>> The problem that I see with the solution below is that text/template
>> execution tries to instantiate the values of each function serially and
>> essentially waits for completion on each, serially. I've spent the last
>> couple hours examining the text/template implementation.
>> --
>> Michael
>>
>>
>> On Wednesday, May 31, 2017 at 8:59:20 AM UTC-5, Egon wrote:
>>>
>>> The best idea I can think of (without digging and modifying
>>> text/template) is to use special tokens and replace them afterwards...
>>> Of course this approach has limitations in what it can do.
>>>
>>> *// note: code untested and incomplete*
>>>
>>> type Token string
>>> type Queries struct {
>>> pending sync.WaitGroup
>>> mu sync.Mutex
>>> responses map[Token]string
>>> }
>>>
>>> func (q *Queries) createToken() Token {
>>> return unique token
>>> }
>>>
>>> func (q *Queries) Do(fn func() string) Token {
>>> token := q.createToken()
>>> q.pending.Add(1)
>>> go func(){
>>> defer q.pending.Done()
>>> result := fn()
>>> q.mu.Lock()
>>> q.responses[token] = result
>>> q.mu.Unlock()
>>> }()
>>> return token
>>> }
>>>
>>> func (q *Queries) Wait(){ q.pending.Wait() }
>>> func (q *Queries) Patch(data []byte) []byte {
>>> // replace tokens with responess
>>> }
>>>
>>> func main() {
>>>
>>> q := NewQueries()
>>>
>>> var funcMap = template.FuncMap {
>>>
>>> "sleep": func() Token { return q.Do(func() string { time.Sleep(1
>>> * time.Second); return "slept" }) },
>>>
>>> }
>>> tmpl, _ := template.New("test").Funcs(funcMap).Parse("{{sleep}}
>>> {{sleep}} {{sleep}}")
>>> var buf bytes.Buffer
>>>
>>> tmpl.Execute(&buf, nil)
>>>
>>> q.Wait()
>>>
>>> os.Stdout.Write(q.Patch(buf.Bytes))
>>>
>>> }
>>>
>>> The other approach would be to do multiple passes:
>>>
>>> 1. execute template
>>> 2. collect funcs that haven't been run yet
>>> 2.1. no funcs left --> output
>>> 3. execute these funcs, cache the func values
>>> 4. goto step 1 using the cache
>>>
>>> On Wednesday, 31 May 2017 16:26:15 UTC+3, Michael Brown wrote:
>>>>
>>>> I am designing a system that will heavily use text/template processing
>>>> and I've run into one issue that is going to be a show stopper for me if I
>>>> can't figure out a way around it.
>>>>
>>>> Execute() on a template will run all of the functions in the template
>>>> serially.
>>>>
>>>> For example, when you run the code below, you can see it output "slept"
>>>> once every second until it completes after 3 seconds. In this example, the
>>>> sleep is simulating an RPC call to another process that may take some
>>>> considerable time (few tenths of a second), but there will be a large
>>>> number of these calls that could all theoretically run in parallel (ie.
>>>> there are no data dependencies between them). I'd really like to know a
>>>> way
>>>> that I could have the templating engine run all of the functions at once
>>>> and collect the output, ie. in the example below, the entire program
>>>> should
>>>> run in 1 second.
>>>>
>>>> package main
>>>>
>>>>
>>>> import (
>>>>
>>>> "text/template"
>>>>
>>>> "os"
>>>>
>>>> "time"
>>>>
>>>> )
>>>>
>>>>
>>>> var funcMap = template.FuncMap {
>>>>
>>>> "sleep": func() string { time.Sleep(1 * time.Second); return
>>>> "slept" },
>>>>
>>>> }
>>>>
>>>>
>>>> func main() {
>>>>
>>>> tmpl, _ := template.New("test").Funcs(funcMap).Parse("{{sleep}}
>>>> {{sleep}} {{sleep}}")
>>>>
>>>> tmpl.Execute(os.Stdout, nil)
>>>>
>>>> }
>>>>
>>>>
>>>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "golang-nuts" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/golang-nuts/j0WwjQE11rw/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> golang-nuts...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
