This is almost certainly going to be a case where you have the correct
solution and I don't have the go experience to properly understand it. I
wasn't quite understanding how your code was "patching" the results and how
the template package knows to wait for it. Let me describe my problem.
I have a REST interface running on an embedded system. The REST api is
mandated by another entity and I have zero control over the output, I have
to produce specific conformant output. The data that I need to build the
interface is on the system in a variety of other processes, and I can get
that data via a couple of different RPC mechanisms, however sometimes these
RPC mechanisms can be slow. (I'm in progress building go bindings for them).
The current code which creates the REST interface is a huge morass of C
code that generates the JSON output, so the exact structure of the JSON
output is hardcoded in the code.
I have an opportunity here, and that is that it appears to me that, with a
little work, I can completely templatize the rest output. That is, I can
have zero page-specific code and render almost all of the output by
providing a small number of generic data access functions. The issue I ran
into is the serial nature of the substitution kills the performance in my
prototype.
Yes, I can make a specific go function per page that gathers all the data I
need and provides it to the template. But that would mean that I'd need to
maintain both the templates and the data access functions. This is still an
improvement on the old way of doing things, but I was hoping to jump
straight to a fully templatized system, which according to my initial
analysis, would be considerably less code and would not really be abusing
the template system (most of the function calls are very straightforward
and there is no heavy processing of the output).
--
Michael
On Wednesday, May 31, 2017 at 10:56:57 AM UTC-5, Egon wrote:
>
> Both of my described approaches run the funcs serially however it does not
> wait for the response and later patxhes the results.
>
> Can you describe the whole thing you are building? Piecing the
> requirements and purpose together from comments is difficult.
>
> I.e. how much memory, how big is request latency, who gets the output,
> where do the templates come from...
>
> On May 31, 2017 6:40 PM, "Michael Brown" <michael...@gmail.com
> <javascript:>> wrote:
>
> The best thing I can think of is to modify text/template to add futures
> support, and then do multi-pass rendering. The place to add this looks
> relatively simple, however the implementation looks complicated (and I just
> wrote my first program in go a couple weeks ago...)
>
> The problem that I see with the solution below is that text/template
> execution tries to instantiate the values of each function serially and
> essentially waits for completion on each, serially. I've spent the last
> couple hours examining the text/template implementation.
> --
> Michael
>
>
> On Wednesday, May 31, 2017 at 8:59:20 AM UTC-5, Egon wrote:
>>
>> The best idea I can think of (without digging and modifying
>> text/template) is to use special tokens and replace them afterwards...
>> Of course this approach has limitations in what it can do.
>>
>> *// note: code untested and incomplete*
>>
>> type Token string
>> type Queries struct {
>> pending sync.WaitGroup
>> mu sync.Mutex
>> responses map[Token]string
>> }
>>
>> func (q *Queries) createToken() Token {
>> return unique token
>> }
>>
>> func (q *Queries) Do(fn func() string) Token {
>> token := q.createToken()
>> q.pending.Add(1)
>> go func(){
>> defer q.pending.Done()
>> result := fn()
>> q.mu.Lock()
>> q.responses[token] = result
>> q.mu.Unlock()
>> }()
>> return token
>> }
>>
>> func (q *Queries) Wait(){ q.pending.Wait() }
>> func (q *Queries) Patch(data []byte) []byte {
>> // replace tokens with responess
>> }
>>
>> func main() {
>>
>> q := NewQueries()
>>
>> var funcMap = template.FuncMap {
>>
>> "sleep": func() Token { return q.Do(func() string { time.Sleep(1
>> * time.Second); return "slept" }) },
>>
>> }
>> tmpl, _ := template.New("test").Funcs(funcMap).Parse("{{sleep}}
>> {{sleep}} {{sleep}}")
>> var buf bytes.Buffer
>>
>> tmpl.Execute(&buf, nil)
>>
>> q.Wait()
>>
>> os.Stdout.Write(q.Patch(buf.Bytes))
>>
>> }
>>
>> The other approach would be to do multiple passes:
>>
>> 1. execute template
>> 2. collect funcs that haven't been run yet
>> 2.1. no funcs left --> output
>> 3. execute these funcs, cache the func values
>> 4. goto step 1 using the cache
>>
>> On Wednesday, 31 May 2017 16:26:15 UTC+3, Michael Brown wrote:
>>>
>>> I am designing a system that will heavily use text/template processing
>>> and I've run into one issue that is going to be a show stopper for me if I
>>> can't figure out a way around it.
>>>
>>> Execute() on a template will run all of the functions in the template
>>> serially.
>>>
>>> For example, when you run the code below, you can see it output "slept"
>>> once every second until it completes after 3 seconds. In this example, the
>>> sleep is simulating an RPC call to another process that may take some
>>> considerable time (few tenths of a second), but there will be a large
>>> number of these calls that could all theoretically run in parallel (ie.
>>> there are no data dependencies between them). I'd really like to know a way
>>> that I could have the templating engine run all of the functions at once
>>> and collect the output, ie. in the example below, the entire program should
>>> run in 1 second.
>>>
>>> package main
>>>
>>>
>>> import (
>>>
>>> "text/template"
>>>
>>> "os"
>>>
>>> "time"
>>>
>>> )
>>>
>>>
>>> var funcMap = template.FuncMap {
>>>
>>> "sleep": func() string { time.Sleep(1 * time.Second); return "slept"
>>> },
>>>
>>> }
>>>
>>>
>>> func main() {
>>>
>>> tmpl, _ := template.New("test").Funcs(funcMap).Parse("{{sleep}}
>>> {{sleep}} {{sleep}}")
>>>
>>> tmpl.Execute(os.Stdout, nil)
>>>
>>> }
>>>
>>>
>>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/j0WwjQE11rw/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts...@googlegroups.com <javascript:>.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
