Re: [go-nuts] Huge system RSS memory usage

[miha . dimec]

Hi I am back with update.

There is most certainly was memory leak connected with openCv. I don't know 
where is the problem yet.
I can confirm this, because I have run test while mocking openCv out, there 
were no leaks of the type that was desbribed before. RSS was constant. 
Except there was one sudden rise of RSS to 6GB, but this was the problem in 
go code that I was able to profile with profiler. This was different than 
before.

I wonder why Valgrind doesn’t report those leaks? I will need some other 
methodology to track the problem down.
But this is out of the scope of this discussion. Any ideas are welcome 
anyway :)

Thanks. 


On Monday, May 8, 2017 at 8:40:11 AM UTC+2, miha@visionect.com wrote:
>
> Hi Justin
>
> I don't think I am leaking goroutines or any other Go resources. Check 
> graphs in attachment.
>
> OpenCV and Zmq cgo bindings are used quite heavily.
> About those I am sure about leaks, as, much I can relay on valgrind. Does 
> valgrind work reliably in Cgo environment?
>
> For the start I will stub out OpenCV dependency, to see what happens. Will 
> report back
>
> We have tried GraphicMagick and ImageMagic in the past and we have found 
> both them quite slow.
> I don't know about OpenImageIO, but I will certainly check it out.
>
> Thanks
>
> Miha
>
> On Saturday, May 6, 2017 at 12:41:33 AM UTC+2, Justin Israel wrote:
>>
>> I also have done an image processing server in the past and used two 
>> different image library cgo bindings. Each one has its own specific way it 
>> manages the C memory and you can leak of you don't observe them. I am not 
>> familiar with the OpenCV binging though. 
>>
>> Are you leaking any goroutines? How extensively are you using the OpenCV 
>> features, and can you swap to another library to confirm whether it is 
>> OpenCV or not? 
>>
>> I used to use ImageMagick bindings (which I still maintain) and now I use 
>> OpenImageIO bindings (which I authored). I haven't had memory leak issues 
>> with these libs in current production use. 
>>
>> Justin 
>>
>> On Sat, May 6, 2017, 12:27 AM  wrote:
>>
>>> Hi 
>>>
>>>
>>>
>>> 
>>>
>>> 1. System
>>>
>>>
>>> Ubuntu 16.04.1 LTS \n \l
>>> Linux G-00017 4.4.0-57-generic #78-Ubuntu SMP Fri Dec 9 23:50:32 UTC 
>>> 2016 x86_64 x86_64 x86_64 GNU/Linux
>>>
>>> 
>>>
>>> 16GB of RAM
>>>
>>> App runs inside docker: Docker version 17.03.1-ce, build c6d412e
>>>
>>>
>>>
>>> 
>>>
>>> 2. Memory usage problem
>>>
>>> I am having memory usage problems in production, where things do fail 
>>> because some Go processes use much more memory than expected.
>>>
>>> I have prepared a test, where I am running my my server application, 
>>> and I am measuring, how memory is consumed for one process. 
>>>
>>> I fount out that more and more system memory is consumed with time. On 
>>> the graph below, you can see how RSS has risen from ~0.5GB up to ~6Gb in a 
>>> few days.
>>>
>>>
>>>
>>> 
>>>
>>>
>>>
>>> 
>>>
>>>
>>> 
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> The app does a lot of image processing for the images from remote source 
>>> (in the range of 100 images each second). All this processing, means a lot 
>>> of allocations, so naturally I tough, this must be memory a memory leak.
>>>
>>> So I run pprof (see attachment: pprof.log), which reports heap usage 
>>> below 300Mb. This heap usage, doesn't seam to change much over the time. 
>>>
>>> Then, there are some CGO bindings with OpenCV, so I thought there must 
>>> be some leak in C code. I run the thing with Valgrind, which didn't find 
>>> anything except some minor leak, which I had already fixed by now.
>>>
>>>
>>>
>>> 
>>>
>>> I also run the app with GOGC=50, but it didn't help.
>>>
>>>
>>> I am looking for the explanation what is happening, and how this can be 
>>> solved?
>>>
>>>
>>>
>>> Thanks!
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to golang-nuts...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>

-- 
You received this messag

[go-nuts] Re: [ANN] Ugarit

[mhhcbon]

HI,

some feedback,

So this package generates epub files ? In versions 20 / 30 ?
Is it correct ?

Can you add an example to use it ?
Small one.

My very personal taste,
I love useful READMEs.

On the code itself,
i suspect you don t know yet about *go fmt*,
I strongly suggest you to use it,
just because its a great idea.

https://github.com/luisfurquim/ugarit/blob/master/epub20/book.go#L66
This hurts :O *if (len(metatag)>0) {*

See also go vet.

(generally speaking a good IDE will provide that for you,
for example i use atom+go-plus: great)

Then, lots of small things here and there,

https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L126
I don t think you need to init values here, see
https://play.golang.org/p/eN9cT9gQlE

https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L137
This kind of construction is redundant, en is the default value.

https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L134
This if *seems* useless 

https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L258
I did not get why options is a param of type interface{},
within the method body it might *only* be an *EPubOptions.
Can it be a param of type *EPubOptions directly ?

https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L456
twice consecutive and identical conditions.

https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L464
I wonder.

https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L487
take advantage of filepath.Join() + os.Separator ? maybe.

https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L626
why 59?


I must say you took great care about error values and documentation
...I m a small player in comparison :p

was it helpful ? I don't know.



On Sunday, May 14, 2017 at 10:49:44 PM UTC+2, Luis Furquim wrote:
>
> Hello Gophers!
>
> I am publishing a first barely usable version of an EPub eBook generator. 
> It still has many improvements to be done. Documentation is poor but 
> exists. There's an example in the epub30 package. Criticism would be a 
> great help. I know it has much to be done. So, please be gentle! ;)
>
> Thank you all!
> Luis Otavio de Colla Furquim
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: adding context.Context to new code

[mhhcbon]

+1 for any feedback.

> One issue was libraries that were used both in paths from http requests 
(so they needed the context propagated through them) but were also used in 
places where their callers (starting at main) didn't have any contexts at 
all

yup. 
Imagine also in the middle of the call you invoke a third party package 
without context support.

The problem mlight be even worst because of the weaknesses of the current 
type system.
You can t simply declare a type *mypackage.TContexted* of *foreign.T* and 
hope it will work.
I suspect it might need a whole re-factoring/re-integration of the foreign 
package *and* its dependencies.

> or, like appengine's log, split packages into one that requires contexts 
and one that doesn't)

yes i think its an approach that can work if taken appropriately.

Taken in its basic form:
- its expensive, money costs * 2
- its prone to bug, duplicated code, duplicated bugs
- its not genuine nor magic, it still needs lots of handcrafted work



A sarcastic question that bugs me, 
and probably wrong in its term as is,

Is go the language of the 21th century, 
or the last language of the 20th century
?


On Saturday, May 13, 2017 at 2:27:21 PM UTC+2, Peter Weinberger wrote:
>
> Hi. I was one of the people who failed in an attempt to auto-insert 
> contexts in all of google3. I no longer remember all the obstacles I failed 
> to overcome, but would encourage others to take on the project.
>
> One issue was libraries that were used both in paths from http requests 
> (so they needed the context propagated through them) but were also used in 
> places where their callers (starting at main) didn't have any contexts at 
> all. Sitting by my coffee this morning this no longer seems like an 
> insuperable obstacle (e.g., pass nils and change them to 
> context.Background() inside, or, like appengine's log, split packages into 
> one that requires contexts and one that doesn't). But I had lots of what I 
> thought were plausible ideas that broke on the hard rock of google3, and I 
> suspect these would too.
>
> (The project wasn't a total failure. It did plumb contexts through a bunch 
> of packages. But the general case was too hard for me.)
>
> On Sat, May 13, 2017 at 3:50 AM, > wrote:
>
>> A reference i found about "reactive programming"
>> https://github.com/dsyer/reactive-notes/blob/master/intro.adoc
>>
>> Hope this introduced the concept correctly, thanks for pointing that.
>>
>> Following are only some thoughts,
>>
>> Two things surprising here,
>> 1/ a programmer prefers not to solve a problem
>> 2/ one failed attempt mean the end of all attempts**
>>
>> That being said, my little understanding so far is that current context 
>> is used for two purposes,
>> - cancellation
>> - data (transmission? not sure) (+/- ts, have not checked i expect so in 
>> regards to previous referenceS cited)
>>
>> While cancellation is a feature i really want to take advantage of,
>> the other one is much more arguable in the sense that
>> it can remains expressed purely by the end user without bloating 
>> his productivity and quality (you don t really need it everywhere, just 
>> where it is needed), 
>> it seems to be highly software context dependent.
>>
>> Whereas cancellation, while it looks likes simple, is maybe more subtle.
>> After all it is about assigning uuid to a chain of call and 
>> appropriately propagate, disjoint/rejoin new uuids with the previous one, 
>> so that we can ask to stop execution of a sub selection of a chain of 
>> calls
>> via an handle. 
>> Am i too simplistic?
>>
>> Its difficulty reside in its requirement to be passed absolutely 
>> everywhere,
>> That reveals an important fact about the nature of cancellation,
>> it is there, it is everywhere, at all time (...within the program 
>> lifetime),
>> it is somewhere in the background of every ops since the very beginning 
>> the program began,
>> but not necessarily enabled, and certainly not a straight line.
>>
>> That is where the syntax might help to establish the plots
>> that the runtime consumes to connect the dots 
>> and support what the developers want to achieve,
>> in my understanding so far.
>>
>> My best comparison to cancellation is 
>> request rays tracing in micro services oriented architecture,
>> on both end it is a multi-tenant,
>> it always start with one ray,
>> the ray always split into multiple rays,
>> because we do programming, we need ways
>> to create, control distribute existing/new rays,
>> and possibly let the userland introduce a new behavior for the rays.
>>
>> So yeah basically main has a ray,
>> if you process an http request,
>> you to create a new ray 
>> to be able to cancel only that request,
>> but it needs to be connected to the main ray
>> because if main should be canceled,
>> that signals should propagate to all rays connected with it,
>> probably.
>>
>> ** i want to put emphasis because in the description prov

[go-nuts] Re: [ANN] Ugarit

[Luis Furquim]

Hi,

Yes, this is correct, it generates epub files in versions 20/30. But it is 
an initial version, you really can gernerate the epubs and open in iBooks 
and Adobe. But it is has a long way to evolve yet.
The example can be found 
at https://godoc.org/github.com/luisfurquim/ugarit/epub30#ex-package . The 
README will come soon, I just hadn't the time yet, I focused on godocs 
first and it is still incomplete.
About the  *EPubOptions x interface{}: the method has to satisfy the 
ugarit.Book interface, which, in turn, is not thought to be tied to EPub, 
but leave room to be satisfied by other eBook formats. 
So, I decided to create an interface parameter letting any package to 
define it's own options. But I recognize that this decision is exactly the 
type I wished to be put in question, I am aware that bad
decisions may ruin an entire API.

The lacking of go fmt/vet: my bad. Will be corrected at the next version!

I can't dive into code and look at the other comments right now. I'll do it 
at night (here is 10:30 AM now), then I'll send another mail.

And yes, your comments are of a great help for me! Thank you!

Luis Otavio de Colla Furquim

Em segunda-feira, 15 de maio de 2017 09:30:20 UTC-3, mhh...@gmail.com 
escreveu:
>
> HI,
>
> some feedback,
>
> So this package generates epub files ? In versions 20 / 30 ?
> Is it correct ?
>
> Can you add an example to use it ?
> Small one.
>
> My very personal taste,
> I love useful READMEs.
>
> On the code itself,
> i suspect you don t know yet about *go fmt*,
> I strongly suggest you to use it,
> just because its a great idea.
>
> https://github.com/luisfurquim/ugarit/blob/master/epub20/book.go#L66
> This hurts :O *if (len(metatag)>0) {*
>
> See also go vet.
>
> (generally speaking a good IDE will provide that for you,
> for example i use atom+go-plus: great)
>
> Then, lots of small things here and there,
>
> https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L126
> I don t think you need to init values here, see
> https://play.golang.org/p/eN9cT9gQlE
>
> https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L137
> This kind of construction is redundant, en is the default value.
>
> https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L134
> This if *seems* useless 
>
> https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L258
> I did not get why options is a param of type interface{},
> within the method body it might *only* be an *EPubOptions.
> Can it be a param of type *EPubOptions directly ?
>
> https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L456
> twice consecutive and identical conditions.
>
> https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L464
> I wonder.
>
> https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L487
> take advantage of filepath.Join() + os.Separator ? maybe.
>
> https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L626
> why 59?
>
>
> I must say you took great care about error values and documentation
> ...I m a small player in comparison :p
>
> was it helpful ? I don't know.
>
>
>
> On Sunday, May 14, 2017 at 10:49:44 PM UTC+2, Luis Furquim wrote:
>>
>> Hello Gophers!
>>
>> I am publishing a first barely usable version of an EPub eBook generator. 
>> It still has many improvements to be done. Documentation is poor but 
>> exists. There's an example in the epub30 package. Criticism would be a 
>> great help. I know it has much to be done. So, please be gentle! ;)
>>
>> Thank you all!
>> Luis Otavio de Colla Furquim
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: New fully featured bash completion for go - written in go

[Eyal Posener]

The right installation commands are:

go get github.com/posener/complete/gocomplete
gocomplete -install

You can also view the code in https://github.com/posener/complete


On Saturday, May 13, 2017 at 11:29:47 AM UTC+3, Eyal Posener wrote:
>
> Simple to install (Assuming GOPATH and PATH are correct):
>
> go install github.com/posener/complete/gocomplete
> gocomplete -install
>
>
> Features:
>
>- Complete go command, sub commands and flags.
>- Complete package names, .go files and ellipsis when necessary.
>- Complete test names after -run flag!
>
> Works with bash / zsh shells
>
> This is also a package, that enables writing bash completion scripts, or 
> add them to an existing go program.
>
> Please, open issues, contribute and star!
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: [ANN] scaffolder - web application servers to order

[Tong Sun]

Quite agree with the following. 

scaffolding just like symfony/RoR scaffold generator! That's something I 
had been hoping and looking for for quite a while. 

Thanks a lot!


On Thursday, May 11, 2017 at 4:37:44 AM UTC-4, mhh...@gmail.com wrote:
>
> It s awesome, 
> in my opinion, this is the way i want to consume go in the future.
>
> pragmatic, correct, fast, repeatable.
>
> in additions to what go provides, 
> fast build, cross platform, easy to package
>
> Although, my very personal opinion,
> lets do smaller program that combines together,
> in the spirit of unix tooling, rather than a
> big boiler plate thing in the spirit of symfony 
> or alike (really just an example sorry i cited your project).
>
> A big Yes for such projects!
>
> Let s generate all the thing and get drinks to talk about it :D
>
> PS: i remember you talked about it earlier, shame i missed it at that time.
>
> On Wednesday, May 10, 2017 at 6:31:22 PM UTC+2, Simon Ritchie wrote:
>>
>> Given a JSON description of some database tables, the scaffolder tool 
>> creates the database and generates a web application server to manage it.  
>> The resulting app server implements the Create, Read, Update and Delete 
>> (CRUD) operations on the tables.
>>
>> The idea for the scaffolder comes from the Ruby-on-Rails scaffold 
>> generator.
>>
>> The app server is presented as Go source code and HTML templates, plus 
>> some unit and integration tests.  The design follows the Model, View 
>> Controller (MVC) pattern.  The HTTP requests follow the REST pattern.
>>
>> The tool is here:  https://github.com/goblimey/scaffolder
>>
>> There are some screen shots of the resulting web pages here:   
>> http://www.goblimey.com/scaffolder/2.4.running.the.server.html
>>
>> Producing any web application involves a lot of boilerplate work, and 
>> this tool aims to automate some of that without imposing too many design 
>> decisions on the result.  
>>
>> The generated web pages are fairly primitive, with very little styling.  
>> This is deliberate - if you want to use the result as a basis for building 
>> your own application, you will want to define your own styling, and the 
>> pages are structured to allow that.
>>
>> The material produced by the scaffolder is defined by a set of text 
>> templates.  For each table it produces from these templates a model, a 
>> controller and set of HTML templates to produce the views.  (So we have 
>> templates producing templates.)
>>
>> The scaffolder tool itself is very simple.  It just reads the JSON 
>> specification into a data structure, enhances that data a little and then 
>> iterates through it, supplying it to the templates.   This approach makes 
>> the tool very flexible - if it doesn't do quite what you want, it's very 
>> easy to tweak it.
>>
>> This idea of using a simple driver program, JSON data and templates to 
>> generate a result is very powerful.  It can be used to produce all sorts of 
>> material that follows a prototypical pattern.
>>
>> All comment on this project are welcome.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: [ANN] gocog - generate code in any language, for any language

[Tong Sun]

I was a big fan of GSL 
,
 
but its old licensing and new 4+ version all gave me troubles. So I ended 
up writing my own:

easygen 
https://github.com/go-easygen/easygen#details


On Wednesday, May 10, 2017 at 5:39:07 PM UTC-4, Andy Green wrote:
>
> This project looks very similar to GSL 
> ,
>  
> which generates source code in multiple programming languages.
>
> On Wednesday, January 16, 2013 at 2:23:37 PM UTC-5, Nate Finch wrote:
>>
>> https://github.com/natefinch/gocog
>>
>> of course, right now it only supports Go as a generating language, 
>> but my next task is to extend it to support whatever language you want (so 
>> long as you can run the code through a command line call).  It's a work in 
>> progress, only basic functionality exists right now, and there's no unit 
>> tests (boo!), but I was excited to get it to a workable alpha stage.
>>
>> Gocog is based off the idea of Ned Batchelder's cog.py 
>>  (which he developed while we were 
>> working together at a now-defunct startup), I wanted something that would 
>> be language agnostic, and able to run without any predefined frameworks in 
>> place. Of course, Go is the perfect language for this, since it creates a 
>> native executable.
>>
>> Now, you obviously need to have the framework installed that runs the 
>> generator code, but gocog itself has no such requirements, which means that 
>> if you're a python programmer, you can run it with only python installed, 
>> or only ruby, etc.
>>
>> One nice thing about using Go to create gocog - it was trivial to turn it 
>> into a concurrent command that'll run on every file you specify 
>> simultaneously, which can greatly speed up generation times for large 
>> numbers of files.
>>
>> I'm still new to Go, so if anything jumps out at you as needing fixing, 
>> let me know.  
>>
>> I welcome your thoughts and criticisms.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: [ANN] scaffolder - web application servers to order

[Simon Ritchie]

Don't wait.  Write it yourself.

Seriously, if my solution does most of what you want, but it's not quite
right, it's very easy to change it.  Just hack the templates.

Regards

Simon

On Mon, May 15, 2017 at 4:41 PM, Tong Sun  wrote:

> Quite agree with the following.
>
> scaffolding just like symfony/RoR scaffold generator! That's something I
> had been hoping and looking for for quite a while.
>
> Thanks a lot!
>
>
> On Thursday, May 11, 2017 at 4:37:44 AM UTC-4, mhh...@gmail.com wrote:
>>
>> It s awesome,
>> in my opinion, this is the way i want to consume go in the future.
>>
>> pragmatic, correct, fast, repeatable.
>>
>> in additions to what go provides,
>> fast build, cross platform, easy to package
>>
>> Although, my very personal opinion,
>> lets do smaller program that combines together,
>> in the spirit of unix tooling, rather than a
>> big boiler plate thing in the spirit of symfony
>> or alike (really just an example sorry i cited your project).
>>
>> A big Yes for such projects!
>>
>> Let s generate all the thing and get drinks to talk about it :D
>>
>> PS: i remember you talked about it earlier, shame i missed it at that
>> time.
>>
>> On Wednesday, May 10, 2017 at 6:31:22 PM UTC+2, Simon Ritchie wrote:
>>>
>>> Given a JSON description of some database tables, the scaffolder tool
>>> creates the database and generates a web application server to manage it.
>>> The resulting app server implements the Create, Read, Update and Delete
>>> (CRUD) operations on the tables.
>>>
>>> The idea for the scaffolder comes from the Ruby-on-Rails scaffold
>>> generator.
>>>
>>> The app server is presented as Go source code and HTML templates, plus
>>> some unit and integration tests.  The design follows the Model, View
>>> Controller (MVC) pattern.  The HTTP requests follow the REST pattern.
>>>
>>> The tool is here:  https://github.com/goblimey/scaffolder
>>>
>>> There are some screen shots of the resulting web pages here:
>>> http://www.goblimey.com/scaffolder/2.4.running.the.server.html
>>>
>>> Producing any web application involves a lot of boilerplate work, and
>>> this tool aims to automate some of that without imposing too many design
>>> decisions on the result.
>>>
>>> The generated web pages are fairly primitive, with very little styling.
>>> This is deliberate - if you want to use the result as a basis for building
>>> your own application, you will want to define your own styling, and the
>>> pages are structured to allow that.
>>>
>>> The material produced by the scaffolder is defined by a set of text
>>> templates.  For each table it produces from these templates a model, a
>>> controller and set of HTML templates to produce the views.  (So we have
>>> templates producing templates.)
>>>
>>> The scaffolder tool itself is very simple.  It just reads the JSON
>>> specification into a data structure, enhances that data a little and then
>>> iterates through it, supplying it to the templates.   This approach makes
>>> the tool very flexible - if it doesn't do quite what you want, it's very
>>> easy to tweak it.
>>>
>>> This idea of using a simple driver program, JSON data and templates to
>>> generate a result is very powerful.  It can be used to produce all sorts of
>>> material that follows a prototypical pattern.
>>>
>>> All comment on this project are welcome.
>>>
>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/golang-nuts/_WxygmYF1F8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Is it ok to separate Open and Close logic?

['Eric Johnson' via golang-nuts]

In this kind of scenario, I recommend a (type) wrapper for *File

Rework the code as:

type Doer *File

func NewDoer(f string) (Doer, err) {
  file, err := os.Open(f)
  // ...
}

func (d Doer) Close() error {
  return d.Close()
}


Everything else being the same, your caller can then expect "New()" 
followed by "defer Close()", so that they can do the right thing.

Eric.

On Wednesday, May 10, 2017 at 8:55:35 AM UTC-7, st ov wrote:
>
> Most examples of opening and closing a file have both calls in the same 
> function
>
> func DoFileStuff() {
>   file, _ := os.Open("file")
>   defer file.Close()
>
>   // do stuff with file
> }
>
> This makes sure any open file is closed.
> But how common is it to separate that logic into functions? 
> Should this be absolutely avoided as it could result in a file left open?
>
>
> func DoStuff(f string) {
>   file := OpenFile(f)
>
>   // call DoingStuff for some practical reason
>   DoingStuff(file)
> }
>
> func DoingStuff(f *File) {   
>// do stuff on file
>Cleanup(f)
> }
>
> func OpenFile(f string) *File {
>   file, _ := os.Open(f)
>   return file
> }
>
> func Cleanup(f *File) {
>   f.Close()
> }
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: [ANN] Ugarit

[Luis Furquim]

https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L487
> take advantage of filepath.Join() + os.Separator ? maybe.
>
> Here we cannot use os.Separator, the EPub file is a zip archive and inside 
a zip, we have to always use '/'.

 

> https://github.com/luisfurquim/ugarit/blob/master/epub30/book.go#L626
> why 59?
>
To remove the string below, which confuses some eBook readers
 


Cheers

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] [go/loader] write a conf to load faster a program ?

[mhhcbon]

Hi,

I wrote this func to load a program,
its faster but it has drawbacks about error managements.
It might hide some messages.

the first time i ran this function,
without a special typechecker error handler,
I felt overwhelmed by the amount of errors generated.

in this func s is an import path, 
and only that path should be loaded (so far).
The import is very superficial.

Is there a better/correct way to write a conf 
that achieves similar speed goal 
but with good error triage ?

Should i use another api ?

I m mostly interested into 
inspecting the definitions,
create new definitions,
update some definitions in place,
to be able to process the source even if it contains errors,
unless they are critical.

Thanks


// GetFastProgramLoader returns a fast program loader
func GetFastProgramLoader(s string) loader.Config {
var conf loader.Config
conf.ParserMode = parser.ParseComments
conf.TypeChecker.IgnoreFuncBodies = true
conf.TypeChecker.DisableUnusedImportCheck = true
conf.TypeChecker.Error = func(err error) {
if !err.(*types.Error).Soft {
panic(err)
}
if strings.Index(err.Error(), "could not import") == -1 ||
strings.Index(err.Error(), "undeclared name:") == -1 {
return
}
log.Println(err)
}

// this really matters otherise its a pain to generate a partial 
program.
conf.AllowErrors = true
originalPkgFinder := (*build.Context).Import
conf.FindPackage = func(ctxt *build.Context, fromDir, importPath string, 
mode build.ImportMode) (*build.Package, error) {
if fromDir == s {
return originalPkgFinder(ctxt, fromDir, importPath, mode)
}
return nil, fmt.Errorf("skipped %v %v", fromDir, importPath)
}
return conf
}



-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Is it ok to separate Open and Close logic?

[Christoph Berger]

Is there any reason that Cleanup(f) should be called within DoingStuff()? (A 
reason that maybe is not visible in your example but might become obvious in a 
more complex scenario?) 

Otherwise you could just call `defer Cleanup(file)` in DoStuff() with exactly 
the same result, and you are back at the same-function logic that you want to 
escape from.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Why does gofmt by default use an 8 space indent?

[hollowaykeanho]

Because it warns you that you're probably nesting too deep and should 
consider breaking them into smaller functions. It's more of a coding 
practice, not some language standards. Read #1 from 
https://www.kernel.org/doc/html/v4.10/process/coding-style.html. 


On Friday, March 7, 2014 at 10:01:19 PM UTC+8, JohnGB wrote:
>
> I can't understand what the reasoning was for an 8 space indent.  I 
> realise that this doesn't affect the functioning of any code, but it does 
> affect whether the code fits on a screen and hence the code readability.  
> Even "A tour of Go" uses 4 space indents instead of 8.
>
> So what is the advantage of 8 space indents that isn't adequately 
> addressed by 4 space indents?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Concurrent access to map

[tang . yan]

Hi,

I am aware of golang map is not safe for concurrent access, especially when 
there is at least one writer.

However, I want to know that if the map has been pre-populated, and 
different coroutines access (read/write) different key/value pair, is it 
safe to do so?  There is no deletion or adding keys after the 
initialization.

I think it should be safe (having been doing this in C++ quite frequently) 
but just want to double check.  Thanks.

Yan

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Why does gofmt by default use an 8 space indent?

[Rob Pike]

The indent character is tab, whose width is your decision. That's in large
part why the indent character is tab: to let you choose how much to indent.

-rob


On Mon, May 15, 2017 at 8:05 PM,  wrote:

> Because it warns you that you're probably nesting too deep and should
> consider breaking them into smaller functions. It's more of a coding
> practice, not some language standards. Read #1 from
> https://www.kernel.org/doc/html/v4.10/process/coding-style.html.
>
>
> On Friday, March 7, 2014 at 10:01:19 PM UTC+8, JohnGB wrote:
>>
>> I can't understand what the reasoning was for an 8 space indent.  I
>> realise that this doesn't affect the functioning of any code, but it does
>> affect whether the code fits on a screen and hence the code readability.
>> Even "A tour of Go" uses 4 space indents instead of 8.
>>
>> So what is the advantage of 8 space indents that isn't adequately
>> addressed by 4 space indents?
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Concurrent access to map

[pierre . curto]

Hello,

Once you have written out your values to the map, you can read them 
concurrently and safely. If you need to write again to it (insert or 
delete), then you must ensure it is done safely.

Note that, from what I can see in tip, in 1.9 there will be a safe 
concurrent map: https://tip.golang.org/pkg/sync/#Map.

Le mardi 16 mai 2017 07:10:56 UTC+2, Yan Tang a écrit :
>
> Hi,
>
> I am aware of golang map is not safe for concurrent access, especially 
> when there is at least one writer.
>
> However, I want to know that if the map has been pre-populated, and 
> different coroutines access (read/write) different key/value pair, is it 
> safe to do so?  There is no deletion or adding keys after the 
> initialization.
>
> I think it should be safe (having been doing this in C++ quite frequently) 
> but just want to double check.  Thanks.
>
> Yan
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Concurrent access to map

[Egon]

On Tuesday, 16 May 2017 08:10:56 UTC+3, Yan Tang wrote:
>
> Hi,
>
> I am aware of golang map is not safe for concurrent access, especially 
> when there is at least one writer.
>
> However, I want to know that if the map has been pre-populated, and 
> different coroutines access (read/write) different key/value pair, is it 
> safe to do so?  There is no deletion or adding keys after the 
> initialization.
>
> I think it should be safe (having been doing this in C++ quite frequently) 
> but just want to double check. 
>

In Go use `-race` to test safety. (https://blog.golang.org/race-detector)
In C++ use `thread-sanitizer` to test safety. 
(https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual)

Given the description, it's not safe in Go... and AFAIK, it isn't safe in 
C++ either.

See 
https://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong
 
for more information. 

+ Egon

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.