As I said, it is a common problem and readily solved if you know how to use the ORM properly.
This is in Java, but the technique is the same https://medium.com/geekculture/resolve-hibernate-n-1-problem-f0e049e689ab > On Dec 21, 2024, at 10:40 AM, Luca Pascali <pascko...@gmail.com> wrote: > > The first one, yes. > > The other 10k no, if they are called after in a loop in the user language > that runs over every row returned by the first query. > Like it is done in a lot of REST interfaces > > That is the difference I was pointing out. > > In fact I also said that in case of simple interfaces ORM does its work well. > > If I got you wrong, please show me some example that can optimize the other > queries after returning the first result set. > > PSK > > On Sat, Dec 21, 2024 at 2:06 PM Robert Engels <reng...@ix.netcom.com > <mailto:reng...@ix.netcom.com>> wrote: >> This is incorrect. Most ORM will efficiently execute the first query with a >> single SQL statement. It’s called the N+1 Select problem and it is >> solvable. >> >>> On Dec 21, 2024, at 3:43 AM, Luca Pascali <pascko...@gmail.com >>> <mailto:pascko...@gmail.com>> wrote: >>> >>> >>> In my experience, ORM has a structural flaw, and it cannot be otherwise (as >>> well as a lot of automatic tools) >>> >>> Using ORM you are driven to write queries like (pseudo go code) >>> >>> objects := Table.filter(some rules) >>> for o := range objects { >>> if o.field == 5 { >>> sub := Table2.filter(some other rules) >>> } >>> } >>> >>> >>> this cannot be optimized a lot by compiler or ORM engine, because queries >>> are separated. >>> So having 10k elements on first query can bring up to 10k subqueries and >>> the (almost) only optimization that the engine can operate is to cache >>> queries result to choose if using an old result (can bring to other issues >>> like memory usage or data retention) or ask to the db (lots, lots, lots of >>> queries) >>> >>> If you manage to reduce 10k automatically generated queries to just 1 >>> handcraft query, you can save hundreds of seconds of execution time. Sorry >>> for stating the obvious. >>> >>> Another aspect that magnifies the problem is that ORM engines creates SQL >>> code that must be parsed by the db server, and this time is definetly not >>> zero. >>> And not all db servers have the possibility to precompile queries. >>> >>> Again the obvious: 1ms over 1 query is not a problem. 1ms over 10k queries >>> takes 10 seconds (plus the execution time, plus the parsing time to build >>> object representation, plus ... ). >>> >>> So ORM can be the wrong choice,but >>> If you don't have a complex db structure (this means a few connected >>> tables) and/or you can use some sort of advanced merges where you can >>> specify the projection in the orm itlself (that's something like writing a >>> raw query) ORM is a fantastic abstraction, because it allows you to write >>> simpler code. >>> >>> In conclusion, it is not ORM that is slow. >>> Sometimes it is just not the right tool. >>> >>> If you need some simple access proceed with ORM. If you need something more >>> complex, optimize the code to reduce db access to the essential (prefetch >>> as much as you can) and write your own queries. In that case sqlc is perfect >>> >>> My 2¢ >>> >>> PSK >>> >>> >>> On Fri, Dec 20, 2024 at 6:54 PM Kerem Üllenoğlu <kerem.ul...@gmail.com >>> <mailto:kerem.ul...@gmail.com>> wrote: >>>> Hey, >>>> >>>> I keep hearing ORMs - especially GORM - is slow. But I have never felted >>>> like it was slow and used in a couple of projects now. Is there a good >>>> article you recommend reading about ORMs in Go - especially GORM - being >>>> slow? >>>> >>>> Sorry, my reply is not actually an answer to the original question. But I >>>> am really curious about it now. Is it really important, really slow? >>>> >>>> Thanks, >>>> Kerem >>>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to golang-nuts+unsubscr...@googlegroups.com >>> <mailto:golang-nuts+unsubscr...@googlegroups.com>. >>> To view this discussion visit >>> https://groups.google.com/d/msgid/golang-nuts/CAJNBTxKjFUr-XYU4qC%3DRb6mvFmhQuRQ%2BmiA%3DyFxd6bypnCzLkg%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/golang-nuts/CAJNBTxKjFUr-XYU4qC%3DRb6mvFmhQuRQ%2BmiA%3DyFxd6bypnCzLkg%40mail.gmail.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/7C866A40-D889-4678-9F65-D3CD1F4FF7DC%40ix.netcom.com.
