I think we are talking at cross purposes - I thought you were talking about delaying the mutation until the 'main' logic has finished, whereby the main logic would push the mutation into a queue which were then executed later. Something like queueing up a bunch of commands to be executed later.
If you are simply talking about isolating/abstracting database logic (but not the _timeliness_ of when that logic is executed) from the mainline code. The Repository pattern, DataAccess pattern then yes, of course, I think we are all on the same page. The original point was do you/don't you pass in the DB as an explicit arg. I still haven't seen a recommendation for handling protocols whose implementation is persistence agnostic. I also think you might be making a few unwarranted assumptions ;-) - who mentioned building up SQL in raw strings?! for example. James Gatannah writes: > On Monday, August 3, 2015 at 5:19:34 AM UTC-5, Colin Yates wrote: >> >> I have heard this approach before, but I have never seen how it works in >> real life. > > > Interesting. I don't think I've ever worked anywhere that didn't take this > kind of approach. Even if we weren't doing it formally. > > > >> For example, what about 'selects' - where do they happen? >> > > They're just part of the overall transaction description. > > It's [thankfully] been a long time since I worked on a project that worked > in raw strings > to build SQL. Even working at that level, the SELECT pieces were just > a subquery in the main query we were building. > > The web server(s) would pre-process the incoming request, hand that > to the middle-tier layers that did this sort of business logic, those would > pass > the data description (in this case raw SQL, in others something like > XML-RPC) > off to the servers that actually have pools of database connections. > > Even in companies that only needed one virtual host for their tiny little > web > server and couldn't be bothered to break it into multiple processes, we > split > this out into multiple logical layers. > > >> What about if my updates are not independent (e.g. if the first >> update works then do the second update otherwise do this completely >> different thing?). >> > > We're working in lisp. This sort of thing is far easier than it is for the > poor schmucks who are doing exactly this sort of thing in, say, C# with > LINQ or building/parsing/translating XML using python. > > > For simple workflows I can see the elegance. For non-trivial workflows >> the problem is that _accessing_ the DB and _switching_ on the DB logic >> tends to be all through the workflow. >> > > I'll argue that this sort of isolation gets more important when your system > gets more complex. You don't build a modern automatic transmission with > sloppier tolerances that were used for a manual 50 years ago. You spend > more time engineering the Eiffel Tower than you would a back-yard shed. > You don't skimp on O-rings when you're building a space shuttle. > > > >> I am genuinely interested in the answers to this as yes, the described >> approach has some great benefits. >> > > Let's see. Specific example... > > This comes from a startup that was bought for IP around 2001. I think it's > safe to describe what I remember of their general architecture, since it > seems > to be extinct. > > Big picture was warehouses where workers are loading up pallets of goods to > be shipped to retail stores. > > End users wore computers that had headsets and mics. Input was by > speech recognition, output through text-to-speech. The wearables didn't have > enough horse-power to actually do the speech recognition, so compressed .wav > files were transmitted to the server over wifi. > > We probably should have been using a web interface, but most web apps were > written in perl. This was a serious technology shop using <s>C++</s>MFC. > > We had a ton of different modules/classes that covered different > possibilities for > any given scenario at various customer sites. They got initialized and wired > together by rules defined in an .ini file. (That should have come from a > database, > but that's a different topic). > > Most of our database interaction happened when a worker logged in: we cached > the various phrases that would be used throughout her shift. > > And we'd select a list of work. As each work item was completed, the > front-end > code would notify our database layer. I don't have any idea how that > notification > worked. It went through some library routine that had a database connection > buried far away from any front-end code. > > "Our" database was really just a cache for interaction phrases in various > languages > and the work queues. > > Periodically, some job would upload our work status reports and download new > pending work queues to/from the "real" database. IIRC, this interaction > happened > over FTP. We absolutely did not have (or want!) handles to work directly > with > whatever ancient server that was their authoritative source of all > knowledge. > > Our portion of the system was ~70-80 KLOC. Since it was C++ (and most of > that was boilerplate), it was *far* simpler (in terms of business logic) > than the 200 KLOC of clojure that we're talking about. Although the big iron > side may have pushed it into that realm. > > You obviously aren't going to be able to predict every possible interaction > in > the next few minutes. Much less the next 4 hours. But you should be able to > predict > most of what might happen for any given web request. > > Life changes when you're working on a highly interactive app like, say, an > IDE > or a photo editor. At my job today we have issues with multiple users adding > and removing the same artifact to a document at something resembling the > same > time. > > That problem has a lot to do with the fact that it's a distributed system > with at > least (I haven't dug into the tier beneath the web server) 4 layers between > the > end-users and the actual database connections. I'm not sure it would be > possible > to get rid of more than one of those layers to get us closer to the > database. > > I'm absolutely positive that it would not be desirable. > > My "database connection" consists of data structures that I put onto and > read > from a message queue. > > This system is about as far from functional purity as you can possibly get. > > It's quite possible that the database guys at the very bottom of the stack > are > using exactly the kind of global connections that were originally suggested. > > The difference is that, even if they are (and I certainly hope not!), I > can't touch > them. When (and it's never "if") a hacker gets control over the front-end > web > server, that's just the first layer of defense. > > Regards, > James > > > >> >> James Gatannah writes: >> >> > On Thursday, July 30, 2015 at 7:44:31 PM UTC-5, J. Pablo Fernández >> wrote: >> >> >> >> Hello Clojurians, >> >> >> >> I found passing around the database connection to each function that >> uses >> >> it very error prone when you are using transactions as passing the >> wrong >> >> one could mean a query runs outside the transaction when in the source >> code >> >> it is inside the with-db-transaction function. >> >> >> > >> > I'll go ahead and make the point that it's error-prone for different >> > reasons. >> > >> > Pretty much by definition, that database connection is a system >> boundary. >> > It's all about something that's *way* more complex than random global >> state >> > changes inside your program. This is a thing that interacts with the >> > outside world, with all the nastiness that implies. >> > >> > Everything that everyone else has already written about this approach is >> > true, but I don't think they've gone far enough. >> > >> > Even if you pass that database connection around as a parameter >> everywhere, >> > you're talking about throwing away a huge part of the benefit of using a >> > functional language. >> > >> > Isolate your side-effects. >> > >> > Think of a castle. You have a moat surrounding it, and a few gates that >> you >> > use to allow your peasants to enter/exit. This particular gate opens up >> to >> > a swamp full of alligators. >> > >> > Your approach amounts to letting the gators wander around loose. >> > >> > Passing the connection around to each function in the call chain is like >> > tying a ribbon around the gator's neck and hoping you can use that as a >> > leash. >> > >> > You can use either approach to great effect. If you're really, really >> good. >> > And so is everyone else on your team (you did mention a 200 KLOC >> project). >> > >> > One of the main benefits to functional programming is that admitting you >> > aren't really, really good is incredibly liberating. I don't have the >> > time/energy to dedicate to trying to maintain this sort of code. (Yes, I >> > spent lots of time recently thinking about how java was designed for >> very >> > average programmers, but it really takes much better programmers than a >> > functional language to actually write correct programs). Even if I were >> > that good...I'd rather be focused on the problems that make my customers >> > happy. >> > >> > I'm going to appeal to authority here for the right >> > answer: http://prog21.dadgum.com/23.html (in my defense, it's a great >> > blog). Have your web response handler (which is another system >> > boundary...this one is next to an active volcano populated by >> > fire-breathing dragons) build up a list of all the nasty side-effects >> that >> > will eventually have to happen. >> > >> > Don't just isolate your side-effects. Quarantine those suckers as if >> each >> > and every one means you're dealing with the most diabolical hacker you >> can >> > imagine. >> >> -- >> Sent with my mu4e >> -- Sent with my mu4e -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
