Joachim - thanks again for adding more insight, have learned (and been reminded) at lot from this thread - and certainly have a few paths forward with some suitable warnings too.
As my project is a "spare time" one, I can at least enjoy the journey and test some of this out. Tim On Wed, 7 Oct 2020, at 5:04 AM, jtuc...@objektfabrik.de wrote: > Am 06.10.20 um 22:41 schrieb Tim Mackinnon: > > Gosh - this is proving much more interesting than I had imagined, and I’m > > getting lots of useful input, so I double appreciate the time and thoughts > > from everyone. > > > > I probably should have said that my "super awesome idea" is just a little > > flashcard spelling app for my daughter (and possibly a few friends in her > > class) - so probably SimplePersistance will do in this case, > > yes, sounds like it. You mentioned it is not a super-complex 30-year > project you are planning. It sounds like you can load all the data into > memory in a few msec and just go form there, Saving the whole model as a > file also sounds like perfectly doable in such a project. So, heck, you > could probably simply just save the image and be good. A chunk of JSON, > Fuel, whatever object serialization is probably second best. > > The great thing about this: nothing external (to the image) to install > or maintain. It is just you and the file system. > > > > however the comments about making a persistence decision at the right > > moment are super interesting - and as always, its about spotting the right > > moment to do that… > > Oh no, don't fear this too much. That's not what I wanted to point out. > Most projects will do well on either NoSQL or using an ORM or an OODB. > Almost anything is possible on all of them. We do boring business stuff > (accounting) using Glorp as an ORM and DB2. Another well-known project > in the Pharo world does something extremely similar (travel cost) with > Voyage/Mongo. It is not so much a question of lost opportunities. The > problem here is that you have to deal with the very same problems > (concurrent access and isolation, fast lookups, caching, ghost objects) > on each of these technologies, but differently. It is the different > approaches to the same problems that make switching from one to the > other that makes the decision so hard. > > Example: we had a hard time finding out why some objects never went away > although we had deleted them from the database. It took a while of > logging all SQL statements until we found out first that they were > really deleted in the Transaction when the user clicked OK. But they > were inserted back in the next, possibly completely unrelated, > transaction, because we had some dangling backpointers that tricked > glorp into thinking "oops, there is this bunch of new objects I have to > insert now". This is why I used object deletion as an example. Smalltalk > has no concept of deleting an object. Databases do. So whatever approach > you chose, deleting an object looks different. You either make the > object completely (!) unreachable from the rest of the object model and > thus just make it irrelevant, or you tell the databse to just throw it > away. Sounds like no big deal. But it can be. > > > > > and ideally you have well factored code with ample tests that help you > > easily move from the idea, to something more scalable or robust… > The hard part here: the example with the reinserted objects is what is > called an unknown unknown in the boringtech site ;-) You would need a > test to check whether an object you deleted really isn't reoccuring > after the next Transaction. You'd have to imagine that there might be a > small chance that you delete an object successfully, but for some reason > you distorted the ORMs internal bookkeeping and trocked it into > inserting teh object back in some consecutive Transaction. A simple fact > once you understood why it happened, but I am 99% sure almost nobody > would come around the corner and say: well, we'll have to write a test > for this scenario where an object still lingers somewhere and gets > reinserted. The whole team would by this guy a beer and move to another > table. > > however, move too soon and you get bogged down with the details and you > > lose site of an MVP. > > move and live with your decision. Or be prepared for a much more > complicated transition than you thought. > > The problem is that nobody (at least that I am aware of) has come up > with an abstraction good enough to make the persistence implications > irrelevant enough and provide good performance and feature richness at > the same time. I know it's been tried. > > > > > > It has been interesting hearing peoples thoughts on all of this… the > > turning tide on ORM’s, the potential sweet allure of a NoSql (but can you > > query it easily) - and then the overarching element of just setting this > > shit up (where hopefully Docker steps in to make that bit at least easy). > Not sure about the need for Docker. You just throw more tech at the > problem. I mean, installing and setting up PostgreSQL or MySQL on a > Linux distro these days is matter of a few commands. apt install, enter > a db administrator password, answer a few questions and go. Same with > Mongo or CouchDB. > > I suppose this is where the Rails scaffolding was/is? such a jumpstart, > > Don't get me started ;-) > > Scaffolding is great if you need to sell a technology to management on a > few slides. Slip in some comment like "works on existing database > schemas too, at no extra cost" and these guys are ready to write > whatever it takes onto a cheque. I mean, come on, that guy showed us how > to make the whole mapping and transaction management for a flight > booking system in 15 minutes, how much harder can our project be? That's > how most multi-million desasters start. > > > > you can get a full thing going quite easily, and deploying seems relatively > > easy too… for us in Smalltalk land, its still a bit too much work for my > > liking, compared to the ease of getting an image and coding, and seeing it > > all work. > > I totally agree. These days, everything is easy. Just install NodeJS and > some super sophisticated package manager and let it install everything > for you in just 5 minutes and you're ready to go. Just make sure you > have at least 12 GB of free memory and a good machine with fast > internet. Your data will be stored in the *bling, stars and glitter * > Cloud and you just forget about this detail. You want a login screen, > sure, just type 'init -n -tfgr "login"' and provide your github > credentials. We'll create a directory structure for you with a src, an > html, a helpers and a .gitignore file. Don't worry, some of that you'll > never touch. Oh, sure just make sure you have a Facebook account and > consent with the cookie policy of that super-georgous technology company > that provides this piece of code. > > Not sure if that is what is really needed, but it's where we're heading > at the moment. Open any JS related book these days and read the first > chapter. I bet it's going to step you through the installation process > of at least three major super-cool open source, reliable and tested > frameworks or infrastructure monsters before you can start. From there > on, everything is a breeze. It's like a free lunch. > > But boy, somebody will have to run this stuff for a while. > > I think the Smalltalk vendors, both commercial and open source are > making great progress here, and I like the fact that we are not followng > the bloating trend. Maybe the "not invented here" meme makes sense to > some degree. What we as developers need is some understanding of what > we're doing and not so much the latest bells and whistles. Storing data > in a file may be unsophisticated, but it does the job even under hard > conditions. > > ..but I am getting slightly off-topic ;-) > > > Joachim > > > > > It does seem to be getting marginally better at least, but I do wish there > > was super easy setup with all the pieces nicely in place so it was just > > your idea that you could focus on… > > > > Anyway, that login screen… oh crap I have to write one of those… > > > > Tim > > > >> On 6 Oct 2020, at 20:56, jtuc...@objektfabrik.de wrote: > >> > >> Sean, > >> > >> thanks for your short overview of what SimplePersistence does. Sounds > >> useful for quite a few scenarios and might even carry you well through > >> production stages for some projects. > >> What I was talking about is also not meant to frustrate people. I've only > >> played with Mongo/Voyage for a few hours and I must say I was blown away > >> by the speed and ease of that stack. We got something running in a few > >> hours and it was impressive. So Mongo/Voyage is a cool thing to use. > >> > >> I always wanted to use Magna on Pharo. I even started to implement my own > >> little clone of Magna in VA Smalltalk. I got into troubles when I tried to > >> nest transactions and get this concurrency stuff streamlined somehow. We > >> all know that besides naming and one-off problems, caching and concurreny > >> are the hardest problems in computing. I think there is something about > >> this. And so I gave up on that project... ;-) > >> > >> In the end I went with Glorp and DB2 (soon PostgreSQL). So far I am in a > >> very solid state somewhere between complete despair and freaking out about > >> how cool things are. I love and hate that stack from the bottom of my > >> heart. > >> > >> The cool thing about an RDB is (and will sure be for quite a while) three > >> letters. S, Q and L. There are lots of highly sophisticated GUI tools to > >> query, manage, correct your data. And you can simply do everything form a > >> command prompt, in an ssh session from your smartphone in a hotel toilet > >> on the other side of the planet. > >> > >> Sure, using pure Smalltalk objects and not worry about n:m relationships, > >> not having to write mappings and not having to end up with an object model > >> that is driven mostly by what your O/R mapper can handle, sound great. And > >> it is. Until you realize you also need to think about query optimizations, > >> reorganizations, indexes and whatnot in an object database. There are also > >> compromises to make. > >> > >> But, hey, I said all of that before. > >> > >> So maybe approaches like fuel, SimplePersistence (or BOSS or Object > >> Swapper) are the best thing to start with when you need to find out about > >> your architectural and business ideas first (am I building the right > >> thing, will this feel good to a user, etc.), but once you are beyond that > >> state, you better dive into your options and decide soon. Maybe using > >> image saving or SimplePersistence is even good for production in your > >> case. It was good enough for dabbleDB for quite a while, iirc, so why > >> shouldn't it work for others? And maybe that is even the best you can do > >> to postpone the decision for as long as possible at minimum opportunity > >> cost. > >> > >> I didn't dig deep enough into Voyage/Mongo to judge how expensive or risky > >> the changes to the design are. How hard is it to restructure the root > >> trees - say you need something that is now beneath some root to be a root > >> of its own? How would you do such changes? > >> > >> I know I can do a lot of things of that kind with SQL. It is a second > >> looking glass and set of tools to view and manipulate the data. Sometimes > >> things are easier to do in Smalltalk, sometimes it is way too slow on top > >> of an ORM and a SQL query can do the same thing in a few milliseconds. > >> > >> But maybe I am asking the wrong questions fo Tim's purposes. I think I > >> understand what you (Tim) are looking for is not a big, complex project > >> but more like an experiment? I don't want to invalidate any of the given > >> suggestions, I know or at least believe that they each do a good job. All > >> I really wanted to warn you is that you will not easily be able to go from > >> one option to another, because each will have a deep impact on your object > >> model and application architecture. > >> > >> Joachim > >> > >> > >> > >> > >> Am 06.10.20 um 16:34 schrieb Sean P. DeNigris via Pharo-users: > >>> jtuchel wrote > >>>> Sigh. Forget about the idea that it will be easy to switch your > >>>> persistence later....I am not commenting on SimplePersistence here, I > >>>> don't even know what it > >>>> does or doesn't. > >>> Joachim, > >>> Thanks for this interesting perspective. I've never had the (mis?!)fortune > >>> of a project growing enough to force me to make those tough choices! For > >>> SimplePersistence I will say that I view it as a way to *delay* making > >>> *any* > >>> choices until you are forced to. It's really just a layer of sugar on top > >>> of > >>> Fuel (it used to use the old school Squeak equivalent serialization > >>> mechanism - I forget the name and that might still work). You tell it what > >>> classes to serialize. Implement two methods for each class that get and > >>> set > >>> the data, and then it saves the whole thing as one object graph. > >>> > >>> Tim, > >>> If you use SimplePersistence, please keep me posted about your experience. > >>> I'm happy to help. > >>> > >>> NB I have maintained and extended the library, but it is the work of Ramon > >>> Leon > >>> > >>> > >>> > >>> ----- > >>> Cheers, > >>> Sean > >>> -- > >>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html > >>> > >> -- > >> ----------------------------------------------------------------------- > >> Objektfabrik Joachim Tuchel mailto:jtuc...@objektfabrik.de > >> Fliederweg 1 http://www.objektfabrik.de > >> D-71640 Ludwigsburg http://joachimtuchel.wordpress.com > >> Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 > >> > > -- > ----------------------------------------------------------------------- > Objektfabrik Joachim Tuchel mailto:jtuc...@objektfabrik.de > Fliederweg 1 http://www.objektfabrik.de > D-71640 Ludwigsburg http://joachimtuchel.wordpress.com > Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 > >
