Thanks, I had completely forgotten about STON, thats a good point too (possibly this is what SimplePersistence uses as well - I'm not sure).
I recall there is a git repo for Pharo tips/tricks - I will try and curate this list over there as a few people have commented how useful this has been, and appreciate everyone who has chipped in. Tim On Thu, 8 Oct 2020, at 1:43 AM, Russ Whaley wrote: > I have been using STON for a while now. It does a really nice job of keeping > everything just the way I need it - and the read/write - even for larger > files process very quickly. Limitations are full writes, no updates, but > hey, it is incredibly easy to use and works in both v8 and v9. I built a > flashcard application in Pharo as well, and the STON files load in a fraction > of a second. > > Cheers! > Russ > > On Wed, Oct 7, 2020 at 12:04 AM jtuc...@objektfabrik.de > <jtuc...@objektfabrik.de> wrote: >> Am 06.10.20 um 22:41 schrieb Tim Mackinnon: >> >> > Gosh - this is proving much more interesting than I had imagined, and I’m >> > getting lots of useful input, so I double appreciate the time and thoughts >> > from everyone. >> >> > >> >> > I probably should have said that my "super awesome idea" is just a little >> > flashcard spelling app for my daughter (and possibly a few friends in her >> > class) - so probably SimplePersistance will do in this case, >> >> >> >> yes, sounds like it. You mentioned it is not a super-complex 30-year >> >> project you are planning. It sounds like you can load all the data into >> >> memory in a few msec and just go form there, Saving the whole model as a >> >> file also sounds like perfectly doable in such a project. So, heck, you >> >> could probably simply just save the image and be good. A chunk of JSON, >> >> Fuel, whatever object serialization is probably second best. >> >> >> >> The great thing about this: nothing external (to the image) to install >> >> or maintain. It is just you and the file system. >> >> >> >> >> >> > however the comments about making a persistence decision at the right >> > moment are super interesting - and as always, its about spotting the right >> > moment to do that… >> >> >> >> Oh no, don't fear this too much. That's not what I wanted to point out. >> >> Most projects will do well on either NoSQL or using an ORM or an OODB. >> >> Almost anything is possible on all of them. We do boring business stuff >> >> (accounting) using Glorp as an ORM and DB2. Another well-known project >> >> in the Pharo world does something extremely similar (travel cost) with >> >> Voyage/Mongo. It is not so much a question of lost opportunities. The >> >> problem here is that you have to deal with the very same problems >> >> (concurrent access and isolation, fast lookups, caching, ghost objects) >> >> on each of these technologies, but differently. It is the different >> >> approaches to the same problems that make switching from one to the >> >> other that makes the decision so hard. >> >> >> >> Example: we had a hard time finding out why some objects never went away >> >> although we had deleted them from the database. It took a while of >> >> logging all SQL statements until we found out first that they were >> >> really deleted in the Transaction when the user clicked OK. But they >> >> were inserted back in the next, possibly completely unrelated, >> >> transaction, because we had some dangling backpointers that tricked >> >> glorp into thinking "oops, there is this bunch of new objects I have to >> >> insert now". This is why I used object deletion as an example. Smalltalk >> >> has no concept of deleting an object. Databases do. So whatever approach >> >> you chose, deleting an object looks different. You either make the >> >> object completely (!) unreachable from the rest of the object model and >> >> thus just make it irrelevant, or you tell the databse to just throw it >> >> away. Sounds like no big deal. But it can be. >> >> >> >> >> >> >> >> > and ideally you have well factored code with ample tests that help you >> > easily move from the idea, to something more scalable or robust… >> >> The hard part here: the example with the reinserted objects is what is >> >> called an unknown unknown in the boringtech site ;-) You would need a >> >> test to check whether an object you deleted really isn't reoccuring >> >> after the next Transaction. You'd have to imagine that there might be a >> >> small chance that you delete an object successfully, but for some reason >> >> you distorted the ORMs internal bookkeeping and trocked it into >> >> inserting teh object back in some consecutive Transaction. A simple fact >> >> once you understood why it happened, but I am 99% sure almost nobody >> >> would come around the corner and say: well, we'll have to write a test >> >> for this scenario where an object still lingers somewhere and gets >> >> reinserted. The whole team would by this guy a beer and move to another >> >> table. >> >> > however, move too soon and you get bogged down with the details and you >> > lose site of an MVP. >> >> >> >> move and live with your decision. Or be prepared for a much more >> >> complicated transition than you thought. >> >> >> >> The problem is that nobody (at least that I am aware of) has come up >> >> with an abstraction good enough to make the persistence implications >> >> irrelevant enough and provide good performance and feature richness at >> >> the same time. I know it's been tried. >> >> >> >> >> >> > >> >> > It has been interesting hearing peoples thoughts on all of this… the >> > turning tide on ORM’s, the potential sweet allure of a NoSql (but can you >> > query it easily) - and then the overarching element of just setting this >> > shit up (where hopefully Docker steps in to make that bit at least easy). >> >> Not sure about the need for Docker. You just throw more tech at the >> >> problem. I mean, installing and setting up PostgreSQL or MySQL on a >> >> Linux distro these days is matter of a few commands. apt install, enter >> >> a db administrator password, answer a few questions and go. Same with >> >> Mongo or CouchDB. >> >> > I suppose this is where the Rails scaffolding was/is? such a jumpstart, >> >> >> >> Don't get me started ;-) >> >> >> >> Scaffolding is great if you need to sell a technology to management on a >> >> few slides. Slip in some comment like "works on existing database >> >> schemas too, at no extra cost" and these guys are ready to write >> >> whatever it takes onto a cheque. I mean, come on, that guy showed us how >> >> to make the whole mapping and transaction management for a flight >> >> booking system in 15 minutes, how much harder can our project be? That's >> >> how most multi-million desasters start. >> >> >> >> >> >> > you can get a full thing going quite easily, and deploying seems >> > relatively easy too… for us in Smalltalk land, its still a bit too much >> > work for my liking, compared to the ease of getting an image and coding, >> > and seeing it all work. >> >> >> >> I totally agree. These days, everything is easy. Just install NodeJS and >> >> some super sophisticated package manager and let it install everything >> >> for you in just 5 minutes and you're ready to go. Just make sure you >> >> have at least 12 GB of free memory and a good machine with fast >> >> internet. Your data will be stored in the *bling, stars and glitter * >> >> Cloud and you just forget about this detail. You want a login screen, >> >> sure, just type 'init -n -tfgr "login"' and provide your github >> >> credentials. We'll create a directory structure for you with a src, an >> >> html, a helpers and a .gitignore file. Don't worry, some of that you'll >> >> never touch. Oh, sure just make sure you have a Facebook account and >> >> consent with the cookie policy of that super-georgous technology company >> >> that provides this piece of code. >> >> >> >> Not sure if that is what is really needed, but it's where we're heading >> >> at the moment. Open any JS related book these days and read the first >> >> chapter. I bet it's going to step you through the installation process >> >> of at least three major super-cool open source, reliable and tested >> >> frameworks or infrastructure monsters before you can start. From there >> >> on, everything is a breeze. It's like a free lunch. >> >> >> >> But boy, somebody will have to run this stuff for a while. >> >> >> >> I think the Smalltalk vendors, both commercial and open source are >> >> making great progress here, and I like the fact that we are not followng >> >> the bloating trend. Maybe the "not invented here" meme makes sense to >> >> some degree. What we as developers need is some understanding of what >> >> we're doing and not so much the latest bells and whistles. Storing data >> >> in a file may be unsophisticated, but it does the job even under hard >> >> conditions. >> >> >> >> ..but I am getting slightly off-topic ;-) >> >> >> >> >> >> Joachim >> >> >> >> >> >> >> >> > It does seem to be getting marginally better at least, but I do wish there >> > was super easy setup with all the pieces nicely in place so it was just >> > your idea that you could focus on… >> >> > >> >> > Anyway, that login screen… oh crap I have to write one of those… >> >> > >> >> > Tim >> >> > >> >> >> On 6 Oct 2020, at 20:56, jtuc...@objektfabrik.de wrote: >> >> >> >> >> >> Sean, >> >> >> >> >> >> thanks for your short overview of what SimplePersistence does. Sounds >> >> useful for quite a few scenarios and might even carry you well through >> >> production stages for some projects. >> >> >> What I was talking about is also not meant to frustrate people. I've only >> >> played with Mongo/Voyage for a few hours and I must say I was blown away >> >> by the speed and ease of that stack. We got something running in a few >> >> hours and it was impressive. So Mongo/Voyage is a cool thing to use. >> >> >> >> >> >> I always wanted to use Magna on Pharo. I even started to implement my own >> >> little clone of Magna in VA Smalltalk. I got into troubles when I tried >> >> to nest transactions and get this concurrency stuff streamlined somehow. >> >> We all know that besides naming and one-off problems, caching and >> >> concurreny are the hardest problems in computing. I think there is >> >> something about this. And so I gave up on that project... ;-) >> >> >> >> >> >> In the end I went with Glorp and DB2 (soon PostgreSQL). So far I am in a >> >> very solid state somewhere between complete despair and freaking out >> >> about how cool things are. I love and hate that stack from the bottom of >> >> my heart. >> >> >> >> >> >> The cool thing about an RDB is (and will sure be for quite a while) three >> >> letters. S, Q and L. There are lots of highly sophisticated GUI tools to >> >> query, manage, correct your data. And you can simply do everything form a >> >> command prompt, in an ssh session from your smartphone in a hotel toilet >> >> on the other side of the planet. >> >> >> >> >> >> Sure, using pure Smalltalk objects and not worry about n:m relationships, >> >> not having to write mappings and not having to end up with an object >> >> model that is driven mostly by what your O/R mapper can handle, sound >> >> great. And it is. Until you realize you also need to think about query >> >> optimizations, reorganizations, indexes and whatnot in an object >> >> database. There are also compromises to make. >> >> >> >> >> >> But, hey, I said all of that before. >> >> >> >> >> >> So maybe approaches like fuel, SimplePersistence (or BOSS or Object >> >> Swapper) are the best thing to start with when you need to find out about >> >> your architectural and business ideas first (am I building the right >> >> thing, will this feel good to a user, etc.), but once you are beyond that >> >> state, you better dive into your options and decide soon. Maybe using >> >> image saving or SimplePersistence is even good for production in your >> >> case. It was good enough for dabbleDB for quite a while, iirc, so why >> >> shouldn't it work for others? And maybe that is even the best you can do >> >> to postpone the decision for as long as possible at minimum opportunity >> >> cost. >> >> >> >> >> >> I didn't dig deep enough into Voyage/Mongo to judge how expensive or >> >> risky the changes to the design are. How hard is it to restructure the >> >> root trees - say you need something that is now beneath some root to be a >> >> root of its own? How would you do such changes? >> >> >> >> >> >> I know I can do a lot of things of that kind with SQL. It is a second >> >> looking glass and set of tools to view and manipulate the data. Sometimes >> >> things are easier to do in Smalltalk, sometimes it is way too slow on top >> >> of an ORM and a SQL query can do the same thing in a few milliseconds. >> >> >> >> >> >> But maybe I am asking the wrong questions fo Tim's purposes. I think I >> >> understand what you (Tim) are looking for is not a big, complex project >> >> but more like an experiment? I don't want to invalidate any of the given >> >> suggestions, I know or at least believe that they each do a good job. All >> >> I really wanted to warn you is that you will not easily be able to go >> >> from one option to another, because each will have a deep impact on your >> >> object model and application architecture. >> >> >> >> >> >> Joachim >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> Am 06.10.20 um 16:34 schrieb Sean P. DeNigris via Pharo-users: >> >> >>> jtuchel wrote >> >> >>>> Sigh. Forget about the idea that it will be easy to switch your >> >> >>>> persistence later....I am not commenting on SimplePersistence here, I >> >> >>>> don't even know what it >> >> >>>> does or doesn't. >> >> >>> Joachim, >> >> >>> Thanks for this interesting perspective. I've never had the >> >>> (mis?!)fortune >> >> >>> of a project growing enough to force me to make those tough choices! For >> >> >>> SimplePersistence I will say that I view it as a way to *delay* making >> >>> *any* >> >> >>> choices until you are forced to. It's really just a layer of sugar on >> >>> top of >> >> >>> Fuel (it used to use the old school Squeak equivalent serialization >> >> >>> mechanism - I forget the name and that might still work). You tell it >> >>> what >> >> >>> classes to serialize. Implement two methods for each class that get and >> >>> set >> >> >>> the data, and then it saves the whole thing as one object graph. >> >> >>> >> >> >>> Tim, >> >> >>> If you use SimplePersistence, please keep me posted about your >> >>> experience. >> >> >>> I'm happy to help. >> >> >>> >> >> >>> NB I have maintained and extended the library, but it is the work of >> >>> Ramon >> >> >>> Leon >> >> >>> >> >> >>> >> >> >>> >> >> >>> ----- >> >> >>> Cheers, >> >> >>> Sean >> >> >>> -- >> >> >>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html >> >> >>> >> >> >> -- >> >> >> ----------------------------------------------------------------------- >> >> >> Objektfabrik Joachim Tuchel mailto:jtuc...@objektfabrik.de >> >> >> Fliederweg 1 http://www.objektfabrik.de >> >> >> D-71640 Ludwigsburg http://joachimtuchel.wordpress.com >> >> >> Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 >> >> >> >> >> >> >> -- >> >> ----------------------------------------------------------------------- >> >> Objektfabrik Joachim Tuchel mailto:jtuc...@objektfabrik.de >> >> Fliederweg 1 http://www.objektfabrik.de >> >> D-71640 Ludwigsburg http://joachimtuchel.wordpress.com >> >> Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 >> >> > -- > Russ Whaley > whaley.r...@gmail.com
