Hi John,
Thanks. With the improved error message I was able to pin the cause of
error better.
The issue was that I was declaring the user picture URL as a ZnURL
instead of as a string in the reStoreDefinition, but I was not telling
ReStore how to store ZnUrl objects. So I made the returning method to
give me a string instead of a ZnUrl, which is kind of counterintuitive
for the my model, but returned something that ReStore knew how to serialize.
I wonder if there is a way, maybe using something like derived keys, to
tell reStoreDefinition: execute this block instead of calling the
current method to store the value in the database. So, the
#profileImageUrl returns a ZnUrl object inside the image model, but is
stored as a string in the database without adding more reStoreDefinition
for particular objects (like a ZnUrl). Could be this possible in
someway? Am I missing something?
The objects storing is advancing now and hopefully we will have results
soon to share, instead of troubleshooting (but we'll share both anyway ;-P).
Cheers,
Offray
On 7/04/22 16:19, John Aspinall wrote:
Hi Offray - could you try modifying
SSWReStore>>createProxyForNewObject: as follows:
...
table := self tableForClass: anObject class.
"Add this ->” table isNil ifTrue: [ self error: 'cannot find table
for ', anObject class name ].
id := table idDataField ifNotNil: [ :idDataField | idDataField
accessor valueIn: anObject].
…
This should change the error message to show the class of object which
doesn’t have a valid table.
Thanks.
John
On 7 Apr 2022, at 20:29, Offray Vladimir Luna Cárdenas
<offray.l...@mutabit.com> wrote:
Hi John,
Effectively ReStore didn't was storing the table object for the
Nitter class. But after following your advice, i'm now able to debug
both the TwitterUser and the NitterUser and obtain the same result
(which makes sense as the later is inherited from the former). I
think that my error was related with the definition of the
synchronized tables, as I was not including explicitly the
NitterUser (as I thought it would suffice with the TwitterUser one).
Now it looks like this:
===
ReStore connection: (SSWSQLite3Connection on: privateDatabase fullName);
connect;
addClasses: { Tweet. "TwitterUser." NitterUser };
synchronizeAllClasses.
===
Now I can get a non nil idDataField that has the same shape when
#tableForClass: and "reStoreDefinition asTableIn: ReStore" are send
to TwitterUser and NitterUser. Despite of that I still get the
pre-commit error: "#idDataField was send to nil". Here is a
screenshot of the trace of the error and the idDataFiled seems right
(or at least equal to the same field in the TwitterUser):
https://i.imgur.com/F3UDC93.png
For the moment, I'm planing to use an extra/temporal
TwitterStoreHelper object that stores via ReStore/SQLite the objects
I'm unable to store so far, but as dictionaries of key/value helper
objects, where the value is just the STON serialization of the object
(with the inconveniences of not being able to query it properly). For
example, a 'profiles' dictionary in TwitterStoreHelper would have a
Twitter/Nitter user id as a key and its Ston representation as a
value. It's a temporal workaround, while I debug the above error further.
I'll keep you posted on the advances and regarding the booklet, I
answered you in a direct message.
Cheers,
Offray
On 7/04/22 2:59, John Aspinall wrote:
Hi Offray,
Regarding the error, I’m guessing that ReStore doesn’t have a table
object for your NitterUser class for some reason. Could you try:
myReStore tableForClass: NitterUser
…and check if this is returning nil? If so there’s an issue with
creating the table for the class; you could try stepping through
NitterUser reStoreDefinition asTableIn: myReStore
...to get an idea of what’s wrong.
If this isn’t the issue please let me know.
Regarding the manual, I’ll send this to you by mail.
Cheers.
John
On 6 Apr 2022, at 19:48, Offray Vladimir Luna Cárdenas
<offray.l...@mutabit.com> wrote:
Thanks Jhon, it definitively helps.
What I did was to extract some important metadata as slots of the
Tweet object and made explicit the authorId, which allows me to
trace tweets authorship:
===
Tweet>>class #reStoreDefinition
^ super reStoreDefinition
defineAsID: #id;
define: #text as: String;
define: #created as: String;
define: #authorId as: String;
define: #timelines as: (Dictionary of: String -> String); yourself
===
Also I'm going to put in the radar the idea of storing STON strings
for metadata residues.
On a related matter, I'm still unable to store properly the
NitterUser, which inherits from TwitterUser into SQLite and always
get a pre-commit error: "#idDataField was send to nil" when I ran
"myNitterUser store". Here is my TwitterUser storing definition:
===
TwitterUser>>class #reStoreDefinition
^ super reStoreDefinition
defineAsID: #id;
define: #userName as: String;
define: #profileImageUrl as: String;
define: #profileBio as: String;
"define: #createdAt as: String;"
yourself.
===
What I'm missing? And more importantly: how can I debug the
message, so I can asign the proper non nil object as receiver of
#idDataField ?
Finally (for now ;-) ), the ReStore manual has been and important
learning resource. Are you the author? And if so, would you be so
kind to upload the Word source code to the Documentation/ folder in
the repository under a permissive license that allow at least some
format changes? We would like to add some table of contents and
translate it to other formats, as we have done before with other
free/libre cultural works (see [1] [2]).
[1] https://mutabit.com/repos.fossil/datafem
[2] https://mutabit.com/repos.fossil/mapeda/
Thanks,
Offray
On 6/04/22 2:52, John Aspinall wrote:
Hi Offray,
You’re correct that ReStore can’t store that kind of mixed
dictionary directly. You could store the entire STON text as one
string and reify it on read, though that would mean you can’t
easily query on the metadata.
A compromise solution would be to define objects and slots for the
key data you’d need to query on and a STON string for the residue.
This could be a better solution anyway as excessive/complex
Dictionaries can be a sign that you need to define a new
class/classes. For your particular project this would depend on
how similar the metadata is between tweets - if there’s not much
commonality then a Dictionary approach may be more appropriate.
Hope this helps,
John
On 5 Apr 2022, at 20:07, Offray Vladimir Luna Cárdenas
<offray.l...@mutabit.com> wrote:
Hi all,
First of all, despite of being on a non-directly related matter
with my question, congrats of Pharo 10.
We (as now we have 2 active Smalltalkers in my country... Yay!)
are creating a civic tech project with Pharo/Lepiter and we would
like to store some Tweet metadata coming from Nitter[1]. As we're
dealing with the differences between the official Twitter API and
the unofficial Nitter one, we put the metadata we need in a
dictionary that has several kinds of objects, from ordered
collections to other dictionaries.
[1] https://nitter.net/about
Currently if we serialize a Tweet object in STON, we get this:
Tweet { #created : 'Tue, 05 Apr 2022 12:37:56 GMT', #text : '
[ANN] Pharo 10 Released: pharo.org/news/pharo10-relea…
<https://pharo.org/news/pharo10-released>
\n', #id : '1511322244353597443', #user : NitterUser { #userName
: 'pharoproject', #profileImageUrl : URL [
'http://nitter.42l.fr/pic/pbs.twimg.com/profile_images/541743734/icone-pharo-1_400x400.png'
] }, #metadata : { 'queries' : OrderedCollection [ { 'date' :
DateAndTime [ '2022-04-05T13:36:58.546011-05:00' ], 'parameters'
: 'https://nitter.42l.fr/pharoproject' } ], 'timelines' : {
'pharoproject' : '1511048498703126529' } } }
As you can see, the metadata slot contains a dictionary with
mixed classes of objects. But I read in the ReStore manual[2] (pg
14):
"""
Like other collections, the class of elements for both key and
value can be any other
persistent class, and will be the same for all elements of that
collection (except in the case of
inheritance).
"""
So, is ReStore unable to store metadata dictionaries like the one
described in the previous STON code? if this is possible, how can
I define it in the Tweet class>>reStoreDefinition?
For the moment, I'm going to create a explicit "timelines" slot
to store what was being stored at the #timelines key of the
metadata dictionary. But, as metadata increases, instead of
moving variables previously inside of a dictionary as explicit
slots of an object, I think that having a explicit way of storing
dictionaries with different kinds of objects, in contrast with
only uniform ones, would be needed (but I don't know if this is
in the design scope of ReStore).
BTW, Lepiter has allow us to build a pretty fluent interface to
browser Twitter/Nitter profiles and messages. Here it is how such
UI looks for browsing last @pharoproject tweets:
https://i.imgur.com/bxFze1g.png
Any help on how to use ReStore in storing mixed dictionaries is
appreciated.
Thanks,
Offray
