Hi Offray,
There isn’t currently a formal way to do the automatic transformation you
describe (it would be a nice enhancement though). What I have done in similar
situations is to carry out the transformation in inst var accessors, e.g.
userPictureURL
^userPictureURL ifNotNil: [ZnUrl fromString: userPictureURL]
userPictureURL: aZnUrl
userPictureURL := aZnUrl pathPrintString “not sure if this is the
correct conversion but you get the idea”
You then define: #userPictureUrl as: String. This works because ReStore
accesses instance variables directly via instVarAt:[put:], but obviously
depends on your own code using accessors rather than direct references.
Hope this helps.
John
> On 12 Apr 2022, at 18:32, Offray Vladimir Luna Cárdenas
> <offray.l...@mutabit.com> wrote:
>
> Hi John,
>
> Thanks. With the improved error message I was able to pin the cause of error
> better.
> The issue was that I was declaring the user picture URL as a ZnURL instead of
> as a string in the reStoreDefinition, but I was not telling ReStore how to
> store ZnUrl objects. So I made the returning method to give me a string
> instead of a ZnUrl, which is kind of counterintuitive for the my model, but
> returned something that ReStore knew how to serialize.
>
> I wonder if there is a way, maybe using something like derived keys, to tell
> reStoreDefinition: execute this block instead of calling the current method
> to store the value in the database. So, the #profileImageUrl returns a ZnUrl
> object inside the image model, but is stored as a string in the database
> without adding more reStoreDefinition for particular objects (like a ZnUrl).
> Could be this possible in someway? Am I missing something?
>
> The objects storing is advancing now and hopefully we will have results soon
> to share, instead of troubleshooting (but we'll share both anyway ;-P).
>
> Cheers,
>
> Offray
> On 7/04/22 16:19, John Aspinall wrote:
>> Hi Offray - could you try modifying SSWReStore>>createProxyForNewObject: as
>> follows:
>>
>> ...
>> table := self tableForClass: anObject class.
>> "Add this ->” table isNil ifTrue: [ self error: 'cannot find table for ',
>> anObject class name ].
>> id := table idDataField ifNotNil: [ :idDataField | idDataField
>> accessor valueIn: anObject].
>> …
>>
>> This should change the error message to show the class of object which
>> doesn’t have a valid table.
>>
>> Thanks.
>>
>> John
>>
>>
>>> On 7 Apr 2022, at 20:29, Offray Vladimir Luna Cárdenas
>>> <offray.l...@mutabit.com <mailto:offray.l...@mutabit.com>> wrote:
>>>
>>> Hi John,
>>>
>>> Effectively ReStore didn't was storing the table object for the Nitter
>>> class. But after following your advice, i'm now able to debug both the
>>> TwitterUser and the NitterUser and obtain the same result (which makes
>>> sense as the later is inherited from the former). I think that my error was
>>> related with the definition of the synchronized tables, as I was not
>>> including explicitly the NitterUser (as I thought it would suffice with
>>> the TwitterUser one). Now it looks like this:
>>>
>>> ===
>>> ReStore connection: (SSWSQLite3Connection on: privateDatabase fullName);
>>> connect;
>>> addClasses: { Tweet. "TwitterUser." NitterUser };
>>> synchronizeAllClasses.
>>> ===
>>> Now I can get a non nil idDataField that has the same shape when
>>> #tableForClass: and "reStoreDefinition asTableIn: ReStore" are send to
>>> TwitterUser and NitterUser. Despite of that I still get the pre-commit
>>> error: "#idDataField was send to nil". Here is a screenshot of the trace of
>>> the error and the idDataFiled seems right (or at least equal to the same
>>> field in the TwitterUser):
>>> https://i.imgur.com/F3UDC93.png <https://i.imgur.com/F3UDC93.png>
>>> For the moment, I'm planing to use an extra/temporal TwitterStoreHelper
>>> object that stores via ReStore/SQLite the objects I'm unable to store so
>>> far, but as dictionaries of key/value helper objects, where the value is
>>> just the STON serialization of the object (with the inconveniences of not
>>> being able to query it properly). For example, a 'profiles' dictionary in
>>> TwitterStoreHelper would have a Twitter/Nitter user id as a key and its
>>> Ston representation as a value. It's a temporal workaround, while I debug
>>> the above error further.
>>> I'll keep you posted on the advances and regarding the booklet, I answered
>>> you in a direct message.
>>>
>>> Cheers,
>>>
>>> Offray
>>> On 7/04/22 2:59, John Aspinall wrote:
>>>> Hi Offray,
>>>>
>>>> Regarding the error, I’m guessing that ReStore doesn’t have a table object
>>>> for your NitterUser class for some reason. Could you try:
>>>>
>>>> myReStore tableForClass: NitterUser
>>>>
>>>> …and check if this is returning nil? If so there’s an issue with creating
>>>> the table for the class; you could try stepping through
>>>>
>>>> NitterUser reStoreDefinition asTableIn: myReStore
>>>>
>>>> ...to get an idea of what’s wrong.
>>>>
>>>> If this isn’t the issue please let me know.
>>>>
>>>> Regarding the manual, I’ll send this to you by mail.
>>>>
>>>> Cheers.
>>>>
>>>> John
>>>>
>>>>
>>>>
>>>>> On 6 Apr 2022, at 19:48, Offray Vladimir Luna Cárdenas
>>>>> <offray.l...@mutabit.com <mailto:offray.l...@mutabit.com>> wrote:
>>>>>
>>>>> Thanks Jhon, it definitively helps.
>>>>>
>>>>> What I did was to extract some important metadata as slots of the Tweet
>>>>> object and made explicit the authorId, which allows me to trace tweets
>>>>> authorship:
>>>>>
>>>>> ===
>>>>> Tweet>>class #reStoreDefinition
>>>>> ^ super reStoreDefinition
>>>>> defineAsID: #id;
>>>>> define: #text as: String;
>>>>> define: #created as: String;
>>>>> define: #authorId as: String;
>>>>> define: #timelines as: (Dictionary of: String -> String); yourself
>>>>>
>>>>> ===
>>>>> Also I'm going to put in the radar the idea of storing STON strings for
>>>>> metadata residues.
>>>>>
>>>>> On a related matter, I'm still unable to store properly the NitterUser,
>>>>> which inherits from TwitterUser into SQLite and always get a pre-commit
>>>>> error: "#idDataField was send to nil" when I ran "myNitterUser store".
>>>>> Here is my TwitterUser storing definition:
>>>>>
>>>>> ===
>>>>> TwitterUser>>class #reStoreDefinition
>>>>> ^ super reStoreDefinition
>>>>> defineAsID: #id;
>>>>> define: #userName as: String;
>>>>> define: #profileImageUrl as: String;
>>>>> define: #profileBio as: String;
>>>>> "define: #createdAt as: String;"
>>>>> yourself.
>>>>>
>>>>> ===
>>>>>
>>>>> What I'm missing? And more importantly: how can I debug the message, so I
>>>>> can asign the proper non nil object as receiver of #idDataField ?
>>>>>
>>>>> Finally (for now ;-) ), the ReStore manual has been and important
>>>>> learning resource. Are you the author? And if so, would you be so kind to
>>>>> upload the Word source code to the Documentation/ folder in the
>>>>> repository under a permissive license that allow at least some format
>>>>> changes? We would like to add some table of contents and translate it to
>>>>> other formats, as we have done before with other free/libre cultural
>>>>> works (see [1] [2]).
>>>>>
>>>>> [1] https://mutabit.com/repos.fossil/datafem
>>>>> <https://mutabit.com/repos.fossil/datafem>
>>>>> [2] https://mutabit.com/repos.fossil/mapeda/
>>>>> <https://mutabit.com/repos.fossil/mapeda/>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Offray
>>>>> On 6/04/22 2:52, John Aspinall wrote:
>>>>>> Hi Offray,
>>>>>>
>>>>>> You’re correct that ReStore can’t store that kind of mixed dictionary
>>>>>> directly. You could store the entire STON text as one string and reify
>>>>>> it on read, though that would mean you can’t easily query on the
>>>>>> metadata.
>>>>>>
>>>>>> A compromise solution would be to define objects and slots for the key
>>>>>> data you’d need to query on and a STON string for the residue. This
>>>>>> could be a better solution anyway as excessive/complex Dictionaries can
>>>>>> be a sign that you need to define a new class/classes. For your
>>>>>> particular project this would depend on how similar the metadata is
>>>>>> between tweets - if there’s not much commonality then a Dictionary
>>>>>> approach may be more appropriate.
>>>>>>
>>>>>> Hope this helps,
>>>>>>
>>>>>> John
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 5 Apr 2022, at 20:07, Offray Vladimir Luna Cárdenas
>>>>>>> <offray.l...@mutabit.com <mailto:offray.l...@mutabit.com>> wrote:
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> First of all, despite of being on a non-directly related matter with my
>>>>>>> question, congrats of Pharo 10.
>>>>>>>
>>>>>>> We (as now we have 2 active Smalltalkers in my country... Yay!) are
>>>>>>> creating a civic tech project with Pharo/Lepiter and we would like to
>>>>>>> store some Tweet metadata coming from Nitter[1]. As we're dealing with
>>>>>>> the differences between the official Twitter API and the unofficial
>>>>>>> Nitter one, we put the metadata we need in a dictionary that has
>>>>>>> several kinds of objects, from ordered collections to other
>>>>>>> dictionaries.
>>>>>>> [1] https://nitter.net/about <https://nitter.net/about>
>>>>>>> Currently if we serialize a Tweet object in STON, we get this:
>>>>>>>
>>>>>>> Tweet { #created : 'Tue, 05 Apr 2022 12:37:56 GMT', #text : '
>>>>>>>
>>>>>>> [ANN] Pharo 10 Released: pharo.org/news/pharo10-relea…
>>>>>>> <https://pharo.org/news/pharo10-released>\n', #id :
>>>>>>> '1511322244353597443', #user : NitterUser { #userName : 'pharoproject',
>>>>>>> #profileImageUrl : URL [
>>>>>>> 'http://nitter.42l.fr/pic/pbs.twimg.com/profile_images/541743734/icone-pharo-1_400x400.png
>>>>>>>
>>>>>>> <http://nitter.42l.fr/pic/pbs.twimg.com/profile_images/541743734/icone-pharo-1_400x400.png>'
>>>>>>> ] }, #metadata : { 'queries' : OrderedCollection [ { 'date' :
>>>>>>> DateAndTime [ '2022-04-05T13:36:58.546011-05:00' ], 'parameters' :
>>>>>>> 'https://nitter.42l.fr/pharoproject
>>>>>>> <https://nitter.42l.fr/pharoproject>' } ], 'timelines' : {
>>>>>>> 'pharoproject' : '1511048498703126529' } } }
>>>>>>> As you can see, the metadata slot contains a dictionary with mixed
>>>>>>> classes of objects. But I read in the ReStore manual[2] (pg 14):
>>>>>>>
>>>>>>> """
>>>>>>>
>>>>>>> Like other collections, the class of elements for both key and value
>>>>>>> can be any other
>>>>>>> persistent class, and will be the same for all elements of that
>>>>>>> collection (except in the case of
>>>>>>> inheritance).
>>>>>>> """
>>>>>>> So, is ReStore unable to store metadata dictionaries like the one
>>>>>>> described in the previous STON code? if this is possible, how can I
>>>>>>> define it in the Tweet class>>reStoreDefinition?
>>>>>>> For the moment, I'm going to create a explicit "timelines" slot to
>>>>>>> store what was being stored at the #timelines key of the metadata
>>>>>>> dictionary. But, as metadata increases, instead of moving variables
>>>>>>> previously inside of a dictionary as explicit slots of an object, I
>>>>>>> think that having a explicit way of storing dictionaries with different
>>>>>>> kinds of objects, in contrast with only uniform ones, would be needed
>>>>>>> (but I don't know if this is in the design scope of ReStore).
>>>>>>>
>>>>>>> BTW, Lepiter has allow us to build a pretty fluent interface to browser
>>>>>>> Twitter/Nitter profiles and messages. Here it is how such UI looks for
>>>>>>> browsing last @pharoproject tweets:
>>>>>>>
>>>>>>> https://i.imgur.com/bxFze1g.png <https://i.imgur.com/bxFze1g.png>
>>>>>>> Any help on how to use ReStore in storing mixed dictionaries is
>>>>>>> appreciated.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Offray
>>>>>>
>>>>
>>
