It would be an uphill task to persuade other repo managers to do that.
Sent from ProtonMail Mobile
On Mon, Sep 30, 2019 at 16:24, ref...@gmx.net <ref...@gmx.net> wrote:
> Less onerous , but of course also less accurate, we could introduce a conf
> entry which could be automatically applied during module uplpad
>
> Sent from my mobile. Please forgive shortness, typos and weird autocorrects.
>
> -------- Original Message --------
> Subject: Re: [sword-devel] Determining the repository a SWModule belongs to
> From: "Troy A. Griffitts"
> To: SWORD Developers' Collaboration Forum
> CC:
>
>> Hi Tobias,
>> Yes, thank you for all this. We certainly could optimize SWMgr c-tor for
>> large repos. My guess is that the creation of every SWModule for large repos
>> is taking the time. Many impls of SWModule check their data files on c-tor.
>> SWMgr also adds all the filters to each SWModule. All of this could be
>> delayed until use. That's my suspicion for the speed difference. I'll keep
>> this in mind for the next release and have a look at your code for a new
>> method to InstallMgr. Thank you!
>>
>> Troy
>>
>> On September 30, 2019 5:13:23 AM MST, cont...@tklein.info wrote:
>>
>>> Hi,
>>>
>>> in my Sword backend (node-sword-interface) I've now implemented a new
>>>
>>> method isModuleAvailableInRepo:
>>>
>>> bool isModuleAvailableInRepo(std::string moduleName, std::string
>>>
>>> repoName="all");
>>>
>>> This method avoids the creation of a SWMgr instance (which is slow for
>>>
>>> large repos), but just quickly goes through the .conf files of
>>>
>>> sword::InstallSource.localShadow. It then extracts the module ids
>>>
>>> (from the first line of each .conf file) and checks whether the given
>>>
>>> module id is found in the overall list.
>>>
>>> On my Windows tablet this operation completes in 0.5s, whereas
>>>
>>> creation of SWMgr + going through the ModMap takes 5.8s.
>>>
>>> When executed for a bunch of modules at start-up this change
>>>
>>> significantly reduces start-up time of Ezra Project (especially on
>>>
>>> slower computers).
>>>
>>> The question remains whether a similar function would make sense
>>>
>>> directly in the SWORD engine.
>>>
>>> Best regards,
>>>
>>> Tobias
>>>
>>> Some links for you to see how this was implemented:
>>>
>>> bool isModuleAvailableInRepo(std::string moduleName, std::string
>>>
>>> repoName="all");
>>> https://github.com/tobias-klein/node-sword-interface/blob/0.68.0/sword_facade.cpp#L483
>>>
>>> std::vector<std::string> getRepoModuleIds(std::string repoName);
>>> https://github.com/tobias-klein/node-sword-interface/blob/0.68.0/sword_facade.cpp#L301
>>>
>>> std::string getModuleIdFromFile(std::string moduleFileName);
>>> https://github.com/tobias-klein/node-sword-interface/blob/0.68.0/sword_facade.cpp#L276
>>>
>>> Quoting cont...@tklein.info:
>>>
>>>> To correct what I wrote earlier ...
>>>>
>>>> It seems to be the construction of SWMgr that takes some time, in my
>>>>
>>>> case InstallSource::getMgr().
>>>>
>>>> The actual iteration over the ModMap is fast.
>>>>
>>>> Best regards,
>>>>
>>>> Tobias
>>>>
>>>> Quoting cont...@tklein.info:
>>>>
>>>>> Hi,
>>>>>
>>>>> For now I'm thinking to store this information (Repository of a
>>>>>
>>>>> module) in my own application-level persistence layer.
>>>>>
>>>>> However, I'm still wondering about the performance of
>>>>>
>>>>> InstallMgr::getModuleStatus.
>>>>>
>>>>> It's not a big issue on current desktop PCs with fast SSDs. On my
>>>>>
>>>>> Windows tablet, though - it takes ~3s to call this function using
>>>>>
>>>>> the "eBible.org" repository. When calling it for all repositories to
>>>>>
>>>>> gather a complete list of modules, it takes ~6s on my Windows tablet.
>>>>>
>>>>> I guess the function walks through all the module files and a long
>>>>>
>>>>> list of files just takes its toll ... Is there a possibility for a
>>>>>
>>>>> performance improvement, though?
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Tobias
>>>>>
>>>>> Quoting cont...@tklein.info:
>>>>>
>>>>>> Hi Troy!
>>>>>>
>>>>>>> I guess InstallMgr could add a line into the .conf when
>>>>>>>
>>>>>>> installing: InstalledFrom, but that could be a side load, a local
>>>>>>>
>>>>>>> repo, another SWORD application install on a local lan, or one of
>>>>>>>
>>>>>>> our remote sources which might not still host the module.
>>>>>>
>>>>>> That would be helpful!
>>>>>>
>>>>>> I have 2 use cases:
>>>>>>
>>>>>> 1) Show the repository that a module was installed from as part of
>>>>>>
>>>>>> other module meta information ("Module About Dialog")
>>>>>>
>>>>>> 2) Check, which repository a certain module is available from in
>>>>>>
>>>>>> the context of a "sync feature". In Ezra Project, I'm storing a
>>>>>>
>>>>>> list of installed modules and this list can be synced between
>>>>>>
>>>>>> different computers. My expectation as a user is that I want to
>>>>>>
>>>>>> work with the same set of modules whichever of my computers I'm
>>>>>>
>>>>>> using. The "sync feature" in Ezra Project basically checks the
>>>>>>
>>>>>> "list of installed modules" (from Ezra Project synced
>>>>>>
>>>>>> configuration, synced via Dropbox or similar service) and then
>>>>>>
>>>>>> automatically installs missing modules if there are any modules in
>>>>>>
>>>>>> the list that have not been installed locally yet. For this to work
>>>>>>
>>>>>> efficiently it would be good to know the source repository of a
>>>>>>
>>>>>> module, because otherwise I have to search all repositories for it
>>>>>>
>>>>>> (That's what I'm currently doing and the performance is the issue).
>>>>>>
>>>>>> Best regards,
>>>>>>
>>>>>> Tobias
>>>>>> ---------------------------------------------------------------
>>>>>>
>>>>>> sword-devel mailing list: sword-devel@crosswire.org
>>>>>>
>>>>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>>>>>
>>>>>> Instructions to unsubscribe/change your settings at above page
>>>>>
>>>>> ---------------------------------------------------------------
>>>>>
>>>>> sword-devel mailing list: sword-devel@crosswire.org
>>>>>
>>>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>>>>
>>>>> Instructions to unsubscribe/change your settings at above page
>>>>
>>>> ---------------------------------------------------------------
>>>>
>>>> sword-devel mailing list: sword-devel@crosswire.org
>>>>
>>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>>>
>>>> Instructions to unsubscribe/change your settings at above page
>>>
>>> ---------------------------------------------------------------
>>>
>>> sword-devel mailing list: sword-devel@crosswire.org
>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>>
>>> Instructions to unsubscribe/change your settings at above page
>>
>> --
>> Sent from my Android device with K-9 Mail. Please excuse my brevity._______________________________________________
sword-devel mailing list: sword-devel@crosswire.org
http://www.crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
