Hi Troy, Thanks for the feedback! 😊 Yes, lazy loading in SWMgr would probably help. Looking forward to getting this improvement in place.
Regarding my original question: With my newly developed method getRepoModuleIds I can now also rather quickly (< 1s) determine the repository a module belongs to (dynamically). The performance could still be optimized further, but it’s already noticably better than by going through SWMgr construction for all remote repositories. I’m still planning to add the repo information for installed modules in my own persistence layer to further improve performance for the usecases mentioned below. I have one more question. Is there an example / easy way on how a SWModule object can be created independently of SWMgr? As of now SWMgr acts as a factory for SWModules, right? When working with specific modules, it may be enough to just create the objects directly, rather going through SWMgr? Best regards, Tobias From: Troy A. Griffitts Sent: Montag, 30. September 2019 16:29 To: SWORD Developers' Collaboration Forum Subject: Re: [sword-devel] Determining the repository a SWModule belongs to Hi Tobias, Yes, thank you for all this. We certainly could optimize SWMgr c-tor for large repos. My guess is that the creation of every SWModule for large repos is taking the time. Many impls of SWModule check their data files on c-tor. SWMgr also adds all the filters to each SWModule. All of this could be delayed until use. That's my suspicion for the speed difference. I'll keep this in mind for the next release and have a look at your code for a new method to InstallMgr. Thank you! Troy On September 30, 2019 5:13:23 AM MST, cont...@tklein.info wrote: Hi, in my Sword backend (node-sword-interface) I've now implemented a new method isModuleAvailableInRepo: bool isModuleAvailableInRepo(std::string moduleName, std::string repoName="all"); This method avoids the creation of a SWMgr instance (which is slow for large repos), but just quickly goes through the .conf files of sword::InstallSource.localShadow. It then extracts the module ids (from the first line of each .conf file) and checks whether the given module id is found in the overall list. On my Windows tablet this operation completes in 0.5s, whereas creation of SWMgr + going through the ModMap takes 5.8s. When executed for a bunch of modules at start-up this change significantly reduces start-up time of Ezra Project (especially on slower computers). The question remains whether a similar function would make sense directly in the SWORD engine. Best regards, Tobias Some links for you to see how this was implemented: bool isModuleAvailableInRepo(std::string moduleName, std::string repoName="all"); https://github.com/tobias-klein/node-sword-interface/blob/0.68.0/sword_facade.cpp#L483 std::vector<std::string> getRepoModuleIds(std::string repoName); https://github.com/tobias-klein/node-sword-interface/blob/0.68.0/sword_facade.cpp#L301 std::string getModuleIdFromFile(std::string moduleFileName); https://github.com/tobias-klein/node-sword-interface/blob/0.68.0/sword_facade.cpp#L276 Quoting cont...@tklein.info: To correct what I wrote earlier ... It seems to be the construction of SWMgr that takes some time, in my case InstallSource::getMgr(). The actual iteration over the ModMap is fast. Best regards, Tobias Quoting cont...@tklein.info: Hi, For now I'm thinking to store this information (Repository of a module) in my own application-level persistence layer. However, I'm still wondering about the performance of InstallMgr::getModuleStatus. It's not a big issue on current desktop PCs with fast SSDs. On my Windows tablet, though - it takes ~3s to call this function using the "eBible.org" repository. When calling it for all repositories to gather a complete list of modules, it takes ~6s on my Windows tablet. I guess the function walks through all the module files and a long list of files just takes its toll ... Is there a possibility for a performance improvement, though? Best regards, Tobias Quoting cont...@tklein.info: Hi Troy! I guess InstallMgr could add a line into the .conf when installing: InstalledFrom, but that could be a side load, a local repo, another SWORD application install on a local lan, or one of our remote sources which might not still host the module. That would be helpful! I have 2 use cases: 1) Show the repository that a module was installed from as part of other module meta information ("Module About Dialog") 2) Check, which repository a certain module is available from in the context of a "sync feature". In Ezra Project, I'm storing a list of installed modules and this list can be synced between different computers. My expectation as a user is that I want to work with the same set of modules whichever of my computers I'm using. The "sync feature" in Ezra Project basically checks the "list of installed modules" (from Ezra Project synced configuration, synced via Dropbox or similar service) and then automatically installs missing modules if there are any modules in the list that have not been installed locally yet. For this to work efficiently it would be good to know the source repository of a module, because otherwise I have to search all repositories for it (That's what I'm currently doing and the performance is the issue). Best regards, Tobias sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
_______________________________________________ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
