I'm now trying to compute some similarity measures but it takes a lot of time each calculation (2-17min). So, I tried to do this tunning https://www.nlm.nih.gov/research/umls/implementation_resources/scripts/README_RRF_MySQL_Output_Stream.html
in my MySQL server, as suggested in the forum https://www.mail-archive.com/[email protected]/msg00172.html After the MySQL tunning the computing speed did not improve. This is the content of my config file: SAB :: include MSH REL :: include RB, RN, PAR, CHD and this is the piece of code that I'm running: my %params = (); $params{"database"} = "umls_2015AB"; $params{"username"} = "emilio"; $params{"password"} = mypassword; $params{"hostname"} = "localhost"; $params{"port"} = "3306"; $params{"config"} = "/home/emilio/workspace/umls_relatedness/test_config_umls-interface-package.cfg"; $params{"intrinsic"} = "sanchez";
my $umls = UMLS::Interface->new(\%params);
my $lch = UMLS::Similarity::lch->new($umls,\%params);
my $lchValue = $lch->getRelatedness("C0018814", "C0003113");
I already created the umlsinterfaceindex for MSH, so I'm not running it
in realtime mode.
Since I need to compute many similarity measures in a lot of CUIs (~14,000), I wonder if I'm doing something wrong or, if not, can I do something to speed up the process.
EmilioOn 08/30/2016 06:53 PM, Albert Max Lai [email protected] [umls-similarity] wrote:
This sounds a little like what I ran into a while back (https://www.mail-archive.com/[email protected]/msg00353.html). I would make sure that the UMLS user has the INDEX privilege on the umlsinterfaceindex. The user account needs to have SELECT, INSERT, DELETE, CREATE, DROP, and INDEX privileges.Without the INDEX privilege, it seemed like the index database just kept getting bigger and bigger.-AlbertOn Aug 30, 2016, at 9:18 AM, Ted Pedersen [email protected] <mailto:[email protected]> [umls-similarity] <[email protected] <mailto:[email protected]>> wrote:Building an index for all the UMLS sources can be time consuming. I don't know that I've ever actually even done that, since some of the sources aren't probably really going to be relevant (depending on the nature of your data and experiments). Three weeks isn't totally surprisingly, although it seems long. I think I might suggest building up to 44 sources, and perhaps start with 1 or 2, see how long goes, and then try 5 or 6, etc. so that you have a sense of how long more sources should take. Also, you may want to be more selective as to which sources you include, just to keep things more efficient. I hope this helps!Good luck, TedOn Tue, Aug 30, 2016 at 6:06 AM, Emilio Centeno Ortiz [email protected] <mailto:[email protected]> [umls-similarity] <umls-similarity@ yahoogroups.com <mailto:[email protected]>> wrote:Hi Ted, We included all UMLS sources (44) in the configuration file, and the indexation process has been running for 3 weeks (still running). The umlsinterfaceindex database is currently using 117G of disk space. We included all these sources because, once the indexing is over, we want to test some similarity measures between concepts of the same semantic type for different semantic types (which implies different sources) i.e. compare diseases with diseases, compounds with compounds, etc. This way we do the indexing just once, and we can do any comparison we want without having to rebuild the umlsinterfaceindex dabatase everytime we switch sources Did I wrong including too many sources? Is the package going to work with so many sources? Thanks in advance, Emilio On 08/08/2016 04:44 AM, Ted Pedersen [email protected] <mailto:[email protected]> [umls-similarity] wrote:The time to build an index really varies quite a bit, and depends both on the hardware you have available and the particular sources and relations you are using. If possible be patient and let the index build finish, since that really does speed up queries. And I hope your index has already been built by the time you read this! Good luck, Ted On Thu, Aug 4, 2016 at 8:57 AM, Emilio Centeno Ortiz [email protected] <mailto:[email protected]> [umls-similarity] <[email protected] <mailto:[email protected]>> wrote: Hi Ted, Thanks for the reply. I finally could make it work. I didn't know that the UMLS Semantic Network was necessary in the initialization of the UMLS::Interface object. Since we don't have the Semantic Network for umls_2016AA I just switched to umls_2015AB. Now it is working. Now that I'm running a test, the package is creating an index database that, in the end, it will save time in future queries. How long this will take? I hope I will ask about more interesting issues shortly :) Thanks again, Emilio On 08/04/2016 03:30 PM, Ted Pedersen [email protected] <mailto:[email protected]> [umls-similarity] wrote:Hi Emilio, I'm afraid I haven't seen this error before. It looks like a fairly generic Perl DBI error, so I wonder if everything is working ok with that module? If possible it might be good to run the DBI tests again just to make sure that is installed and working ok. Please let us know what you find if you are able to do that... Sorry I can't be more specific, but keep us posted and we might be able to do more... Good luck, Ted On Tue, Aug 2, 2016 at 2:21 AM, Emilio Centeno Ortiz [email protected] <mailto:[email protected]> [umls-similarity] <[email protected] <mailto:[email protected]>> wrote: Hi again, Now that our sysadmin granted a full working access to the UMLS database, I'm trying to create the UMLS::Interface object: my $umls = UMLS::Interface->new({"driver" => "mysql", "database" => "umls_2016AA", "username" => "myusername", "password" => "mypassword", "hostname" => "localhost", "port" => "3306" }); But I get this output: UMLS-Interface Configuration Information: (Default Information - no config file) Sources (SAB): MSH Relations (REL): PAR CHD Sources (SABDEF): UMLS_ALL Relations (RELDEF): UMLS_ALL ERROR: UMLS::Interface::STFinder->_loadSemanticNetwork Database error (Error Code 1). Error executing database query: DBI::st=HASH(0x3b643e8)->errstr()). Any hint about this? Thanks in advance, Emilio On 07/30/2016 03:18 AM, juliana md [email protected] <mailto:[email protected]> [umls-similarity] wrote:Hi Emilio, Did you grant access to myusername@mylocalmachine to your umls_2016AA database? Are you able to connect to your database from mylocalmachine (by using workbench for example) using those credentials? Regards, Juliana Em 29 de jul de 2016 21:05, "Emilio Centeno Ortiz [email protected] <mailto:[email protected]> [umls-similarity]" <[email protected] <mailto:[email protected]>> escreveu: Hello, I have just installed the UMLS::Similarity package and copy-pasted the example code Since the MySQL is hosted in another machine (172.20.16.15) I tried to initialize the interface with our connection parameters: use UMLS::Interface; use UMLS::Similarity::lch; use UMLS::Similarity::path;$umls = UMLS::Interface->new({"driver" => "mysql","database" => "umls_2016AA", "username" => "myusername", "password" => "mypassword", "hostname" => "myMySQLHostIP", "port" => "3306"}); die "Unable to create UMLS::Interface object.\n" if(!$umls);my $lch = UMLS::Similarity::lch->new($umls);die "Unable to create measure object.\n" if(!$lch);my $path = UMLS::Similarity::path->new($umls);die "Unable to create measure object.\n" if(!$path);my $cui1 = "C0005767";my $cui2 = "C0007634";$ts1 = $umls->getTermList($cui1);my $term1 = pop @{$ts1};$ts2 = $umls->getTermList($cui2);my $term2 = pop @{$ts2};my $lvalue = $lch->getRelatedness($cui1, $cui2); my $pvalue = $path->getRelatedness($cui1, $cui2); print "The lch similarity between $cui1 ($term1) and $cui2 ($term2) is $lvalue\n"; print "The path similarity between $cui1 ($term1) and $cui2 ($term2) is $pvalue\n";but it complains like that: DBI connect('database=umls_2016AA;mysql_socket=/var/run/mysqld/mysqld.sock;host=myMySQLHostIP','myusername',...) failed: Access denied for user 'myusername'@'mylocalmachinename' (using password: YES) at /soft/devel/perl-5.16.3/lib/site_perl/5.16.3/UMLS/Interface/CuiFinder.pm line 2458. Can't call method "err" on an undefined value at /soft/devel/perl-5.16.3/lib/site_perl/5.16.3/UMLS/Interface/ErrorHandler.pm line 113. I have just replaced hostnames, user, etc. with "my..." names. It looks like it tries to connect to MySQL using sockets? Any advice about how I could overcome this issue? Thanks in advance, Emilio-- Emilio Centeno OrtizResearch Programme on Biomedical Informatics (GRIB) Department of Experimental and Health Sciences Universitat Pompeu Fabra IMIM (Hospital del Mar Medical Research Institute) C/ Dr. Aiguader, 88 Barcelona, Spain Tel.: +34 93 316 0536 <tel:%2B34%2093%20316%200536> E-mail: [email protected] <mailto:[email protected]> http://ibi.imim.es-- Emilio Centeno OrtizResearch Programme on Biomedical Informatics (GRIB) Department of Experimental and Health Sciences Universitat Pompeu Fabra IMIM (Hospital del Mar Medical Research Institute) C/ Dr. Aiguader, 88 Barcelona, Spain Tel.: +34 93 316 0536 <tel:%2B34%2093%20316%200536> E-mail: [email protected] <mailto:[email protected]> http://ibi.imim.es-- Emilio Centeno OrtizResearch Programme on Biomedical Informatics (GRIB) Department of Experimental and Health Sciences Universitat Pompeu Fabra IMIM (Hospital del Mar Medical Research Institute) C/ Dr. Aiguader, 88 Barcelona, Spain Tel.: +34 93 316 0536 <tel:%2B34%2093%20316%200536> E-mail: [email protected] <mailto:[email protected]> http://ibi.imim.es-- Emilio Centeno OrtizResearch Programme on Biomedical Informatics (GRIB) Department of Experimental and Health Sciences Universitat Pompeu Fabra IMIM (Hospital del Mar Medical Research Institute) C/ Dr. Aiguader, 88 Barcelona, Spain Tel.: +34 93 316 0536 <tel:%2B34%2093%20316%200536> E-mail: [email protected] <mailto:[email protected]> http://ibi.imim.es
-- Emilio Centeno Ortiz Research Programme on Biomedical Informatics (GRIB) Department of Experimental and Health Sciences Universitat Pompeu Fabra IMIM (Hospital del Mar Medical Research Institute) C/ Dr. Aiguader, 88 Barcelona, Spain Tel.: +34 93 316 0536 E-mail: [email protected] <mailto:[email protected]> http://ibi.imim.es
