[go-nuts] Re: Best way to do this in golang

2019-02-26 Thread Louki Sumirniy
When you say you only have read permissions, I assume you mean on the database server. Surely the application could generate and run a ticker or triggers to update the index when (some number of) changes are made on these tables (in some DBMS systems you could subscribe to notifications of upda

[go-nuts] Re: Best way to do this in golang

2019-02-26 Thread Louki Sumirniy
I never went that far into learning SQL, but I assume that 'LIKE' is a substring match. As I see it, the two ways to improve the speed of it are to prioritise the easier to detect, and most frequent conditions, and whenever there is a shortcut, use it. This might mean double or more as many lin

[go-nuts] Re: Best way to do this in golang

2019-02-25 Thread Tamás Gulácsi
What about combining all the urls into one big pattern, and using regex ("SIMILAR TO" or "~") ? I'm sure the number of cases in OR is the culprit of the slowness. Parallelizing may help in wall time, but the query is very inefficient, and running it 1000 times (even if only 32 concurrently) will

[go-nuts] Re: Best way to do this in golang

2019-02-25 Thread RZ
Thanks Tamas, Query itself is slow if i include all url strings. it takes about 10 mins. But when i hit one at a time, i see better response overall. Yes so was planning on running them in parallel. We only have read permissions and hence i am not allowed to add new index, but good idea, will