Re: Fast full-text searching in Python (job for Whoosh?)

[Peter J. Holzer]

On 2023-03-07 04:05:19 +, rbowman wrote:
> On Mon, 6 Mar 2023 21:55:37 -0500, Dino wrote:
> > ne issue that was also correctly foreseen by some is that there's going
> > to be a new request at every user key stroke. Known problem. JavaScript
> > programmers use a trick called "debounceing" to be reasonably sure that
> > the user is done typing before a request is issued:
> > 
> > https://schier.co/blog/wait-for-user-to-stop-typing-using-javascript
> 
> That could be annoying. My use case is address entry. When the user types

It can be. The delay is short but noticeable.

A somewhat smarter strategy is to send each query as soon as the user
hit the key but keep track of what you sent and received and discard
responses for obsolete requests (This is necessary because if you first
send "ma" and then "mas", the response to the first query might arrive
after the response to the second query and you don't want to display
"mansion" if the user already typed "mas".)

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fast full-text searching in Python (job for Whoosh?)

[David Lowry-Duda]

On 22:43 Sat 04 Mar 2023, Dino wrote:
How can I implement this? A library called Whoosh seems very promising 
(albeit it's so feature-rich that it's almost like shooting a fly with 
a bazooka in my case), but I see two problems:


1) Whoosh is either abandoned or the project is a mess in terms of 
community and support 
(https://groups.google.com/g/whoosh/c/QM_P8cGi4v4 ) and


2) Whoosh seems to be a Python only thing, which is great for now, 
but I wouldn't want this to become an obstacle should I need port it to 
a different language at some point.


As others have noted, it sounds like relatively straightforward 
implementations will be sufficient.


But I'll note that I use whoosh from time to time and I find it stable 
and pleasant to work with. It's true that development stopped, but it 
stopped in a very stable place. I don't recommend using whoosh here, but 
I would recommend experimenting with it more generally.


- DLD
--
https://mail.python.org/mailman/listinfo/python-list

RE: Fast full-text searching in Python (job for Whoosh?)

[avi.e.gross]

Some of the discussions here leave me confused as the info we think we got
early does not last long intact and often morphs into something else and we
find much of the discussion is misdirected or wasted.

Wouldn't it have been nice if this discussion had not started with a mention
of a package/module few have heard of along with a vague request on how best
to search for lines that match something in a file?

I still do not know enough to feel comfortable even after all this time. It
now seems to be a web-based application in which a web page wants to use
autocompletion as the user types.

So was the web page a static file that the user runs, or is it dynamically
created by something like a python program? How is the fact that a user has
typed a letter in a textbox or drop down of sorts reflected in a request
being sent to a python program to return possible choices? Is the same
process called anew each time or is it, or perhaps a group of similar
processes or threads going to stick around and be called repeatedly?

Lots of details are missing and in particular, much of what is being
described sounds like it is happening in the browser, presumably in
JavaScript. Also noted is that the first keystroke or two may return too
much data.

So does the OP still think this is a python question? So much of the
discussion sounds like it is in the browser deciding whether to wait for the
user to type more before making a request, or throwing away results of an
older request.

So my guess is that a possible design for this amount of data may simply be
to read the file into the browser at startup, or when the first letter is
typed, and do all the searches internally, perhaps cascaded as long as
backspace or editing is not used.

If the data gets much larger, of course, then using a server makes sense
albeit it need not use python unless lots more in the project is also ...

-Original Message-
From: Python-list  On
Behalf Of David Lowry-Duda
Sent: Tuesday, March 7, 2023 1:29 PM
To: python-list@python.org
Subject: Re: Fast full-text searching in Python (job for Whoosh?)

On 22:43 Sat 04 Mar 2023, Dino wrote:
>How can I implement this? A library called Whoosh seems very promising 
>(albeit it's so feature-rich that it's almost like shooting a fly with 
>a bazooka in my case), but I see two problems:
>
> 1) Whoosh is either abandoned or the project is a mess in terms of 
>community and support 
>(https://groups.google.com/g/whoosh/c/QM_P8cGi4v4 ) and
>
> 2) Whoosh seems to be a Python only thing, which is great for now, 
>but I wouldn't want this to become an obstacle should I need port it to 
>a different language at some point.

As others have noted, it sounds like relatively straightforward 
implementations will be sufficient.

But I'll note that I use whoosh from time to time and I find it stable 
and pleasant to work with. It's true that development stopped, but it 
stopped in a very stable place. I don't recommend using whoosh here, but 
I would recommend experimenting with it more generally.

- DLD
-- 
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

test

[Jim Byrnes]

haven't received anything from the list for quite awhile. Got no 
response when I tried to contact the administrator.

--
https://mail.python.org/mailman/listinfo/python-list

Re: test

[dn via Python-list]

On 08/03/2023 11.48, Jim Byrnes wrote:
haven't received anything from the list for quite awhile. Got no 
response when I tried to contact the administrator.


ACK

--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

ANN: The long white computing cloud

[dn via Python-list]

The long white computing cloud
- hybrid meeting, ie both in-person and virtual attendance
Wed 15 March, 1800 for 1830 (NZDT) = 0530 UTC


1 Cloud and Proud - Digital Sovereignty in Aotearoa
Doug Dixon, CEO of Catalyst Cloud, Aotearoa New Zealand's cloud provider.

2 Python in the cloud
How to get-started with the admin-stuff, and build a toy-application 
live-demo.

(what could possibly go wrong!?)


NB This will be a hybrid event and RSVPs are requested: 
https://www.meetup.com/nzpug-auckland/events/291789444/



All welcome!
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Winodws10 Command Prompt unresponsive to .py commands

[Thomas Gregg]

Hi,  I got python 11 to work with the esptool a few days ago.  However, I
must have something wrong, because now, when I enter any command with .py,
Windows Command Prompt just returns without doing anything.  Example

C:\Users\gregg>esptool.py version

C:\Users\gregg>

I tried to change the Windows default by filetype. but no luck.  Any
ideas?  Thanks
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fast full-text searching in Python (job for Whoosh?)

[Dino]

On 3/6/2023 11:05 PM, rbowman wrote:


It must be nice to have a server or two...


No kidding

About everything else you wrote, it makes a ton of sense, in fact it's a 
dilemma I am facing now. My back-end returns 10 entries (I am limiting 
to max 10 matches server side for reasons you can imagine).
As the user keeps typing, should I restrict the existing result set 
based on the new information or re-issue a API call to the server?
Things get confusing pretty fast for the user. You don't want too many 
cooks in kitchen, I guess.
Played a little bit with both approaches in my little application. 
Re-requesting from the server seems to win hands down in my case.
I am sure that them google engineers reached spectacular levels of UI 
finesse with stuff like this.



On Mon, 6 Mar 2023 21:55:37 -0500, Dino wrote:


https://schier.co/blog/wait-for-user-to-stop-typing-using-javascript


That could be annoying. My use case is address entry. When the user types

102 ma

the suggestions might be

main
manson
maple
massachusetts
masten

in a simple case. When they enter 's' it's narrowed down. Typically I'm
only dealing with a city or county so the data to be searched isn't huge.
The maps.google.com address search covers the world and they're also
throwing in a geographical constraint so the suggestions are applicable to
the area you're viewing.  



--
https://mail.python.org/mailman/listinfo/python-list

Re: Fast full-text searching in Python (job for Whoosh?)

[rbowman]

On Tue, 7 Mar 2023 07:33:01 -0500, Dino wrote:

> Played a little bit with both approaches in my little application.
> Re-requesting from the server seems to win hands down in my case.

That's necessary for a non-trivial data set. Assume you get 10 suggestions 
after the user type 'to'. 

today
tomorrow
tomato
tonsil
torque
totem
toad
toque
toward
touch

If the user type 'l' next and is trying for 'tolerance' you'll need a new 
set. You'll need a little refinement. If the user is a proficient typist 
and wants to type 'tolerance' they may get ahead of you. 

Another consideration is a less proficient typist or someone who can't 
spell. Again, play with maps.google.com. They're good at it. Put '123 
thomd' in the search bar. YMMV but I get 5 variations on 123 Thomas. When 
they were working down 'thompd' had zero matches so they backed up to 
'thom'.

If you play with their search they're using some more magic too.  Try '123 
ellekt'.  They may be using a variation on soundex or something more 
sophisticated. 
-- 
https://mail.python.org/mailman/listinfo/python-list

Python 3.12.0 alpha 6 released

[Thomas Wouters]

I'm pleased to announce the release of Python 3.12 alpha 6.

https://www.python.org/downloads/release/python-3120a6/


*This is an early developer preview of Python 3.12.*
Major new features of the 3.12 series, compared to 3.11

Python 3.12 is still in development. This release, 3.12.0a6 is the sixth of
seven planned alpha releases.

Alpha releases are intended to make it easier to test the current state of
new features and bug fixes and to test the release process.

During the alpha phase, features may be added up until the start of the
beta phase (2023-05-08) and, if necessary, may be modified or deleted up
until the release candidate phase (2023-07-31). Please keep in mind that
this is a preview release and its use is not recommended for production
environments.

Many new features for Python 3.12 are still being planned and written.
Among the new major new features and changes so far:


   - Even more improved error messages. More exceptions potentially caused
   by typos now make suggestions to the user.
   - Support for the Linux perf profiler to report Python function names in
   traces.
   - The deprecated wstr and wstr_length members of the C implementation of
   unicode objects were removed, per PEP 623
   .
   - In the unittest module, a number of long deprecated methods and
   classes were removed. (They had been deprecated since Python 3.1 or 3.2).
   - The deprecated smtpd and distutils modules have been removed (see PEP
   594  and PEP 632
   . The setuptools package (installed by
   default in virtualenvs and many other places) continues to provide the
   distutils module.
   - A number of other old, broken and deprecated functions, classes and
   methods have been removed.
   - Invalid backslash escape sequences in strings now warn with
   SyntaxWarning instead of DeprecationWarning, making them more visible.
   (They will become syntax errors in the future.)
   - The internal representation of integers has changed in preparation for
   performance enhancements. (This should not affect most users as it is an
   internal detail, but it may cause problems for Cython-generated code.)
   - (Hey, fellow core developer, if a feature you find important is
   missing from this list, let Thomas know .)

For more details on the changes to Python 3.12, see What's new in Python
3.12. The next pre-release of Python 3.12 will be 3.12.0a7, currently
scheduled for 2023-04-03.

More resources

Online Documentation .
PEP 693 , the Python 3.12 Release
Schedule.
Report bugs via GitHub Issues .
Help fund Python and its community .

And now for something completely different

Let me not to the marriage of true minds
> Admit impediments. Love is not love
> Which alters when it alteration finds,
> Or bends with the remover to remove:
> O, no! it is an ever-fixed mark,
> That looks on tempests and is never shaken;
> It is the star to every wandering bark,
> Whose worth’s unknown, although his height be taken.
> Love’s not Time’s fool, though rosy lips and cheeks
> Within his bending sickle’s compass come;
> Love alters not with his brief hours and weeks,
> But bears it out even to the edge of doom.


> If this be error, and upon me prov’d,
> I never writ, nor no man ever lov’d.


*Sonnet 116*, by William Shakespeare.

Enjoy the new releases

Thanks to all of the many volunteers who help make Python Development and
these releases possible! Please consider supporting our efforts by
volunteering yourself or through organization contributions to the Python
Software Foundation.

Regards from unexpectedly chilly California,

Your release team,
Thomas Wouters
Ned Deily
Steve Dower

-- 
Thomas Wouters 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fast full-text searching in Python (job for Whoosh?)

[Thomas Passin]

On 3/7/2023 7:33 AM, Dino wrote:

It must be nice to have a server or two...


No kidding

About everything else you wrote, it makes a ton of sense, in fact it's a 
dilemma I am facing now. My back-end returns 10 entries (I am limiting 
to max 10 matches server side for reasons you can imagine).
As the user keeps typing, should I restrict the existing result set 
based on the new information or re-issue a API call to the server?
Things get confusing pretty fast for the user. You don't want too many 
cooks in kitchen, I guess.
Played a little bit with both approaches in my little application. 
Re-requesting from the server seems to win hands down in my case.
I am sure that them google engineers reached spectacular levels of UI 
finesse with stuff like this.


Subject of course to trying this out, I would be inclined to send a much 
larger list of responses to the client, and let the client reduce the 
number to be displayed.  The latency for sending a longer list will be 
smaller than establishing a new connection or even reusing an old one to 
send a new, short list of responses.  When the client types more, it can 
only reduce the number of possibilities - among the (possibly imaginary) 
larger original number of them. After the next round of user typing, the 
client can check and see if there are enough surviving responses to 
list.  If not, it can then request a new list from the server.


Using this in reverse, if the user deletes some characters from the end, 
there should be no need to go back to the server.  The possible 
responses would already have been sent to the client.  They could be 
interned in an associative array keyed by the string the client had 
typed to get those responses.


--
https://mail.python.org/mailman/listinfo/python-list

Re: Winodws10 Command Prompt unresponsive to .py commands

[Thomas Passin]

On 3/7/2023 2:31 PM, Thomas Gregg wrote:

Hi,  I got python 11 to work with the esptool a few days ago.  However, I
must have something wrong, because now, when I enter any command with .py,
Windows Command Prompt just returns without doing anything.  Example

C:\Users\gregg>esptool.py version

C:\Users\gregg>

I tried to change the Windows default by filetype. but no luck.  Any
ideas?  Thanks


Never mind file associations for now.  Just run it with Python (that is, 
the python executable program) -


python esptool.py

Notes:
1.  If esptool is not in the current directory then use the full path to it.
2. Use the right name (instead of just "python") to launch your desired 
version of Python.  On a standard Windows installation it will probably 
be "py" (without the quotes), or "python3.11" or "py -3.11".  But it 
might possibly be just "python".  You will have to try it to see.


If you ever do decide you want to run Python programs using file 
associations, you can read up on how to get them working.


Probably the easiest way to get a file association set up is to open the 
File Explorer and navigate to some directory that has a .py file.  Right 
click on that file and choose OpenWith.  Then click on "Choose another 
app".  This will open a dialog that will let you navigate to your Python 
executable program and assign it to run all .py files.


I don't really recommend setting up and depending on a file association, 
though.  That's because it can be a problem if and when you end up with 
more than one version of Python on your computer.  Sometimes you want to 
use one version to run a Python file, sometimes another. It's hard to 
make that work if you run them using file associations.


Also if anything goes wrong, you probably will not see the error messages.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Winodws10 Command Prompt unresponsive to .py commands

[Mike Dewhirst]

On 8/03/2023 6:31 am, Thomas Gregg wrote:

Hi,  I got python 11 to work with the esptool a few days ago.  However, I
must have something wrong, because now, when I enter any command with .py,
Windows Command Prompt just returns without doing anything.  Example

C:\Users\gregg>esptool.py version

C:\Users\gregg>

I tried to change the Windows default by filetype. but no luck.  Any
ideas?  Thanks


Prove it works in the command prompt by using the full path to python 
for example,


C:\Python311\python C:\Users\gregg>esptool.py version

If that works, look at your path environment variables and see where 
python is sitting. Substitute the real location of Python 3.11 on your 
machine. Sadly, I think it defaults to C:\Program Files nowadays.


Once you can get Python working by omitting the path, for example,

C:\Users\gregg>python esptool.py version

... you can then right-click the .py file and choose whichever program 
you like to open with.


I'm inclined to advise you to focus on getting virtual environments 
working next and leave all that auto-opening to later. You might find 
you prefer to right click and select every time.


M

--
Signed email is an absolute defence against phishing. This email has
been signed with my private key. If you import my public key you can
automatically decrypt my signature and be sure it came from me. Your
email software can handle signing.



OpenPGP_signature
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Winodws10 Command Prompt unresponsive to .py commands

[Barry]

> On 8 Mar 2023, at 05:55, Mike Dewhirst  wrote:
> 
> On 8/03/2023 6:31 am, Thomas Gregg wrote:
>> Hi,  I got python 11 to work with the esptool a few days ago.  However, I
>> must have something wrong, because now, when I enter any command with .py,
>> Windows Command Prompt just returns without doing anything.  Example
>> 
>> C:\Users\gregg>esptool.py version
>> 
>> C:\Users\gregg>
>> 
>> I tried to change the Windows default by filetype. but no luck.  Any
>> ideas?  Thanks
> 
> Prove it works in the command prompt by using the full path to python for 
> example,
> 
> C:\Python311\python C:\Users\gregg>esptool.py version

On windows use the py.exe command rather then python.exe.

Does `py esptool.py version’
> 
> If that works, look at your path environment variables and see where python 
> is sitting. Substitute the real location of Python 3.11 on your machine. 
> Sadly, I think it defaults to C:\Program Files nowadays.
> 
> Once you can get Python working by omitting the path, for example,
> 
> C:\Users\gregg>python esptool.py version
> 
> ... you can then right-click the .py file and choose whichever program you 
> like to open with.

Use py.exe which should in the windows folder to run .py files.

Barry

> I'm inclined to advise you to focus on getting virtual environments working 
> next and leave all that auto-opening to later. You might find you prefer to 
> right click and select every time.
> 
> M
> 
> -- 
> Signed email is an absolute defence against phishing. This email has
> been signed with my private key. If you import my public key you can
> automatically decrypt my signature and be sure it came from me. Your
> email software can handle signing.
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list