Hi On Tue, Jun 5, 2018 at 6:25 PM, Dave Page <dp...@pgadmin.org> wrote:
> > > On Tue, Jun 5, 2018 at 1:49 PM, Aditya Toshniwal <aditya.toshniwal@ > enterprisedb.com> wrote: > >> Hi Dave, >> >> The problem of SQL ASCII is solved with the patch, and not related to >> setting the client encoding of the sql window. >> > > No it's not. It doesn't work for me as I said (and showed the example of). > After setting the client_encoding to SQL_ASCII you got the output. Previously, it used to fail in the back end itself because python encoding failure. That is fixed. The error ERROR: invalid byte sequence for encoding "UTF8": 0x80 is thrown by postgres and not python or pgAdmin4. You will get the same error even if you connect from psql. > > >> I can see there is no SET call in pgAdmin3 for client_encoding. I can >> remove the SET client_encoding='UNICODE'; that will solve the problem. >> But, can you please let me know why that was added. >> > > There is, but it's inside an API call (PQsetClientEncoding): > > 300 > <https://git.postgresql.org/gitweb/?p=pgadmin3.git;a=blob;f=pgadmin/db/pgConn.cpp#l300> > wxLogInfo(wxT("Setting client_encoding to '% > s'"), encoding.c_str()); > 301 > <https://git.postgresql.org/gitweb/?p=pgadmin3.git;a=blob;f=pgadmin/db/pgConn.cpp#l301> > if (PQsetClientEncoding(conn, encoding.ToAscii())) > 302 > <https://git.postgresql.org/gitweb/?p=pgadmin3.git;a=blob;f=pgadmin/db/pgConn.cpp#l302> > { > 303 > <https://git.postgresql.org/gitweb/?p=pgadmin3.git;a=blob;f=pgadmin/db/pgConn.cpp#l303> > wxLogError(wxT("%s"), GetLastError().c_str()); > 304 > <https://git.postgresql.org/gitweb/?p=pgadmin3.git;a=blob;f=pgadmin/db/pgConn.cpp#l304> > } > 305 > <https://git.postgresql.org/gitweb/?p=pgadmin3.git;a=blob;f=pgadmin/db/pgConn.cpp#l305> > > Oops ! Missed that. Apologies. > > >> >> Will remove the set call and will send you the updated patch if >> everything works fine. >> > > No, we need to ensure the client encoding is set correctly. It just needs > to be set to SQL_ASCII if it's a SQL_ASCII database (I believe). > > Need to rework on the initialise method. Will come with an updated. patch. Sorry for trouble. > >> >> On Tue, Jun 5, 2018 at 6:05 PM, Dave Page <dp...@pgadmin.org> wrote: >> >>> Hi >>> >>> On Tue, Jun 5, 2018 at 1:21 PM, Aditya Toshniwal < >>> aditya.toshni...@enterprisedb.com> wrote: >>> >>>> Hi Dave, >>>> >>>> >>>> On Tue, Jun 5, 2018 at 4:56 PM, Dave Page <dp...@pgadmin.org> wrote: >>>> >>>>> Hi >>>>> >>>>> On Tue, Jun 5, 2018 at 9:50 AM, Aditya Toshniwal < >>>>> aditya.toshni...@enterprisedb.com> wrote: >>>>> >>>>>> Hi Hackers, >>>>>> >>>>>> PFA updated patch. The sqleditor change is sent separately and >>>>>> removed from current patch as suggested. >>>>>> The test cases were running fine when the module was specified using >>>>>> --pkg but were failing in complete run. Fixed that. >>>>>> >>>>> >>>>> I did a quick test by creating a SQL_ASCII database containing a >>>>> simple table: >>>>> >>>>> CREATE TABLE sql_ascii (id serial primary key, data text); >>>>> >>>>> And then populated it with data: >>>>> >>>>> /Library/PostgreSQL/9.4/bin/psql -d sql_ascii -U postgres -c "INSERT >>>>> INTO sql_acsii (data) VALUES ('[Windows-1252] Euro: \x80 Double >>>>> dagger: >>>>> \x87');" >>>>> /Library/PostgreSQL/9.4/bin/psql -d sql_ascii -U postgres -c "INSERT >>>>> INTO sql_ascii (data) VALUES ('[Latin-1] Yen: \xa5 Half: \xbd');" >>>>> /Library/PostgreSQL/9.4/bin/psql -d sql_ascii -U postgres -c "INSERT >>>>> INTO sql_ascii (data) VALUES ('[Japanese] Ship: \xe8\x88\xb9');" >>>>> /Library/PostgreSQL/9.4/bin/psql -d sql_ascii -U postgres -c "INSERT >>>>> INTO sql_ascii (data) VALUES ('[Invalid UTF-8] Blob: \xf4\xa5\xa3\xa5');" >>>>> >>>>> I then right-clicked the table in the treeview, and selected the >>>>> option to view all rows, and immediately saw an error: >>>>> >>>>> 2018-06-05 12:23:27,319: SQL pgadmin: Execute (async) for server #1 - >>>>> CONN:1187535 (Query-id: 8522474): >>>>> SELECT * FROM public.sql_ascii >>>>> ORDER BY id ASC >>>>> 2018-06-05 12:23:27,320: ERROR pgadmin: Failed to execute query >>>>> (execute_async) for the server #1 - CONN:1187535(Query-id: 8522474): >>>>> Error Message:ERROR: invalid byte sequence for encoding "UTF8": 0x80 >>>>> SQL state: 22021 >>>>> >>>>> Running "SELECT * FROM sql_ascii" in the query tool resulted in the >>>>> same error, however, if I ran "SET client_encoding = 'SQL_ASCII';" first, >>>>> I >>>>> do see results. >>>>> >>>>> I have confirmed that I've restarted the server after applying the >>>>> patch. >>>>> >>>>> What am I missing? Why don't we just set the client_encoding to >>>>> SQL_ASCII if it's a SQL_ASCII database? >>>>> >>>> >>>> It is by default same as the server encoding. But, the following >>>> existing code in web/pgadmin/utils/driver/psycopg2/connection.py makes >>>> the client_encoding as UNICODE for every connection. I am not sure it >>>> should be removed. >>>> >>>> status = _execute(cur, "SET DateStyle=ISO;" >>>> >>>> "SET client_min_messages=notice;" >>>> >>>> "SET bytea_output=escape;" >>>> >>>> "SET client_encoding='UNICODE';") >>>> >>> >>> It was probably before you joined, but I have said a number of times >>> that pgAdmin 3 handled this differently and that maybe we should do it the >>> same way here. See https://git.postgresql.org >>> /gitweb/?p=pgadmin3.git;a=blob;f=pgadmin/db/pgConn.cpp, in the >>> pgConn::Initialize() function. >>> >>> Either way, your patch isn't working for me. >>> >>> >>>> >>>> >>>> Note that this testing was on Python 2.7.10 on MacOS. >>>>> >>>>> >>>>>> >>>>>> Kindly review. >>>>>> >>>>>> Thanks and Regards, >>>>>> Aditya Toshniwal >>>>>> Software Engineer | EnterpriseDB Software Solutions | Pune >>>>>> "Don't Complain about Heat, Plant a tree" >>>>>> >>>>>> On Tue, Jun 5, 2018 at 10:15 AM, Aditya Toshniwal < >>>>>> aditya.toshni...@enterprisedb.com> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> On Tue, Jun 5, 2018 at 1:08 AM, Joao De Almeida Pereira < >>>>>>> jdealmeidapere...@pivotal.io> wrote: >>>>>>> >>>>>>>> Hello Aditya, >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> There is no change related to notifications in this patch. >>>>>>>>> The below code is minor fix related to connection status of sql >>>>>>>>> editor. Can you please share the code snippet if it is not the below. >>>>>>>>> >>>>>>>>> - # Check for the asynchronous notifies statements. >>>>>>>>> - conn.check_notifies(True) >>>>>>>>> - notifies = conn.get_notifies() >>>>>>>>> + if status is not None: >>>>>>>>> + # Check for the asynchronous notifies statements. >>>>>>>>> + conn.check_notifies(True) >>>>>>>>> + notifies = conn.get_notifies() >>>>>>>>> >>>>>>>>> >>>>>>>> This is a minor fix, but is it related to querying SQL_ASCII >>>>>>>> database? >>>>>>>> >>>>>>> No its not. It is something I found when I was working on SQL_ASCII >>>>>>> related changes. >>>>>>> Well then, will send a separate patch for it. >>>>>>> >>>>>>>> >>>>>>>> Thanks >>>>>>>> Victoria && Joao >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Dave Page >>>>> Blog: http://pgsnake.blogspot.com >>>>> Twitter: @pgsnake >>>>> >>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>>> >>>> >>>> >>> >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >> >> > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company >
