[pgadmin4][Patch]: Feature #3388 Allow to set the connection timeout on the server properties dialog

[Akshay Joshi]

Hi Hackers,

Attached is the patch for Feature #3388 Allow to set the connection timeout
on the server properties dialog. Please review it.


-- 
*Akshay Joshi*

*Sr. Software Architect *



*Phone: +91 20-3058-9517Mobile: +91 976-788-8246*


RM_3388.patch
Description: Binary data

Re: [pgAdmin4][RM#3289] Can't query SQL_ASCII database.

[Aditya Toshniwal]

Hi Hackers,

Attached is the updated patch which includes the fix for Download CSV fail
in SQL_ASCII database, which is RM3250

This should fix RM3289 and RM3250. As they interrelated, sending the
combined patch.
Kindly review.

On Fri, Jun 15, 2018 at 2:33 PM, Aditya Toshniwal <
aditya.toshni...@enterprisedb.com> wrote:

> Hi Dave,
>
> On Fri, Jun 15, 2018 at 2:08 PM, Dave Page  wrote:
>
>> Hi
>>
>> On Thu, Jun 14, 2018 at 7:05 AM, Aditya Toshniwal <
>> aditya.toshni...@enterprisedb.com> wrote:
>>
>>> I am sorry I missed the attachment. :(
>>> PFA.
>>>
>>
>> It looks like the encoding names are getting munged somewhere. I see
>> you've accounted for that to some degree in connection.py (you have both
>> SQL_ASCII/MULE_INTERNAL and SQLASCII/MULEINTERNAL), however it doesn't seem
>> to be enough as I'm getting the following error when trying to download CSV
>> from the query tool. Can we ensure that conn.encoding contains an un-munged
>> value at all times, or is that coming from psycopg2?
>>
> ​That is done by pyscopg2 and conn.encoding is a psycopg2 connection
> property.​
>
>
>>
>> 2018-06-15 09:32:28,799: INFO werkzeug: 127.0.0.1 - - [15/Jun/2018
>> 09:32:28] "GET /sqleditor/query_tool/download/2732923?query=SELECT%20*%
>> 20FROM%20public.sql_ascii%0AORDER%20BY%20id%20ASC%20&filename=sql_ascii.csv
>> HTTP/1.1" 500 -
>> 2018-06-15 09:32:28,801: ERROR werkzeug: Error on request:
>> Traceback (most recent call last):
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
>> line 270, in run_wsgi
>> execute(self.server.app)
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
>> line 260, in execute
>> for data in application_iter:
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/wsgi.py",
>> line 870, in __next__
>> return self._next()
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/wrappers.py",
>> line 82, in _iter_encoded
>> for item in iterable:
>>   File 
>> "/Users/dpage/git/pgadmin4/web/pgadmin/utils/driver/psycopg2/connection.py",
>> line 752, in gen
>> column_name = column_name.decode(conn_encoding)
>> LookupError: unknown encoding: SQLASCII
>>
>
> ​This is because there is code bug here. Below is code used to decode a
> column name. Connection encoding and python encoding are two different
> things. Python does not know what SQLASCII is. This will work with UTF-8
> because python has decoder with same name. I tried to download CSV with the
> original code without changes and it fails there too. I will fix this and
> will send the updated patch. I should have checked this.
> conn_encoding = cur.connection.encoding
> column_name = column_name.decode(conn_encoding)​
>
>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>
>
>
> --
> Thanks and Regards,
> Aditya Toshniwal
> Software Engineer | EnterpriseDB Software Solutions | Pune
> "Don't Complain about Heat, Plant a tree"
>



-- 
Thanks and Regards,
Aditya Toshniwal
Software Engineer | EnterpriseDB Software Solutions | Pune
"Don't Complain about Heat, Plant a tree"


RM3289_3250.patch
Description: Binary data

[pgadmin4][Patch]: RM #3361 Create database dialog does not show template0 and template1 database in Template dropdown

[Akshay Joshi]

Hi Hackers,

Attached is the patch for RM #3361 Create database dialog does not show
template0 and template1 database in Template drop down.

-- 
*Akshay Joshi*

*Sr. Software Architect *



*Phone: +91 20-3058-9517Mobile: +91 976-788-8246*


RM_3361.patch
Description: Binary data

Build failed in Jenkins: pgadmin4-master-python33 #663

[pgAdmin 4 Jenkins]

See 


--
[...truncated 8.51 KB...]
  Cache entry deserialization failed, entry ignored
Collecting Flask-Paranoid==0.2.0 (from -r requirements.txt (line 30))
  Cache entry deserialization failed, entry ignored
  Using cached 
https://files.pythonhosted.org/packages/0e/14/359033a218c5403872e11ff7267e6c638c0279067fa932d93653188f4052/Flask_Paranoid-0.2.0-py2.py3-none-any.whl
Collecting psycopg2>=2.7.4 (from -r requirements.txt (line 35))
  Cache entry deserialization failed, entry ignored
  Using cached 
https://files.pythonhosted.org/packages/b2/c1/7bf6c464e903ffc4f3f5907c389e5a4199666bf57f6cd6bf46c17912a1f9/psycopg2-2.7.5.tar.gz
Collecting python-dateutil>=2.7.1 (from -r requirements.txt (line 36))
  Cache entry deserialization failed, entry ignored
  Using cached 
https://files.pythonhosted.org/packages/cf/f5/af2b09c957ace60dcfac112b669c45c8c97e32f94aa8b56da4c6d1682825/python_dateutil-2.7.3-py2.py3-none-any.whl
Collecting htmlmin==0.1.12 (from -r requirements.txt (line 37))
  Cache entry deserialization failed, entry ignored
Collecting Flask-HTMLmin==1.3.2 (from -r requirements.txt (line 38))
  Cache entry deserialization failed, entry ignored
Collecting SQLAlchemy>=1.2.5 (from -r requirements.txt (line 39))
  Cache entry deserialization failed, entry ignored
Collecting Flask-Security>=3.0.0 (from -r requirements.txt (line 40))
  Cache entry deserialization failed, entry ignored
  Using cached 
https://files.pythonhosted.org/packages/88/47/4908a5040120768ff4fb2465c7eeafeb9239c27d2919bd67c4ccc1b43e14/Flask_Security-3.0.0-py2.py3-none-any.whl
Collecting sshtunnel>=0.1.3 (from -r requirements.txt (line 41))
  Cache entry deserialization failed, entry ignored
Collecting testtools>=0.9.22 (from fixtures==3.0.0->-r requirements.txt (line 
8))
  Cache entry deserialization failed, entry ignored
  Using cached 
https://files.pythonhosted.org/packages/87/74/a4d55da28d7bba6d6f49430f22a62afd8472cb24a63fa61daef80d3e821b/testtools-2.3.0-py2.py3-none-any.whl
Collecting Werkzeug>=0.7 (from Flask==0.12.2->-r requirements.txt (line 9))
  Cache entry deserialization failed, entry ignored
  Using cached 
https://files.pythonhosted.org/packages/20/c4/12e3e56473e52375aa29c4764e70d1b8f3efa6682bef8d0aae04fe335243/Werkzeug-0.14.1-py2.py3-none-any.whl
Collecting Jinja2>=2.4 (from Flask==0.12.2->-r requirements.txt (line 9))
  Cache entry deserialization failed, entry ignored
  Using cached 
https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl
Collecting itsdangerous>=0.21 (from Flask==0.12.2->-r requirements.txt (line 9))
  Cache entry deserialization failed, entry ignored
Collecting click>=2.0 (from Flask==0.12.2->-r requirements.txt (line 9))
  Cache entry deserialization failed, entry ignored
  Using cached 
https://files.pythonhosted.org/packages/34/c1/8806f99713ddb993c5366c362b2f908f18269f8d792aff1abfd700775a77/click-6.7-py2.py3-none-any.whl
Collecting alembic>=0.6 (from Flask-Migrate==2.1.1->-r requirements.txt (line 
13))
  Cache entry deserialization failed, entry ignored
Collecting webencodings (from html5lib==1.0.1->-r requirements.txt (line 17))
  Cache entry deserialization failed, entry ignored
  Using cached 
https://files.pythonhosted.org/packages/f4/24/2a3e3df732393fed8b3ebf2ec078f05546de641fe1b667ee316ec1dcf3b7/webencodings-0.5.1-py2.py3-none-any.whl
Collecting Flask-BabelEx>=0.9.3 (from Flask-Security>=3.0.0->-r 
requirements.txt (line 40))
  Cache entry deserialization failed, entry ignored
Collecting paramiko>=1.15.2 (from sshtunnel>=0.1.3->-r requirements.txt (line 
41))
  Cache entry deserialization failed, entry ignored
  Using cached 
https://files.pythonhosted.org/packages/3e/db/cb7b6656e0e7387637ce850689084dc0b94b44df31cc52e5fc5c2c4fd2c1/paramiko-2.4.1-py2.py3-none-any.whl
Collecting unittest2>=1.0.0 (from testtools>=0.9.22->fixtures==3.0.0->-r 
requirements.txt (line 8))
  Cache entry deserialization failed, entry ignored
  Using cached 
https://files.pythonhosted.org/packages/72/20/7f0f433060a962200b7272b8c12ba90ef5b903e218174301d0abfd523813/unittest2-1.1.0-py2.py3-none-any.whl
Collecting traceback2 (from testtools>=0.9.22->fixtures==3.0.0->-r 
requirements.txt (line 8))
  Cache entry deserialization failed, entry ignored
  Using cached 
https://files.pythonhosted.org/packages/17/0a/6ac05a3723017a967193456a2efa0aa9ac4b51456891af1e2353bb9de21e/traceback2-1.4.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from Jinja2>=2.4->Flask==0.12.2->-r 
requirements.txt (line 9))
  Cache entry deserialization failed, entry ignored
Collecting Mako (from alembic>=0.6->Flask-Migrate==2.1.1->-r requirements.txt 
(line 13))
  Cache entry deserialization failed, entry ignored
Collecting python-editor>=0.3 (from alembic>=0.6->Flask-Migrate==2.1.1->-r 
requirements.txt (line 13))
  Cache entry deserialization failed, 

pgAdmin 4 commit: README update

[Dave Page]

README update

Branch
--
master

Details
---
https://git.postgresql.org/gitweb?p=pgadmin4.git;a=commitdiff;h=521985c11a3101745e5255dc2c6a531e9cd160cc
Author: Khushboo Vashi 

Modified Files
--
README | 1 +
1 file changed, 1 insertion(+)

Re: [pgAdmin4][Patch]: Improve the database migration steps in README

[Dave Page]

Thanks, applied.

On Mon, Jun 18, 2018 at 7:26 AM, Khushboo Vashi <
khushboo.va...@enterprisedb.com> wrote:

> Hi,
>
> Please find the attached minor patch to improve the database migration
> steps in README.
>
> Thanks,
> Khushboo
>



-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Jenkins build is back to normal : pgadmin4-master-python33 #664

[pgAdmin 4 Jenkins]

See 

Re: [pgadmin4][Patch]: Feature #3388 Allow to set the connection timeout on the server properties dialog

[Dave Page]

Hi

On Mon, Jun 18, 2018 at 12:18 PM, Akshay Joshi <
akshay.jo...@enterprisedb.com> wrote:

> Hi Hackers,
>
> Attached is the patch for Feature #3388 Allow to set the connection
> timeout on the server properties dialog. Please review it.
>

I took a quick look at this and had some questions/comments:

- Doc updates seem to be missing.

- The timeout should be passed through to external utilities like
pg_dump/pg_restore.

- Can/should the timeout be extended to the SSH tunnels?

- Any refactoring opportunities in the code you're touching?

Thanks.

-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

pgAdmin 4 commit: Ensure that template0 and template1 are available as

[Dave Page]

Ensure that template0 and template1 are available as tempates when creating 
databasesa. Fixes #3361

[regardless of the Show System Objects? option]

Branch
--
master

Details
---
https://git.postgresql.org/gitweb?p=pgadmin4.git;a=commitdiff;h=a45c33cf76707700a4106645146d135a91095161
Author: Akshay Joshi 

Modified Files
--
.../server_groups/servers/databases/__init__.py | 21 -
.../servers/databases/static/js/database.js |  2 +-
2 files changed, 21 insertions(+), 2 deletions(-)

pgAdmin 4 commit: Fix indent.

[Dave Page]

Fix indent.

Branch
--
master

Details
---
https://git.postgresql.org/gitweb?p=pgadmin4.git;a=commitdiff;h=717637c5b90c34b7cba2772ff190bb08cdd72c00

Modified Files
--
.../browser/server_groups/servers/databases/static/js/database.js   | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Re: [pgadmin4][Patch]: RM #3361 Create database dialog does not show template0 and template1 database in Template dropdown

[Dave Page]

Thanks, applied.

On Mon, Jun 18, 2018 at 2:38 PM, Akshay Joshi  wrote:

> Hi Hackers,
>
> Attached is the patch for RM #3361 Create database dialog does not show
> template0 and template1 database in Template drop down.
>
> --
> *Akshay Joshi*
>
> *Sr. Software Architect *
>
>
>
> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246*
>



-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4][RM#3289] Can't query SQL_ASCII database.

[Dave Page]

Hi

On Mon, Jun 18, 2018 at 2:14 PM, Aditya Toshniwal <
aditya.toshni...@enterprisedb.com> wrote:

> Hi Hackers,
>
> Attached is the updated patch which includes the fix for Download CSV fail
> in SQL_ASCII database, which is RM3250
> 
> This should fix RM3289 and RM3250. As they interrelated, sending the
> combined patch.
> Kindly review.
>

This is definitely looking better - both view and save now work as
expected. However, using the test data the I posted upthread, if I try to
edit a value (in this case by adding a couple of chars to the end of the
data in row 2) I get:

2018-06-18 16:41:40,895: SQL pgadmin: Execute (void) for server #1 -
DB:ascii (Query-id: 3093186):
UPDATE public.ascii SET
data = %(data)s::text WHERE
id = '2';
2018-06-18 16:41:41,027: INFO werkzeug: 127.0.0.1 - - [18/Jun/2018
16:41:41] "POST /sqleditor/save/2805058 HTTP/1.1" 500 -
2018-06-18 16:41:41,042: ERROR werkzeug: Error on request:
Traceback (most recent call last):
  File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
line 270, in run_wsgi
execute(self.server.app)
  File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
line 258, in execute
application_iter = app(environ, start_response)
  File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1997, in __call__
return self.wsgi_app(environ, start_response)
  File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1985, in wsgi_app
response = self.handle_exception(e)
  File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1540, in handle_exception
reraise(exc_type, exc_value, tb)
  File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1982, in wsgi_app
response = self.full_dispatch_request()
  File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
  File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
  File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1612, in full_dispatch_request
rv = self.dispatch_request()
  File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1598, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
  File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py",
line 792, in decorated_view
return func(*args, **kwargs)
  File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py",
line 776, in save
default_conn)
  File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/sqleditor/command.py",
line 829, in save
item['sql'], item['data'])
  File
"/Users/dpage/git/pgadmin4/web/pgadmin/utils/driver/psycopg2/connection.py",
line 975, in execute_void
self.__internal_blocking_execute(cur, query, params)
  File
"/Users/dpage/git/pgadmin4/web/pgadmin/utils/driver/psycopg2/connection.py",
line 629, in __internal_blocking_execute
cur.execute(query, params)
  File
"/Users/dpage/git/pgadmin4/web/pgadmin/utils/driver/psycopg2/cursor.py",
line 176, in execute
return _cursor.execute(self, query, params)
UnicodeEncodeError: 'ascii' codec can't encode characters in position
19-21: ordinal not in range(128)


>
> On Fri, Jun 15, 2018 at 2:33 PM, Aditya Toshniwal  enterprisedb.com> wrote:
>
>> Hi Dave,
>>
>> On Fri, Jun 15, 2018 at 2:08 PM, Dave Page  wrote:
>>
>>> Hi
>>>
>>> On Thu, Jun 14, 2018 at 7:05 AM, Aditya Toshniwal <
>>> aditya.toshni...@enterprisedb.com> wrote:
>>>
 I am sorry I missed the attachment. :(
 PFA.

>>>
>>> It looks like the encoding names are getting munged somewhere. I see
>>> you've accounted for that to some degree in connection.py (you have both
>>> SQL_ASCII/MULE_INTERNAL and SQLASCII/MULEINTERNAL), however it doesn't seem
>>> to be enough as I'm getting the following error when trying to download CSV
>>> from the query tool. Can we ensure that conn.encoding contains an un-munged
>>> value at all times, or is that coming from psycopg2?
>>>
>> ​That is done by pyscopg2 and conn.encoding is a psycopg2 connection
>> property.​
>>
>>
>>>
>>> 2018-06-15 09:32:28,799: INFO werkzeug: 127.0.0.1 - - [15/Jun/2018
>>> 09:32:28] "GET /sqleditor/query_tool/download
>>> /2732923?query=SELECT%20*%20FROM%20public.sql_ascii%0AORDER%
>>> 20BY%20id%20ASC%20&filename=sql_ascii.csv HTTP/1.1" 500 -
>>> 2018-06-15 09:32:28,801: ERROR werkzeug: Error on request:
>>> Traceback (most recent call last):
>>>   File 
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
>>> line 270, in run_wsgi
>>> execute(self.server.app)
>>>   File 
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/pytho