Hi,
Please find the attached patch to fix below issues:
1. #2963 - Backup database, Restore database and Maintenance Database
failed for é object
2. #3157 - Process viewer doesn't show complete command executed.
Test cases are not included for these fixes as we don't have test cases for
these modules (backup, restore, maintenance).
I will create one separate RM for the same which will cover this.
Thanks,
Khushboo
On Fri, Jan 12, 2018 at 3:24 PM, Dave Page <dp...@pgadmin.org> wrote:
> Hi
>
> On Wed, Jan 10, 2018 at 6:59 AM, Khushboo Vashi <
> khushboo.va...@enterprisedb.com> wrote:
>
>>
>>
>> On Wed, Jan 3, 2018 at 8:54 PM, Dave Page <dp...@pgadmin.org> wrote:
>>
>>> Hi
>>>
>>> On Mon, Dec 25, 2017 at 11:03 AM, Khushboo Vashi <
>>> khushboo.va...@enterprisedb.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Please find the attached patch to fix RM #2963: Backup database,
>>>> Restore database and Maintenance Database failed for é object.
>>>>
>>>
>>> With the patch applied, I get:
>>>
>>> 2018-01-03 15:23:00,110: INFO pgadmin: Executing the process executor
>>> with the arguments: ['/Users/dpage/.virtualenvs/pgadmin4/bin/python',
>>> '/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/process_executor.py',
>>> '/usr/local/pgsql/bin/pg_dump', u'--file', u'/Users/dpage/e.sql',
>>> u'--host', u'localhost', u'--port', '5432', u'--username', u'postgres',
>>> u'--no-password', u'--verbose', u'--format=c', u'--blobs', u'\xe9']
>>> 2018-01-03 15:23:00,117: INFO werkzeug: 127.0.0.1 - - [03/Jan/2018
>>> 15:23:00] "POST /backup/job/1/object HTTP/1.1" 200 -
>>> Exception in thread Thread-6:
>>> Traceback (most recent call last):
>>> File
>>> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py",
>>> line 810, in __bootstrap_inner
>>> self.run()
>>> File
>>> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py",
>>> line 763, in run
>>> self.__target(*self.__args, **self.__kwargs)
>>> File
>>> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
>>> line 602, in process_request_thread
>>> self.handle_error(request, client_address)
>>> File
>>> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
>>> line 599, in process_request_thread
>>> self.finish_request(request, client_address)
>>> File
>>> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
>>> line 334, in finish_request
>>> self.RequestHandlerClass(request, client_address, self)
>>> File
>>> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
>>> line 655, in __init__
>>> self.handle()
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
>>> line 200, in handle
>>> rv = BaseHTTPRequestHandler.handle(self)
>>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/li
>>> b/python2.7/BaseHTTPServer.py", line 340, in handle
>>> self.handle_one_request()
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
>>> line 235, in handle_one_request
>>> return self.run_wsgi()
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
>>> line 177, in run_wsgi
>>> execute(self.server.app)
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
>>> line 165, in execute
>>> application_iter = app(environ, start_response)
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 2000, in __call__
>>> return self.wsgi_app(environ, start_response)
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 1991, in wsgi_app
>>> response = self.make_response(self.handle_exception(e))
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 1567, in handle_exception
>>> reraise(exc_type, exc_value, tb)
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 1988, in wsgi_app
>>> response = self.full_dispatch_request()
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 1641, in full_dispatch_request
>>> rv = self.handle_user_exception(e)
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 1544, in handle_user_exception
>>> reraise(exc_type, exc_value, tb)
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 1639, in full_dispatch_request
>>> rv = self.dispatch_request()
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 1625, in dispatch_request
>>> return self.view_functions[rule.endpoint](**req.view_args)
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py",
>>> line 792, in decorated_view
>>> return func(*args, **kwargs)
>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/__init__.py",
>>> line 62, in index
>>> return make_response(response=BatchProcess.list())
>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/processes.py",
>>> line 512, in list
>>> desc = loads(p.desc)
>>> File
>>> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
>>> line 1381, in loads
>>> file = StringIO(str)
>>> UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in
>>> position 138: ordinal not in range(128)
>>>
>>> Fixed. Please find the attached updated patch.
>>
>>>
> That's getting further, but still not right. Please see the attached
> screenshot - the beginning of the executed command has been lost.
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
diff --git a/web/pgadmin/misc/bgprocess/processes.py b/web/pgadmin/misc/bgprocess/processes.py
index 02b953d..385b7bd 100644
--- a/web/pgadmin/misc/bgprocess/processes.py
+++ b/web/pgadmin/misc/bgprocess/processes.py
@@ -78,10 +78,20 @@ class BatchProcess(object):
_("Could not find a process with the specified ID.")
)
+ try:
+ tmp_desc = loads(p.desc.encode('latin-1')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+ except UnicodeDecodeError:
+ tmp_desc = loads(p.desc.encode('utf-8')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+ except Exception as e:
+ tmp_desc = loads(p.desc.encode('utf-8', 'ignore')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+
# ID
self.id = _id
# Description
- self.desc = loads(p.desc)
+ self.desc = tmp_desc
# Status Acknowledged time
self.atime = p.acknowledge
# Command
@@ -171,6 +181,16 @@ class BatchProcess(object):
csv_writer.writerow(_args)
args_val = args_csv_io.getvalue().strip(str('\r\n'))
+ tmp_desc = dumps(self.desc)
+ try:
+ tmp_desc =tmp_desc.decode('utf-8') if\
+ IS_PY2 and hasattr(tmp_desc, 'decode') else tmp_desc
+ except UnicodeDecodeError:
+ tmp_desc = tmp_desc.decode('latin-1') if \
+ IS_PY2 and hasattr(tmp_desc, 'decode') else tmp_desc
+ except Exception:
+ tmp_desc = tmp_desc.decode('utf-8', 'ignore') if \
+ IS_PY2 and hasattr(tmp_desc, 'decode') else tmp_desc
j = Process(
pid=int(id),
@@ -178,7 +198,7 @@ class BatchProcess(object):
arguments=args_val.decode('utf-8', 'replace')
if IS_PY2 and hasattr(args_val, 'decode') else args_val,
logdir=log_dir,
- desc=dumps(self.desc),
+ desc=tmp_desc,
user_id=current_user.id
)
db.session.add(j)
@@ -534,7 +554,17 @@ class BatchProcess(object):
etime = parser.parse(p.end_time or get_current_time())
execution_time = (etime - stime).total_seconds()
- desc = loads(p.desc)
+ desc = ""
+ try:
+ desc = loads(p.desc.encode('latin-1')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+ except UnicodeDecodeError:
+ desc = loads(p.desc.encode('utf-8')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+ except Exception:
+ desc = loads(p.desc.encode('utf-8', 'ignore')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+
details = desc
if isinstance(desc, IProcessDesc):
diff --git a/web/pgadmin/misc/bgprocess/static/js/bgprocess.js b/web/pgadmin/misc/bgprocess/static/js/bgprocess.js
index 5e30590..8ad2c65 100644
--- a/web/pgadmin/misc/bgprocess/static/js/bgprocess.js
+++ b/web/pgadmin/misc/bgprocess/static/js/bgprocess.js
@@ -248,7 +248,7 @@ define('misc.bgprocess', [
if (!self.notifier) {
var header = $('<div></div>', {
class: 'h5 pg-bg-notify-header',
- }).append($('<span></span>').text(self.desc)),
+ }).append($('<span></span>').text(_.escape(self.desc))),
content = $('<div class="pg-bg-bgprocess row"></div>').append(
header
).append(
@@ -366,7 +366,7 @@ define('misc.bgprocess', [
self.logs[0].scrollTop = self.logs[0].scrollHeight;
});
// set bgprocess detailed description
- $header.find('.bg-detailed-desc').html(self.detailed_desc);
+ $header.find('.bg-detailed-desc').html(_.unescape(self.detailed_desc));
}
// set bgprocess start time
@@ -562,4 +562,4 @@ define('misc.bgprocess', [
});
return pgBrowser.BackgroundProcessObsorver;
-});
\ No newline at end of file
+});
diff --git a/web/pgadmin/tools/backup/__init__.py b/web/pgadmin/tools/backup/__init__.py
index 715ff6e..9e70c4f 100644
--- a/web/pgadmin/tools/backup/__init__.py
+++ b/web/pgadmin/tools/backup/__init__.py
@@ -97,11 +97,9 @@ class BackupMessage(IProcessDesc):
def cmdArg(x):
if x:
- # x = html.safe_str(x)
x = x.replace('\\', '\\\\')
x = x.replace('"', '\\"')
x = x.replace('""', '\\"')
-
return ' "' + x + '"'
return ''
@@ -111,6 +109,7 @@ class BackupMessage(IProcessDesc):
else:
self.cmd += cmdArg(arg)
+
@property
def message(self):
# Fetch the server details like hostname, port, roles etc
@@ -189,7 +188,7 @@ class BackupMessage(IProcessDesc):
res += '</div><div class="h5">'
res += _("Running command:")
res += '</b><br><span class="pg-bg-cmd enable-selection">'
- res += html.safe_str(self.cmd)
+ res += html.safe_str(cmd + self.cmd)
res += '</span></div>'
return res
