Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4]

[Harshal Dhumal]

yes i'm working on that only :)


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Jun 26, 2017 at 5:22 PM, Dave Page  wrote:

> I'm sorry, it needs rebasing again. If you can do it quickly, I'll
> make sure it's the next patch I work on in that area.
>
> Thanks.
>
> On Mon, Jun 26, 2017 at 5:16 AM, Harshal Dhumal
>  wrote:
> > Hi Dave,
> >
> > Please find updated rebased patch for RM2137
> >
> > On Fri, Jun 23, 2017 at 9:00 PM, Dave Page  wrote:
> >>
> >> Hi Harshal,
> >>
> >> When can we expect an updated version of this patch? I think it's
> >> important to get this into the next release.
> >>
> >> Thanks!
> >>
> >> On Fri, Jun 16, 2017 at 10:55 AM, Dave Page  wrote:
> >> > Hi,
> >> >
> >> > That's better - the failures are far less random now :-). I got the
> >> > following two though, on both PG and EPAS 9.5:
> >> >
> >> > 
> ==
> >> > ERROR: runTest
> >> > (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest)
> >> > Query tool feature test
> >> > 
> --
> >> > Traceback (most recent call last):
> >> >   File
> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/
> query_tool_tests.py",
> >> > line 95, in runTest
> >> > self._query_tool_explain_analyze_buffers()
> >> >   File
> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/
> query_tool_tests.py",
> >> > line 443, in _query_tool_explain_analyze_buffers
> >> > canvas.find_element_by_xpath("//*[contains(string(), 'Shared Read
> >> > Blocks')]")
> >> >   File
> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-
> packages/selenium/webdriver/remote/webelement.py",
> >> > line 260, in find_element_by_xpath
> >> > return self.find_element(by=By.XPATH, value=xpath)
> >> >   File
> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-
> packages/selenium/webdriver/remote/webelement.py",
> >> > line 508, in find_element
> >> > {"using": by, "value": value})['value']
> >> >   File
> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-
> packages/selenium/webdriver/remote/webelement.py",
> >> > line 491, in _execute
> >> > return self._parent.execute(command, params)
> >> >   File
> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-
> packages/selenium/webdriver/remote/webdriver.py",
> >> > line 238, in execute
> >> > self.error_handler.check_response(response)
> >> >   File
> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-
> packages/selenium/webdriver/remote/errorhandler.py",
> >> > line 193, in check_response
> >> > raise exception_class(message, screen, stacktrace)
> >> > NoSuchElementException: Message: no such element: Unable to locate
> >> > element: {"method":"xpath","selector":"//*[contains(string(), 'Shared
> >> > Read Blocks')]"}
> >> >   (Session info: chrome=58.0.3029.110)
> >> >   (Driver info: chromedriver=2.29.461585
> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3
> >> > x86_64)
> >> >
> >> >
> >> > 
> ==
> >> > ERROR: runTest
> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest)
> >> > Validate Insert, Update operations in View data with given test data
> >> > 
> --
> >> > Traceback (most recent call last):
> >> >   File
> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/
> view_data_dml_queries.py",
> >> > line 104, in runTest
> >> > self._add_row()
> >> >   File
> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/
> view_data_dml_queries.py",
> >> > line 255, in _add_row
> >> > self._up

Re: [pgAdmin4][Patch]: Using client-side 'url_for' implementation in the datagrid module

[Harshal Dhumal]

Hi Dave,

Due to this commit all features test cases which requires query tool are
failing. Reason for this is additional dummy iframe added in panel.

@@ -463,7 +501,16 @@ define([
* create new panel and add it to the dashboard panel.
*/
   var dashboardPanel = pgBrowser.docker.findPanels('dashboard');
-  queryToolPanel = pgBrowser.docker.addPanel('frm_datagrid',
wcDocker.DOCK.STACKED, dashboardPanel[0]);
+  var $frameArea = $('');
+
+  var queryToolPanel =
pgBrowser.docker.addPanel('frm_datagrid', wcDocker.DOCK.STACKED,
dashboardPanel[0]);
+  queryToolPanel.layout().addItem($frameArea);
+  // Initialize empty frame
+  var frame = new wcIFrame($frameArea, queryToolPanel);
+  $(queryToolPanel).data('frameInitialized', false);
+  $(queryToolPanel).data('embeddedFrame', frame);
+
+  // Set panel title and icon
   queryToolPanel.title(''+panel_title+'');
   queryToolPanel.icon('fa fa-bolt');

Surinder is looking into this. I'm holding patch for RM2137 (on demand
loading of result set) until this is fixed.

Thanks,



-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Jun 26, 2017 at 5:13 PM, Dave Page  wrote:

> Thanks, patch applied.
>
> On Mon, Jun 26, 2017 at 2:05 AM, Surinder Kumar
>  wrote:
> > Hi
> >
> > Changes:
> >
> > 1. Replace Jinja {{ url_for }} with js 'url_for'
> >
> > 2. While minifying JS using webpack, the code to open QueryToolPanel and
> > DatagridPanel wasn't working. This is fixed.
> >
> > Please find attached patch.
> >
> > Thanks,
> > Surinder Kumar
>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
>

Re: [pgadmin-hackers][patch] History Detail Pane

[Harshal Dhumal]

Hi,

New history pane is really nice and informative than existing one.
I'm just thinking about one minor improvement we can make, to allow history
tab to respond to up and down arrow keys so that user can easily switch
between executed queries without using mouse.


Thanks,

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jun 27, 2017 at 8:26 PM, Dave Page  wrote:

> Thanks - patch applied (just in time for Raffi's & my talk)!
>
> On Tue, Jun 27, 2017 at 10:05 AM, Joao Pedro De Almeida Pereira <
> jdealmeidapere...@pivotal.io> wrote:
>
>> Yep,
>> please see attached
>>
>> On Tue, Jun 27, 2017 at 9:11 AM, Dave Page  wrote:
>>
>>> Can you rebase this patch please?
>>>
>>> Thankfully I think we're basically at the end of our changes in this
>>> area!
>>>
>>> On Mon, Jun 26, 2017 at 10:08 AM, Joao Pedro De Almeida Pereira <
>>> jdealmeidapere...@pivotal.io> wrote:
>>>
>>>> Hi Surinder,
>>>>
>>>> You can find our answers inline and attached the patch with the change
>>>> of scrolls from "scroll" to "auto"
>>>>
>>>> On Mon, Jun 26, 2017 at 4:47 AM, Surinder Kumar <
>>>> surinder.ku...@enterprisedb.com> wrote:
>>>>
>>>>> Hi
>>>>> On Fri, Jun 23, 2017 at 11:39 PM, George Gelashvili <
>>>>> ggelashv...@pivotal.io> wrote:
>>>>>
>>>>>> On Fri, Jun 23, 2017 at 11:24 AM, Surinder Kumar <
>>>>>> surinder.ku...@enterprisedb.com> wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> Review comments:
>>>>>>>
>>>>>>> ​1. ​
>>>>>>> Can we set "message"(in message detail) style property scroll to
>>>>>>> ​'​
>>>>>>> auto
>>>>>>> ​'​
>>>>>>>  instead of
>>>>>>> ​'​
>>>>>>> scroll
>>>>>>> ​'​
>>>>>>>  ?
>>>>>>>
>>>>>>
>>>>>> Could you elaborate why 'auto' is what we want?
>>>>>>
>>>>> ​The scroll bars should appear only when content is beyond the
>>>>> width/height of container.​ Now with 'scroll', the border layout around
>>>>> container appears irrespective of content width/height. If we set 'auto',
>>>>> it won't appear.
>>>>>
>>>>
>>>> We changed that in the attached patch. The scroll bars will now appear
>>>> when the text exceeds the window.
>>>>
>>>>
>>>>>> ​2. CSS property flex is supported for IE >= 9 as per reference
>>>>>>>> <https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Flexible_Box_Layout/Using_CSS_flexible_boxes>
>>>>>>>> ​. I tested this patch on IE and layout is broken.​
>>>>>>>>
>>>>>>>>  Anyways IE-9/10 are outdated and no longer supported by
>>>>>>> Microsoft, the only supported browsers are IE-11+.
>>>>>>>
>>>>>>
>>>>>> Does this patch work in supported IEs?
>>>>>>
>>>>> ​I use Windows 7 which has IE9,10, but if we can fix it for IE9,10 we
>>>>> should fix. Majority of people are still using IE9,10.​
>>>>>
>>>>
>>>> We based almost 100% of this patch on Flexbox. In order to use another
>>>> structure, we would have to rewrite the html blocks (the majority of the
>>>> UI). If the community believes that is important to keep giving support to
>>>> Browsers that are no longer supported then we can use a library like
>>>> modernizr <https://modernizr.com> to try to support all the existing
>>>> browser. Nevertheless we believe that this should be done in a future 
>>>> patch.
>>>>
>>>>
>>>>>> 3. ​Can the CSS styles in ‘history_detail_message.jsx’ moved out in
>>>>>>> a separate stylesheet file
>>>>>>> ​ as inline styles in html are never recommended.​
>>>>>>>
>>>>>>
>>>>>> We've been trying to use inline styles rather than classes for our
>>>>>> react jsx code

[RM2522] Improve grid/column select all operation

[Harshal Dhumal]

Hi,

Please find attached patch for RM2522.

With this patch grid/column select all time is reduce to ~1 second from
8-10 seconds.

The solution was to use simple array concatenation instead of underscore
union while getting index of all selected complete rows.
Underscore union function is only useful when user selects different ranges
from grid and those ranges overlaps. In this case union function removes
duplicate (overlapped) rows.
However result grid in sqleditor do not support overlapped row selection so
we can simply cancat rows from different ranges without worrying about
overlapped row selection.



-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/static/js/selection/range_selection_helper.js b/web/pgadmin/static/js/selection/range_selection_helper.js
index 25cd63c..4442f2c 100644
--- a/web/pgadmin/static/js/selection/range_selection_helper.js
+++ b/web/pgadmin/static/js/selection/range_selection_helper.js
@@ -94,7 +94,7 @@ define(['slickgrid'], function () {
 var indexArray = [];
 ranges.forEach(function (range) {
   if (rangeHasCompleteRows(grid, range))
-indexArray = _.union(indexArray, _.range(range.fromRow, range.toRow + 1));
+indexArray = indexArray.concat(_.range(range.fromRow, range.toRow + 1));
 });
 
 return indexArray;

Re: [RM2522] Improve grid/column select all operation

[Harshal Dhumal]

On Thu, Jun 29, 2017 at 4:33 PM, Harshal Dhumal <
harshal.dhu...@enterprisedb.com> wrote:

> Hi,
>
> Please find attached patch for RM2522.
>
> With this patch grid/column select all time is reduce to ~1 second from
> 8-10 seconds.
>
This benchmarking is performed with 100k rows and 2 columns (int, text).



>
> The solution was to use simple array concatenation instead of underscore
> union while getting index of all selected complete rows.
> Underscore union function is only useful when user selects different
> ranges from grid and those ranges overlaps. In this case union function
> removes duplicate (overlapped) rows.
> However result grid in sqleditor do not support overlapped row selection
> so we can simply cancat rows from different ranges without worrying about
> overlapped row selection.
>
>
>
> --
> *Harshal Dhumal*
> *Sr. Software Engineer*
>
> EnterpriseDB India: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

Re: [RM2522] Improve grid/column select all operation

[Harshal Dhumal]

Hi,

This patch is not related to improve copy/past operation. It's only related
to grid selection.
The improvement is only between when user clicks on grid select all column
to until all row are selected (turned into blue). With large columns though
improvement is not large but it not putting any overhead. Other hand with
less columns but large row count the improvement it quite noticeable. Code
change is only single line to use array concatenation instead of underscore
union <http://underscorejs.org/#union>. I had look at union function
implementation. It first flattens result set recursively and then removes
duplicate which is not required in our case. All we want is indexes of
complete rows in selected rows (ranges).





-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, Jun 30, 2017 at 1:30 PM, Dave Page  wrote:

>
>
> On Fri, Jun 30, 2017 at 8:57 AM, Murtuza Zabuawala  enterprisedb.com> wrote:
>
>> I guess so, number of columns do have overhead on selection & copy/paste
>> operation because of arbitrary copy paste feature, now we have to scan
>> start cell (first selected cell) & end cell (last selected cell) for each
>> row, In old code we were selecting complete row which was fast but now user
>> can select different columns as well, so we have scan for each selected
>> columns.
>>
>
> Yeah, but 14 - 15 seconds for 100K rows? It takes a fraction of that time
> to find them in the database, return the results to the pgAdmin server, and
> render them.
>
> We should be able to generate a CSV representation of all the selected
> fields in far less time.
>
>
>>
>> --
>> Regards,
>> Murtuza Zabuawala
>> EnterpriseDB: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>> On Fri, Jun 30, 2017 at 12:07 PM, Dave Page  wrote:
>>
>>> Hi
>>>
>>> On Thursday, June 29, 2017, Harshal Dhumal <
>>> harshal.dhu...@enterprisedb.com> wrote:
>>>
>>>>
>>>>
>>>> On Thu, Jun 29, 2017 at 4:33 PM, Harshal Dhumal <
>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Please find attached patch for RM2522.
>>>>>
>>>>> With this patch grid/column select all time is reduce to ~1 second
>>>>> from 8-10 seconds.
>>>>>
>>>> This benchmarking is performed with 100k rows and 2 columns (int, text).
>>>>
>>>
>>> I see very little improvement (~15s to 14s). My test case has  108786
>>> rows in it, and is generated from the query:
>>>
>>> select * from pg_class c, pg_attribute a where c.oid = a.attrelid
>>> union all
>>> select * from pg_class c, pg_attribute a where c.oid = a.attrelid
>>> union all
>>> select * from pg_class c, pg_attribute a where c.oid = a.attrelid
>>> union all
>>> select * from pg_class c, pg_attribute a where c.oid = a.attrelid
>>>
>>> Maybe the number of columns has something to do with it?
>>>
>>> AND... I'm still left with nothing being copied to the clipboard (the
>>> main issue by the way, which no patch for this so far seems to have
>>> tackled).
>>>
>>> Thanks.
>>>
>>>
>>>>
>>>>
>>>>>
>>>>> The solution was to use simple array concatenation instead of
>>>>> underscore union while getting index of all selected complete rows.
>>>>> Underscore union function is only useful when user selects different
>>>>> ranges from grid and those ranges overlaps. In this case union function
>>>>> removes duplicate (overlapped) rows.
>>>>> However result grid in sqleditor do not support overlapped row
>>>>> selection so we can simply cancat rows from different ranges without
>>>>> worrying about overlapped row selection.
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Harshal Dhumal*
>>>>> *Sr. Software Engineer*
>>>>>
>>>>> EnterpriseDB India: http://www.enterprisedb.com
>>>>> The Enterprise PostgreSQL Company
>>>>>
>>>>
>>>>
>>>
>>> --
>>> Dave Page
>>> Blog: http://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EnterpriseDB UK: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>>>
>>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

Patch for sqleditor js syntax issue

[Harshal Dhumal]

Hi,

Please find attached minor patch which fixes js syntax issue in sqleditor.js


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
index 173301b..124c894 100644
--- a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
+++ b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
@@ -891,7 +891,7 @@ define([
 }
 dataView.setItems(collection, self.client_primary_key);
   },
-  fetch_next_all(cb) {
+  fetch_next_all: function(cb) {
 this.fetch_next(true, cb);
   },
   fetch_next: function(fetch_all, cb) {

[RM1972][pgAdmin4] Improve query tool close workflow.

[Harshal Dhumal]

Hi,

Please find attached patch for to allow user to save changed query or data
when closing query tool/datagrid.




-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py
index 75b4222..ac50ea1 100644
--- a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py
+++ b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py
@@ -61,7 +61,7 @@ class CheckForXssFeatureTest(BaseFeatureTest):
 
 # Query tool
 self._check_xss_in_query_tool()
-self._close_query_tool()
+self.page.close_query_tool()
 
 def after(self):
 time.sleep(1)
@@ -123,7 +123,6 @@ class CheckForXssFeatureTest(BaseFeatureTest):
 "Properties tab (Backform Control)"
 )
 
-
 def _check_xss_in_sql_tab(self):
 self.page.click_tab("SQL")
 # Fetch the inner html & check for escaped characters
@@ -173,19 +172,6 @@ class CheckForXssFeatureTest(BaseFeatureTest):
 "Query tool (SlickGrid)"
 )
 
-def _close_query_tool(self):
-self.page.driver.switch_to_default_content()
-self.page.click_element(
-self.page.find_by_xpath("//*[@id='dockerContainer']/div/div[3]/div/div[2]/div[1]")
-)
-time.sleep(0.5)
-self.page.driver.switch_to.frame(self.page.driver.find_element_by_tag_name('iframe'))
-time.sleep(1)
-self.page.click_element(self.page.find_by_xpath("//button[contains(.,'Yes')]"))
-time.sleep(0.5)
-self.page.driver.switch_to_default_content()
-
-
 def _check_escaped_characters(self, source_code, string_to_find, source):
 # For XSS we need to search against element's html code
 if source_code.find(string_to_find) == -1:
diff --git a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
index 2ea37fc..0584df5 100644
--- a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
+++ b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
@@ -253,11 +253,11 @@ define([
   var data_store = self.handler.data_store;
   if(data_store && (_.size(data_store.added) ||
   _.size(data_store.updated))) {
-msg = gettext("The data has been modified, but not saved. Are you sure you wish to discard the changes?");
+msg = gettext("The data has changed. Do you want to save changes?");
 notify = true;
   }
 } else if(self.handler.is_query_tool && self.handler.is_query_changed) {
-  msg = gettext("The query has been modified, but not saved. Are you sure you wish to discard the changes?");
+  msg = gettext("The text has changed. Do you want to save changes?");
   notify = true;
 }
 if(notify) {return self.user_confirmation(p, msg);}
@@ -407,23 +407,60 @@ define([
   user_confirmation: function(panel, msg) {
 // If there is anything to save then prompt user
 var that = this;
-alertify.confirm(gettext("Unsaved changes"), msg,
-  function() {
-// Do nothing as user do not want to save, just continue
-window.onbeforeunload = null;
-panel.off(wcDocker.EVENT.CLOSING);
-// remove col_size object on panel close
-if (!_.isUndefined(that.handler.col_size)) {
-  delete that.handler.col_size;
+
+alertify.confirmSave || alertify.dialog('confirmSave', function() {
+  return {
+main: function(title, message) {
+var content = ''
++ gettext('The text has changed. Do you want to save changes?')
++ '';
+  this.setHeader(title);
+  this.setContent(message);
+},
+setup: function () {
+  return {
+buttons: [
+{
+  text: gettext('Save'),
+  className: 'btn btn-primary',
+},{
+  text: gettext('Don\'t save'),
+  className: 'btn btn-danger',
+},{
+  text: gettext('Cancel'),
+  key: 27, // ESC
+  invokeOnClose: true,
+  className: 'btn btn-warning',
+  

Fix clear history option [pgAdmin4][RM2535]

[Harshal Dhumal]

Hi,

Please find attached patch to fix clear history functionality in query tool.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/static/js/history/history_collection.js b/web/pgadmin/static/js/history/history_collection.js
index f7b6acd..22a86f7 100644
--- a/web/pgadmin/static/js/history/history_collection.js
+++ b/web/pgadmin/static/js/history/history_collection.js
@@ -25,10 +25,14 @@ export default class HistoryCollection {
 
   reset() {
 this.historyList = [];
-this.onChangeHandler(this.historyList);
+this.onResetHandler(this.historyList);
   }
 
   onChange(onChangeHandler) {
 this.onChangeHandler = onChangeHandler;
   }
+
+  onReset(onResetHandler) {
+this.onResetHandler = onResetHandler;
+  }
 }
\ No newline at end of file
diff --git a/web/pgadmin/static/jsx/history/query_history.jsx b/web/pgadmin/static/jsx/history/query_history.jsx
index 0d9222d..270dbf1 100644
--- a/web/pgadmin/static/jsx/history/query_history.jsx
+++ b/web/pgadmin/static/jsx/history/query_history.jsx
@@ -36,6 +36,10 @@ export default class QueryHistory extends React.Component {
 this.props.historyCollection.onChange((historyList) => {
   this.resetCurrentHistoryDetail(historyList);
 });
+
+this.props.historyCollection.onReset((historyList) => {
+  this.clearCurrentHistoryDetail(historyList);
+});
   }
 
   componentDidMount() {
@@ -58,6 +62,14 @@ export default class QueryHistory extends React.Component {
 this.setCurrentHistoryDetail(0, historyList);
   }
 
+  clearCurrentHistoryDetail(historyList) {
+this.setState({
+  history: historyList,
+  currentHistoryDetail: undefined,
+  selectedEntry: 0,
+});
+  }
+
   retrieveOrderedHistory() {
 return _.chain(this.state.history)
   .sortBy(historyEntry => historyEntry.start_time)
diff --git a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
index 0584df5..8a797ab 100644
--- a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
+++ b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
@@ -1325,8 +1325,7 @@ define([
 alertify.confirm(gettext("Clear history"),
   gettext("Are you sure you wish to clear the history?"),
   function() {
-// Remove any existing grid first
-if (self.history_grid) {
+if (self.history_collection) {
   self.history_collection.reset();
 }
   },
diff --git a/web/regression/javascript/history/history_collection_spec.js b/web/regression/javascript/history/history_collection_spec.js
index 9c10b90..b952c92 100644
--- a/web/regression/javascript/history/history_collection_spec.js
+++ b/web/regression/javascript/history/history_collection_spec.js
@@ -10,13 +10,15 @@
 import HistoryCollection from '../../../pgadmin/static/js/history/history_collection';
 
 describe('historyCollection', function () {
-  let historyCollection, historyModel, onChangeSpy;
+  let historyCollection, historyModel, onChangeSpy, onResetSpy;
   beforeEach(() => {
 historyModel = [{some: 'thing', someOther: ['array element']}];
 historyCollection = new HistoryCollection(historyModel);
 onChangeSpy = jasmine.createSpy('onChangeHandler');
+onResetSpy = jasmine.createSpy('onResetHandler');
 
 historyCollection.onChange(onChangeSpy);
+historyCollection.onReset(onResetSpy);
   });
 
   describe('length', function () {
@@ -61,8 +63,8 @@ describe('historyCollection', function () {
   expect(historyCollection.length()).toBe(0);
 });
 
-it('calls the onChange function', function () {
-  expect(onChangeSpy).toHaveBeenCalledWith([]);
+it('calls the onReset function', function () {
+  expect(onResetSpy).toHaveBeenCalledWith([]);
 });
   });
 

[RM2534][pgAdmin4] Fix issues related to table OF TYPE property.

[Harshal Dhumal]

Hi,

Please find attached patch to fix issue related to table OF TYPE property.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/static/js/table.js b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/static/js/table.js
index 7f6aa0c..7560be4 100644
--- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/static/js/table.js
+++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/static/js/table.js
@@ -725,19 +725,33 @@ define('pgadmin.node.table', [
   arg = undefined,
   column_collection = self.model.get('columns');
 
-if (!_.isUndefined(tbl_name) &&
+if (!_.isUndefined(tbl_name) && !_.isNull(tbl_name) &&
 tbl_name !== '' && column_collection.length !== 0) {
-  var msg = gettext('Changing of type table will clear columns collection');
-  alertify.confirm(msg, function (e) {
-if (e) {
+  var title = gettext('Remove column definitions?'),
+  msg = gettext('Changing \'Of type\' will remove column definitions.');
+
+  alertify.confirm(title, msg, function (e) {
   // User clicks Ok, lets clear columns collection
-  column_collection.reset();
-} else {
-  return this;
+  column_collection.remove(
+column_collection.filter(function(model) {
+  return true;
+  }
+)
+  );
+},
+function() {
+  setTimeout(function() {
+self.model.set('typname', null);
+  }, 10)
 }
-  });
+  );
 } else if (!_.isUndefined(tbl_name) && tbl_name === '') {
-  column_collection.reset();
+  column_collection.remove(
+column_collection.filter(function(model) {
+  return true;
+  }
+)
+  );
 }
 
 // Run Ajax now to fetch columns
diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/templates/table/sql/default/create.sql b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/templates/table/sql/default/create.sql
index 5ec8ffc..ffb6b84 100644
--- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/templates/table/sql/default/create.sql
+++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/templates/table/sql/default/create.sql
@@ -18,7 +18,7 @@
 {% endif %}
 CREATE {% if data.relpersistence %}UNLOGGED {% endif %}TABLE {{conn|qtIdent(data.schema, data.name)}}{{empty_bracket}}
 {% if data.typname %}
-OF {{ conn|qtTypeIdent(data.typname) }}
+OF {{ data.typname }}
 {% endif %}
 {% if data.like_relation or data.coll_inherits or data.columns|length > 0 or data.primary_key|length > 0 or data.unique_constraint|length > 0 or data.foreign_key|length > 0 or data.check_constraint|length > 0 or data.exclude_constraint|length > 0 %}
 (

Re: pgAdmin 4 commit: Release notes for 1.6.

[Harshal Dhumal]

Hi Dave,

Below corrections required in release notes.

Feature #2137:
1) Redmine link is pointing to issue 2437 it should be 2137
2) Feature title should On-demand loading for the query tool result and
not Fixed Row selection issue and row past issue.


Fearture# 2191:
1) Redmine link is pointing to issue 2491 it should be 2191


I have listed incorrect feature entries below for reference.
+| `Feature #2137 <https://redmine.postgresql.org/issues/2437>`_ -
Fixed Row selection issue and row past issue
+| `Feature #2191 <https://redmine.postgresql.org/issues/2491
>`_ - Add support for the hostaddr connection parameter. This helps us play 
>nicely with Kerberos/SSPI and friends









-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Jul 10, 2017 at 7:22 PM, Dave Page  wrote:

> Release notes for 1.6.
>
> Branch
> --
> master
>
> Details
> ---
> https://git.postgresql.org/gitweb?p=pgadmin4.git;a=commitdiff;h=
> 4bfb31b03b3c6f1337f5ef317c69569fdf940d6c
> Author: Karen Blatchley 
>
> Modified Files
> --
> docs/en_US/release_notes.rst |   1 +
> docs/en_US/release_notes_1_6.rst | 102 ++
> +
> 2 files changed, 103 insertions(+)
>
>

[RM2551][pgAdmin4] show tablespace on partitions.

[Harshal Dhumal]

Hi,

Please find attached patch to show tablespace on partitions.
Also reformatted sql slightly in sql tab.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/templates/partition/sql/10_plus/create.sql b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/templates/partition/sql/10_plus/create.sql
index b5e9404..19eea8c 100644
--- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/templates/partition/sql/10_plus/create.sql
+++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/templates/partition/sql/10_plus/create.sql
@@ -19,9 +19,15 @@ CREATE {% if data.relpersistence %}UNLOGGED {% endif %}TABLE {{conn|qtIdent(data
 {% endif %}
 {{ data.partition_value }}{% if data.is_partitioned is defined and data.is_partitioned %}
 
-PARTITION BY {{ data.partition_scheme }}{% endif %};
+PARTITION BY {{ data.partition_scheme }}{% endif %}
+{### SQL for Tablespace ###}
+{% if data.spcname %}
 
+TABLESPACE {{ conn|qtIdent(data.spcname) }};
+{% else %}
+;
 
+{% endif %}
 {### Alter SQL for Owner ###}
 {% if data.relowner %}
 
diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/utils.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/utils.py
index 555133f..1b16454 100644
--- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/utils.py
+++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/utils.py
@@ -1030,26 +1030,27 @@ class BaseTableView(PGChildNodeView):
 if not status:
 return internal_server_error(errormsg=rset)
 
-sql_header = u"\n-- Partitions SQL"
-partition_sql = ''
-for row in rset['rows']:
-part_data = dict()
-part_data['partitioned_table_name'] = data['name']
-part_data['parent_schema'] = data['schema']
-part_data['schema'] = row['schema_name']
-part_data['relispartition'] = True
-part_data['name'] = row['name']
-part_data['partition_value'] = row['partition_value']
-part_data['is_partitioned'] = row ['is_partitioned']
-part_data['partition_scheme'] = row['partition_scheme']
-
-partition_sql += render_template("/".join(
-[self.partition_template_path, 'create.sql']),
-data=part_data, conn=self.conn)
-
-# Add into main sql
-partition_sql = re.sub('\n{2,}', '\n\n', partition_sql)
-main_sql.append(sql_header + '\n\n' + partition_sql.strip('\n'))
+if len(rset['rows']):
+sql_header = u"\n-- Partitions SQL"
+partition_sql = ''
+for row in rset['rows']:
+part_data = dict()
+part_data['partitioned_table_name'] = data['name']
+part_data['parent_schema'] = data['schema']
+part_data['schema'] = row['schema_name']
+part_data['relispartition'] = True
+part_data['name'] = row['name']
+part_data['partition_value'] = row['partition_value']
+part_data['is_partitioned'] = row ['is_partitioned']
+part_data['partition_scheme'] = row['partition_scheme']
+
+partition_sql += render_template("/".join(
+[self.partition_template_path, 'create.sql']),
+data=part_data, conn=self.conn)
+
+# Add into main sql
+partition_sql = re.sub('\n{2,}', '\n\n', partition_sql)
+main_sql.append(sql_header + '\n\n' + partition_sql.strip('\n'))
 
 sql = '\n'.join(main_sql)
 

[RM2544][pgAdmin4] Fix for malformed query generated when using custom type

[Harshal Dhumal]

Hi,

Please find attached patch to fix malformed sql for data type.


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/types/templates/type/macros/get_full_type_sql_format.macros b/web/pgadmin/browser/server_groups/servers/databases/schemas/types/templates/type/macros/get_full_type_sql_format.macros
index 4b9c0e9..c3f7707 100644
--- a/web/pgadmin/browser/server_groups/servers/databases/schemas/types/templates/type/macros/get_full_type_sql_format.macros
+++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/types/templates/type/macros/get_full_type_sql_format.macros
@@ -13,7 +13,7 @@ time({{ type_length }}) with time zone{% endif %}{% if is_type_array %}
 {## Need to check against separate time types - END ##}
 {#}
 {% else %}
-{{ conn|qtTypeIdent(type_name) }}{% if type_length %}
+{{ type_name }}{% if type_length %}
 ({{ type_length }}{% if type_precision%}, {{ type_precision }}{% endif %}){% endif %}{% if is_type_array %}
 []{% endif %}
 {% endif %}

[RM2153][pgAdmin4] handle large file uploads

[Harshal Dhumal]

Hi,

Please find attached patch to handle upload of large files and also to show
upload error if
error occurs instead of showing undefined.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/misc/file_manager/__init__.py b/web/pgadmin/misc/file_manager/__init__.py
index 0e39fa9..2239cae 100644
--- a/web/pgadmin/misc/file_manager/__init__.py
+++ b/web/pgadmin/misc/file_manager/__init__.py
@@ -886,10 +886,14 @@ class Filemanager(object):
 newName = u"{0}{1}".format(orig_path, file_name)
 
 with open(newName, 'wb') as f:
-f.write(file_obj.read())
+while True:
+data = file_obj.read(4194304)  # 4MB chunk (4 * 1024 * 1024 Bytes)
+if not data:
+break
+f.write(data)
 except Exception as e:
 code = 0
-err_msg = u"Error: {0}".format(e.strerror)
+err_msg = u"Error: {0}".format(e.strerror if hasattr(e, 'strerror') else u'Unknown')
 
 try:
 Filemanager.check_access_permission(dir, path)
diff --git a/web/pgadmin/misc/file_manager/templates/file_manager/js/utility.js b/web/pgadmin/misc/file_manager/templates/file_manager/js/utility.js
index d0eaf7a..5de8c1c 100755
--- a/web/pgadmin/misc/file_manager/templates/file_manager/js/utility.js
+++ b/web/pgadmin/misc/file_manager/templates/file_manager/js/utility.js
@@ -1624,7 +1624,7 @@ if (has_capability(data, 'upload')) {
   complete: function(file) {
 if (file.status == "error") {
   var alertifyWrapper = new AlertifyWrapper();
-  alertifyWrapper.error(lg.ERROR_UPLOADING_FILE);
+  alertifyWrapper.error(lg.upload_error);
 }
 $('.upload_file .dz_cross_btn').removeAttr('disabled');
 getFolderInfo(path);
diff --git a/web/pgadmin/tools/sqleditor/__init__.py b/web/pgadmin/tools/sqleditor/__init__.py
index d34836c..d51e241 100644
--- a/web/pgadmin/tools/sqleditor/__init__.py
+++ b/web/pgadmin/tools/sqleditor/__init__.py
@@ -1490,14 +1490,15 @@ def load_file():
 errormsg=gettext("File type not supported")
 )
 
-with codecs.open(file_path, 'r', encoding=enc) as fileObj:
-data = fileObj.read()
-
-return make_json_response(
-data={
-'status': True, 'result': data,
-}
-)
+def gen():
+with codecs.open(file_path, 'r', encoding=enc) as fileObj:
+while True:
+data = fileObj.read(4194304)  # 4MB chunk (4 * 1024 * 1024 Bytes)
+if not data:
+break
+yield data
+
+return Response(gen(), mimetype='text/plain')
 
 
 @blueprint.route('/save_file/', methods=["PUT", "POST"], endpoint='save_file')
diff --git a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
index 8a797ab..72904a9 100644
--- a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
+++ b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
@@ -2542,11 +2542,9 @@ define([
 contentType: "application/json",
 data: JSON.stringify(data),
 success: function(res) {
-  if (res.data.status) {
-self.gridView.query_tool_obj.setValue(res.data.result);
-self.gridView.current_file = e;
-self.setTitle(self.gridView.current_file.split('\\').pop().split('/').pop());
-  }
+  self.gridView.query_tool_obj.setValue(res);
+  self.gridView.current_file = e;
+  self.setTitle(self.gridView.current_file.split('\\').pop().split('/').pop());
   self.trigger('pgadmin-sqleditor:loading-icon:hide');
   // hide cursor
   $busy_icon_div.removeClass('show_progress');

[RM2074][[RM2080]][pgAdmin4] handle large bytea and bytea[] data in datagrid

[Harshal Dhumal]

Hi,

Please find attached patch to handle bytea and bytea[] data in datagrid.

Now instead of showing actual data we can show placeholders like  and . Also placeholders will only appear if data
actually exists otherwise null will be shown.


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/feature_tests/pg_datatype_validation_test.py b/web/pgadmin/feature_tests/pg_datatype_validation_test.py
index 26c787b..7bedde8 100644
--- a/web/pgadmin/feature_tests/pg_datatype_validation_test.py
+++ b/web/pgadmin/feature_tests/pg_datatype_validation_test.py
@@ -101,7 +101,7 @@ class PGDataypeFeatureTest(BaseFeatureTest):
 '922337203685.922337203685', '-92233720368547758.08',
 '{1,2,3}', '{NaN,NaN,NaN}',
 'Infinity', '{Infinity}',
-r'\336\255\276\357', r'{"\\336\\255\\276\\357","\\336\\255\\276\\357"}'
+r'<binary data>', r'<binary data[]>'
 ]
 
 self.page.open_query_tool()
diff --git a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
index 8a797ab..bf29a39 100644
--- a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
+++ b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
@@ -578,6 +578,7 @@ define([
   name: c.label,
   display_name: c.display_name,
   column_type: c.column_type,
+  column_type_internal: c.column_type_internal,
   not_null: c.not_null,
   has_default_val: c.has_default_val
 };
@@ -744,6 +745,11 @@ define([
 // Listener function which will be called before user updates existing cell
 // This will be used to collect primary key for that row
 grid.onBeforeEditCell.subscribe(function (e, args) {
+if (args.column.column_type_internal == 'bytea' ||
+args.column.column_type_internal == 'bytea[]') {
+  return false;
+}
+
 var before_data = args.item;
 
 // If newly added row is saved but grid is not refreshed,
@@ -2000,9 +2006,9 @@ define([
pg_types[pg_types.length - 1][0] : 'unknown';
 
 if (!is_primary_key)
-  col_type += ' ' + type;
+  col_type += type;
 else
-  col_type += ' [PK] ' + type;
+  col_type += '[PK] ' + type;
 
 if (c.precision && c.precision >= 0 && c.precision != 65535) {
   col_type += ' (' + c.precision;
@@ -2051,6 +2057,7 @@ define([
   'name': c.name,
   'display_name': c.display_name,
   'column_type': col_type,
+  'column_type_internal': type,
   'pos': c.pos,
   'label': column_label,
   'cell': col_cell,
diff --git a/web/pgadmin/utils/driver/psycopg2/__init__.py b/web/pgadmin/utils/driver/psycopg2/__init__.py
index 8a20521..7510f80 100644
--- a/web/pgadmin/utils/driver/psycopg2/__init__.py
+++ b/web/pgadmin/utils/driver/psycopg2/__init__.py
@@ -60,8 +60,8 @@ psycopg2.extensions.register_type(
 psycopg2.extensions.register_type(
 psycopg2.extensions.new_type(
 (
-# To cast bytea, bytea[] and interval type
-17, 1001, 1186,
+# To cast interval type
+1186,
 
 # to cast int4range, int8range, numrange tsrange, tstzrange,
 # daterange
@@ -76,6 +76,28 @@ psycopg2.extensions.register_type(
 'TYPECAST_TO_STRING', psycopg2.STRING)
 )
 
+psycopg2.extensions.register_type(
+psycopg2.extensions.new_type(
+(
+# To cast bytea type
+17,
+ ),
+'BYTEA_PLACEHOLDER',
+# Only show placeholder if data actually exists.
+lambda value, cursor: '' if value is not None else None)
+)
+
+psycopg2.extensions.register_type(
+psycopg2.extensions.new_type(
+(
+# To cast bytea[] type
+1001,
+ ),
+'BYTEA_ARRAY_PLACEHOLDER',
+# Only show placeholder if data actually exists.
+lambda value, cursor: '' if value is not None else None)
+)
+
 
 def register_string_typecasters(connection):
 if connection.encoding != 'UTF8':

Re: [RM2074][[RM2080]][pgAdmin4] handle large bytea and bytea[] data in datagrid

[Harshal Dhumal]

Hi Dave,


On Mon, Jul 17, 2017 at 9:33 PM, Dave Page  wrote:

> Hi
>
> On Mon, Jul 17, 2017 at 1:09 PM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi,
>>
>> Please find updated patch. Now placeholder string for bytea and bytea[]
>> data will only appear in datagrid (view all/1000/500 rows). If user
>> executes query using Query tool then placeholder won't appear (similar to
>>  pgAdminIII behaviour)
>>
>
> I'm getting the following error when testing this:
>

I ran all feature test cases 5-6 times and each time they ran successfully.
May be this is occasional failure which we get after running test cases for
many times.

Please let me know if you are getting this failure consistently at your end.



>
> ==
> ERROR: runTest (pgadmin.feature_tests.view_data_dml_queries.
> CheckForViewDataTest)
> Validate Insert, Update operations in View/Edit data with given test data
> --
> Traceback (most recent call last):
>   File 
> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
> line 104, in runTest
> self._add_row()
>   File 
> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
> line 264, in _add_row
> self._update_cell(cell_xpath, config_data[str(idx)])
>   File 
> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
> line 164, in _update_cell
> cell_el = self.page.find_by_xpath(xpath)
>   File 
> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 148, in find_by_xpath
> return self.wait_for_element(lambda driver:
> driver.find_element_by_xpath(xpath))
>   File 
> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 231, in wait_for_element
> return self._wait_for("element to exist", element_if_it_exists)
>   File 
> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 281, in _wait_for
> "Timed out waiting for " + waiting_for_message)
>   File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-
> packages/selenium/webdriver/support/wait.py", line 71, in until
> value = method(self._driver)
>   File 
> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 226, in element_if_it_exists
> if element.is_displayed() and element.is_enabled():
>   File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-
> packages/selenium/webdriver/remote/webelement.py", line 157, in is_enabled
> return self._execute(Command.IS_ELEMENT_ENABLED)['value']
>   File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-
> packages/selenium/webdriver/remote/webelement.py", line 491, in _execute
> return self._parent.execute(command, params)
>   File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-
> packages/selenium/webdriver/remote/webdriver.py", line 238, in execute
> self.error_handler.check_response(response)
>   File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-
> packages/selenium/webdriver/remote/errorhandler.py", line 193, in
> check_response
> raise exception_class(message, screen, stacktrace)
> StaleElementReferenceException: Message: stale element reference: element
> is not attached to the page document
>   (Session info: chrome=59.0.3071.115)
>   (Driver info: chromedriver=2.29.461585 
> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac
> OS X 10.12.5 x86_64)
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

Re: [RM2074][[RM2080]][pgAdmin4] handle large bytea and bytea[] data in datagrid

[Harshal Dhumal]

Hi Dave,

On Tue, Jul 18, 2017 at 1:24 PM, Dave Page  wrote:

>
>
> On Tue, Jul 18, 2017 at 8:26 AM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi Dave,
>>
>>
>> On Mon, Jul 17, 2017 at 9:33 PM, Dave Page  wrote:
>>
>>> Hi
>>>
>>> On Mon, Jul 17, 2017 at 1:09 PM, Harshal Dhumal <
>>> harshal.dhu...@enterprisedb.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Please find updated patch. Now placeholder string for bytea and bytea[]
>>>> data will only appear in datagrid (view all/1000/500 rows). If user
>>>> executes query using Query tool then placeholder won't appear (similar to
>>>>  pgAdminIII behaviour)
>>>>
>>>
>>> I'm getting the following error when testing this:
>>>
>>
>> I ran all feature test cases 5-6 times and each time they ran
>> successfully.
>> May be this is occasional failure which we get after running test cases
>> for many times.
>>
>> Please let me know if you are getting this failure consistently at your
>> end.
>>
>
> I ran it twice and it failed twice. I can't keep running tests over and
> over again as they take quite a while to run and I can't use my machine for
> anything else at the same time as occasionally the test browser will grab
> focus.
>

Please find updated patch.
I have slightly modified feature test case for 'view data dml queries'
(though I wasn't getting failure as mentioned).


>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
diff --git a/web/pgadmin/feature_tests/view_data_dml_queries.py b/web/pgadmin/feature_tests/view_data_dml_queries.py
index 6454a3e..3a14e47 100644
--- a/web/pgadmin/feature_tests/view_data_dml_queries.py
+++ b/web/pgadmin/feature_tests/view_data_dml_queries.py
@@ -100,6 +100,7 @@ CREATE TABLE public.defaults
 # Open Object -> View/Edit data
 self._view_data_grid()
 
+self.page.wait_for_query_tool_loading_indicator_to_disappear()
 # Run test to insert a new row in table with default values
 self._add_row()
 self._verify_row_data(True)
@@ -158,6 +159,7 @@ CREATE TABLE public.defaults
 Returns: None
 
 """
+
 self.wait.until(EC.visibility_of_element_located(
 (By.XPATH, xpath)), CheckForViewDataTest.TIMEOUT_STRING
 )
@@ -195,6 +197,8 @@ CREATE TABLE public.defaults
 self.page.toggle_open_tree_item(self.server['name'])
 self.page.toggle_open_tree_item('Databases')
 self.page.toggle_open_tree_item('acceptance_test_db')
+# wait until all database dependant modules/js are loaded.
+time.sleep(5)
 self.page.toggle_open_tree_item('Schemas')
 self.page.toggle_open_tree_item('public')
 self.page.toggle_open_tree_item('Tables')
@@ -261,6 +265,7 @@ CREATE TABLE public.defaults
 cell_xpath = CheckForViewDataTest._get_cell_xpath(
 'r'+str(idx), 1
 )
+time.sleep(0.4)
 self._update_cell(cell_xpath, config_data[str(idx)])
 
 self.page.find_by_id("btn-save").click()  # Save data
diff --git a/web/pgadmin/tools/datagrid/__init__.py b/web/pgadmin/tools/datagrid/__init__.py
index 95c83dd..08b01ab 100644
--- a/web/pgadmin/tools/datagrid/__init__.py
+++ b/web/pgadmin/tools/datagrid/__init__.py
@@ -117,7 +117,8 @@ def initialize_datagrid(cmd_type, obj_type, sid, did, obj_id):
 conn_id = str(random.randint(1, 999))
 try:
 manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(sid)
-conn = manager.connection(did=did, conn_id=conn_id)
+conn = manager.connection(did=did, conn_id=conn_id,
+  use_binary_placeholder=True)
 except Exception as e:
 return internal_server_error(errormsg=str(e))
 
diff --git a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
index 72904a9..ce65f6f 100644
--- a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
+++ b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js
@@ -578,6 +578,7 @@ define([
   name: c.label,
   display_name: c.display_name,
   column_type: c.column_type,
+  column_type_internal: c.column_type_internal,
   not_null: c.not_null,
   has_default_val: c.has_default_val
 };
@@ -744,6 +745,11 @@ define([
 // Listener function which will be called before user updates existi

Re: Build failed in Jenkins: pgadmin4-master-python27-feature #8

[Harshal Dhumal]

On Tue, Jul 18, 2017 at 9:33 PM, pgAdmin 4 Jenkins 
wrote:

> See  feature/8/display/redirect?page=changes>
>
> Changes:
>
> [Dave Page] Use client-side url_for in the sql editor and data grid
> modules.
>
> [Dave Page] Ensure revoked public privileges are displayed in the RE-SQL
> for
>
> [Dave Page] Ensure saved passwords are effective immediately, not just
> following a
>
> [Dave Page] Un-break query tool feature tests.
>
> [Dave Page] Add support for the hostaddr connection parameter. This helps
> us play
>
> [Dave Page] Use on-demand loading for results in the query tool. Fixes
> #2137
>
> [Dave Page] Allow the user to close the dashboard panel. Fixes #2506
>
> [Dave Page] Add preferences to enable brace matching and brace closing in
> the SQL
>
> [Dave Page] Revert "Allow the user to close the dashboard panel. Fixes
> #2506"
>
> [Dave Page] Add stylesheet to override brace matching styles, missed from
> the
>
> [Dave Page] Overhaul the query history tab to allow browsing of the
> history and full
>
> [Dave Page] Allow non-superusers to debug their own functions and prevent
> them from
>
> [Dave Page] Update preferences docs.
>
> [Dave Page] Fix image size.
>
> [Dave Page] Fix encoding issue when saving servers. Fixes #2518
>
> [Dave Page] A number of minor cleanups to the recent changes to the query
> tool.
>
> [akshay.joshi] 1) Edit cellEditing function, in some cases grid object is
> undefined. 2)
>
> [Dave Page] Improve speed of Select All in the results grid. Fixes #2522
>
> [Dave Page] Allow the dashboard panel to be closed. Fixes #2506
>
> [Dave Page] Add support for SCSS building in webpack.
>
> [Dave Page] Update alertify alerts to use the styling defined in the
> styleguide.
>
> [Dave Page] Update alertify alerts to use the styling defined in the
> styleguide
>
> [Dave Page] Fix a couple of gripes from the JS linter on Windows.
>
> [Dave Page] Fix typo that caused a layout issue with the Edit Grid.
>
> [Dave Page] Fix an issue where duplicate entry is made of successful
> executed query,
>
> [Dave Page] Fix a syntax error in the SQL Editor.
>
> [Dave Page] Fix clipboard handling with large datasets. Fixes #2489
>
> [Dave Page] Improve the debugger's default page layout, and prevent
> attempts to
>
> [Dave Page] Fix deletion of table rows with the column definition having
> NOT NULL
>
> [Dave Page] Allow breakpoints to be set on triggers on views. Fixes #2528
>
> [Dave Page] Allow debugging of functions in packages. Fixes #1948
>
> [Dave Page] Remove unnecessary whitespace from stored procedure SQL. Fixes
> #2146
>
> [Dave Page] Resolve a number of issues with domains and domain
> constraints. Fixes
>
> [Dave Page] Improve styling for alerts by highlighting the icon.
>
> [Dave Page] Prompt the user to save dirty queries rather than discard them
> for a
>
> [Dave Page] Fix alert layout when more than one are shown at once.
>
> [Dave Page] Allow use of Shift+Tab to un-indent selected text. Fixes #1988
>
> [Dave Page] Add some useful make targets for bundling and linting.
>
> [Dave Page] Remove unused minimisation code
>
> [Dave Page] UI tweaks for the query history.
>
> [Dave Page] In subnodes without dropdowns, the rows had smaller height,
> making them
>
> [Dave Page] Ensure the save password option is enabled when creating a
> server. Fixes
>
> [akshay.joshi] Added support of Declarative Partitioning (Range, List) for
> PostgreSQL
>
> [akshay.joshi] Fixed issue by adding extra check for 'relkind' column
>
> [akshay.joshi] Partitioned table does not present in grant wizard. Fixes
> #2543
>
> [Dave Page] Fix clear history.
>
> [Dave Page] Enable pretty html output of our karma test results. While
> running yarn
>
> [Dave Page] Fix the formatting of the ROWS option for functions etc. Fixes
> #2537
>
> [Dave Page] Improve spacing around delete icons in backform tables.
>
> [Dave Page] Fix handling of CREATE TABLE OF . Fixes #2534
>
> [Dave Page] Fix alert animation on IE11
>
> [Dave Page] Open query tool and debugger windows in new tabs by default.
>
> [Dave Page] Revert "Open query tool and debugger windows in new tabs by
> default."
>
> [Dave Page] Update dialogue error styling to match alerts
>
> [Dave Page] Fix history view in IE11.
>
> [Dave Page] Use the correct font for alerts on IE.
>
> [Dave Page] Fix history font on IE.
>
> [Dave Page] Update Mingw build for changes in QtWebKit
>
> [Dave Page] Refresh nodes correctly when there is a single child that is
> updated.
>
> [Dave Page] Fix IE11 alert layout again.
>
> [Dave Page] Handle un-mounted drives correctly on Windows. Fixes #1999
>
> [Dave Page] Handle partitioned tables created "OF TYPE". Fixes #2545
>
> [Dave Page] Release notes for 1.6.
>
> [Dave Page] Bump version numbers prior to release.
>
> [Dave Page] Couple of typos in the release notes.
>
> [Dave Page] Fix drop/drop cascade for partitioned tables. Fixes #2550
>
> [Dave Page] Ensure the initial password is properly 

Re: [RM2074][[RM2080]][pgAdmin4] handle large bytea and bytea[] data in datagrid

[Harshal Dhumal]

Hi Dave,

On Tue, Jul 18, 2017 at 8:31 PM, Dave Page  wrote:

>
>
> On Tue, Jul 18, 2017 at 2:40 PM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi Dave,
>>
>> On Tue, Jul 18, 2017 at 1:24 PM, Dave Page  wrote:
>>
>>>
>>>
>>> On Tue, Jul 18, 2017 at 8:26 AM, Harshal Dhumal <
>>> harshal.dhu...@enterprisedb.com> wrote:
>>>
>>>> Hi Dave,
>>>>
>>>>
>>>> On Mon, Jul 17, 2017 at 9:33 PM, Dave Page  wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> On Mon, Jul 17, 2017 at 1:09 PM, Harshal Dhumal <
>>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Please find updated patch. Now placeholder string for bytea and
>>>>>> bytea[] data will only appear in datagrid (view all/1000/500 rows). If 
>>>>>> user
>>>>>> executes query using Query tool then placeholder won't appear (similar to
>>>>>>  pgAdminIII behaviour)
>>>>>>
>>>>>
>>>>> I'm getting the following error when testing this:
>>>>>
>>>>
>>>> I ran all feature test cases 5-6 times and each time they ran
>>>> successfully.
>>>> May be this is occasional failure which we get after running test cases
>>>> for many times.
>>>>
>>>> Please let me know if you are getting this failure consistently at your
>>>> end.
>>>>
>>>
>>> I ran it twice and it failed twice. I can't keep running tests over and
>>> over again as they take quite a while to run and I can't use my machine for
>>> anything else at the same time as occasionally the test browser will grab
>>> focus.
>>>
>>
>> Please find updated patch.
>> I have slightly modified feature test case for 'view data dml queries'
>> (though I wasn't getting failure as mentioned).
>>
>>
> Thanks, that passes.
>
> Now I've played with it though, I think that the query tool should also
> hide the binary data.  I don't see a good reason to see it in one place but
> not another - ultimately it's still not human readable.
>
> Also, we need to make it look like the [null] placeholder I think; e.g.
> use [ ] instead of < > and give it a lighter colour.
>
> Please find updated patch. I have fixed all of above review comments.


>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
diff --git a/web/pgadmin/feature_tests/pg_datatype_validation_test.py b/web/pgadmin/feature_tests/pg_datatype_validation_test.py
index 149a496..703f0c5 100644
--- a/web/pgadmin/feature_tests/pg_datatype_validation_test.py
+++ b/web/pgadmin/feature_tests/pg_datatype_validation_test.py
@@ -103,7 +103,7 @@ class PGDataypeFeatureTest(BaseFeatureTest):
 '922337203685.922337203685', '-92233720368547758.08',
 '{1,2,3}', '{NaN,NaN,NaN}',
 'Infinity', '{Infinity}',
-r'\336\255\276\357', r'{"\\336\\255\\276\\357","\\336\\255\\276\\357"}'
+r'[binary data]', r'[binary data[]]'
 ]
 
 self.page.open_query_tool()
@@ -122,7 +122,7 @@ class PGDataypeFeatureTest(BaseFeatureTest):
 cells.pop(0)
 for val, cell in zip(expected_output, cells):
 try:
-source_code = cell.get_attribute('innerHTML')
+source_code = cell.text
 
 PGDataypeFeatureTest.check_result(
 source_code,
diff --git a/web/pgadmin/feature_tests/view_data_dml_queries.py b/web/pgadmin/feature_tests/view_data_dml_queries.py
index 622387e..e8cf598 100644
--- a/web/pgadmin/feature_tests/view_data_dml_queries.py
+++ b/web/pgadmin/feature_tests/view_data_dml_queries.py
@@ -101,6 +101,7 @@ CREATE TABLE public.defaults
 # Open Object -> View/Edit data
 self._view_data_grid()
 
+self.page.wait_for_query_tool_loading_indicator_to_disappear()
 # Run test to insert a new row in table with default values
 self._add_row()
 self._verify_row_data(True)
@@ -160,6 +161,7 @@ CREATE TABLE public.defaults
 Returns: None
 
 """
+
 self.wait.until(EC.visibility_of_element_located(
 (By.XPATH, xpath)), CheckForViewDataTest.TIMEOUT_STRING
 )
@@ -197,6 +199,

Re: [pgadmin4][PATCH] Query History Arrow Navigation and Styling

[Harshal Dhumal]

Hi,

Currently if user clicks on empty white area below history entry then arrow
navigation does not work.
Attached patch fixes this issue.

[image: Inline image 1]




-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, Jul 21, 2017 at 1:21 AM, Dave Page  wrote:

> Ah, OK - I was working on them one at a time.
>
> Thanks, patches applied!
>
> On Thu, Jul 20, 2017 at 7:01 PM, Matthew Kleiman 
> wrote:
>
>> Hi Dave,
>>
>> In our previous email, the second patch fixed the linting issue from the
>> first patch.
>> However, we've remade these two patches so they both pass the linting
>> test.
>>
>> Thanks,
>> Matt and João
>>
>>
>> On Thu, Jul 20, 2017 at 12:56 PM, Dave Page  wrote:
>>
>>>
>>>
>>> On Thu, Jul 20, 2017 at 2:18 PM, Joao De Almeida Pereira <
>>> jdealmeidapere...@pivotal.io> wrote:
>>>
>>>> Hello Dave,
>>>> Did you use the latest one patch we sent?
>>>> We realised that this was an issue in the second email we sent.
>>>>
>>> Just checked again, and yes, that was the third and latest version I've
>>> received..
>>>
>>>
>>>
>>>> Thanks
>>>> Joao
>>>>
>>>> On Thu, Jul 20, 2017, 8:15 AM Dave Page  wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> On Wed, Jul 19, 2017 at 8:33 PM, Shruti Iyer  wrote:
>>>>>
>>>>>> Hi again!
>>>>>>
>>>>>> Attached are the most updated patches. We found an issue with the
>>>>>> previous implementation. So we generated the new patches.
>>>>>>
>>>>>
>>>>> The 01 patch fails linting:
>>>>>
>>>>> /Users/dpage/git/pgadmin4/web/regression/javascript/history/
>>>>> query_history_spec.jsx
>>>>>   188:9   error  Expected indentation of 10 spaces but found 8   indent
>>>>>   196:7   error  Expected indentation of 8 spaces but found 6indent
>>>>>   198:7   error  Expected indentation of 8 spaces but found 6indent
>>>>>   202:12  error  Expected indentation of 10 spaces but found 11  indent
>>>>>   205:11  error  Expected indentation of 8 spaces but found 10   indent
>>>>>   212:9   error  Expected indentation of 10 spaces but found 8   indent
>>>>>   214:11  error  Expected indentation of 8 spaces but found 10   indent
>>>>>   217:11  error  Expected indentation of 12 spaces but found 10  indent
>>>>>   219:9   error  Expected indentation of 6 spaces but found 8
>>>>>  indent
>>>>>
>>>>> Can you please fix that up and double-check the tests pass?
>>>>>
>>>>> Thanks.
>>>>>
>>>>> --
>>>>> Dave Page
>>>>> Blog: http://pgsnake.blogspot.com
>>>>> Twitter: @pgsnake
>>>>>
>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>> The Enterprise PostgreSQL Company
>>>>>
>>>>
>>>
>>>
>>> --
>>> Dave Page
>>> Blog: http://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EnterpriseDB UK: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>>
>>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
diff --git a/web/pgadmin/static/jsx/history/query_history.jsx b/web/pgadmin/static/jsx/history/query_history.jsx
index a276e53..940731c 100644
--- a/web/pgadmin/static/jsx/history/query_history.jsx
+++ b/web/pgadmin/static/jsx/history/query_history.jsx
@@ -172,13 +172,14 @@ export default class QueryHistory extends React.Component {
   
 
+ className="query-history"
+ onKeyDown={this.navigateUpAndDown}
+ tabIndex='0'>
   
 {this.retrieveOrderedHistory()
   .map((entry, index) =>
-
+
   
diff --git a/web/pgadmin/static/scss/sqleditor/_history.scss b/web/pgadmin/static/scss/sqleditor/_history.scss
index ed0c10f..942af03 100644
--- a/web/pgadmin/static/scss/sqleditor/_history.scss
+++ b/web/pgadmin/static/scss/sqleditor/_history.scss
@@ -1,4 +1,5 @@
 .query-history {
+  height: 100%;
   .list-item {
 border-bottom: 1px solid $color-gray-3;
   }

Re: [pgadmin4][PATCH] Query History Arrow Navigation and Styling

[Harshal Dhumal]

Hi,

On Fri, Jul 21, 2017 at 2:24 PM, Dave Page  wrote:

> Hi
>
> On Fri, Jul 21, 2017 at 9:38 AM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi,
>>
>> Currently if user clicks on empty white area below history entry then
>> arrow navigation does not work.
>> Attached patch fixes this issue.
>>
>
> This fixes that problem, but stops it working if I *don't* click on the
> white area first. We need both I think :-)
>
Yes we need both. Please find updated patch


>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
diff --git a/web/pgadmin/static/jsx/history/query_history.jsx b/web/pgadmin/static/jsx/history/query_history.jsx
index a276e53..8efe08b 100644
--- a/web/pgadmin/static/jsx/history/query_history.jsx
+++ b/web/pgadmin/static/jsx/history/query_history.jsx
@@ -57,13 +57,13 @@ export default class QueryHistory extends React.Component {
 
   refocus() {
 if (this.state.history.length > 0) {
-  this.retrieveSelectedQuery().parentElement.focus();
+  this.retrieveQueryListPane().focus();
 }
   }
 
-  retrieveSelectedQuery() {
+  retrieveQueryListPane() {
 return ReactDOM.findDOMNode(this)
-  .getElementsByClassName('selected')[0];
+  .getElementsByClassName('query-history')[0];
   }
 
   getCurrentHistoryDetail() {
@@ -172,13 +172,14 @@ export default class QueryHistory extends React.Component {
   
 
+ className="query-history"
+ onKeyDown={this.navigateUpAndDown}
+ tabIndex='0'>
   
 {this.retrieveOrderedHistory()
   .map((entry, index) =>
-
+
   
@@ -198,4 +199,4 @@ QueryHistory.propTypes = {
 
 export {
   QueryHistory,
-};
\ No newline at end of file
+};
diff --git a/web/pgadmin/static/scss/sqleditor/_history.scss b/web/pgadmin/static/scss/sqleditor/_history.scss
index ed0c10f..942af03 100644
--- a/web/pgadmin/static/scss/sqleditor/_history.scss
+++ b/web/pgadmin/static/scss/sqleditor/_history.scss
@@ -1,4 +1,5 @@
 .query-history {
+  height: 100%;
   .list-item {
 border-bottom: 1px solid $color-gray-3;
   }
diff --git a/web/regression/javascript/history/query_history_spec.jsx b/web/regression/javascript/history/query_history_spec.jsx
index 2fa6a7d..c3b09ba 100644
--- a/web/regression/javascript/history/query_history_spec.jsx
+++ b/web/regression/javascript/history/query_history_spec.jsx
@@ -36,11 +36,11 @@ describe('QueryHistory', () => {
 describe('when we switch to the query history tab', () => {
   beforeEach(() => {
 historyWrapper.node.refocus();
-spyOn(historyWrapper.node, 'retrieveSelectedQuery');
+spyOn(historyWrapper.node, 'retrieveQueryListPane');
   });
 
   it('does not try to focus on any element', () => {
-expect(historyWrapper.node.retrieveSelectedQuery).not.toHaveBeenCalled();
+expect(historyWrapper.node.retrieveQueryListPane).not.toHaveBeenCalled();
   });
 });
 
@@ -49,7 +49,7 @@ describe('QueryHistory', () => {
   expect(foundChildren.length).toBe(0);
   done();
 });
-
+
 it('nothing is displayed on right panel', (done) => {
   let foundChildren = historyWrapper.find(QueryHistoryDetail);
   expect(foundChildren.length).toBe(1);
@@ -264,7 +264,7 @@ describe('QueryHistory', () => {
 
   beforeEach(() => {
 selectedListItem = ReactDOM.findDOMNode(historyWrapper.node)
-  .getElementsByClassName('selected')[0].parentElement;
+  .getElementsByClassName('query-history')[0];
 
 spyOn(selectedListItem, 'focus');
 historyWrapper.node.refocus();

Re: [pgadmin4][PATCH] Query History Arrow Navigation and Styling

[Harshal Dhumal]

On Jul 21, 2017 4:23 PM, "Dave Page"  wrote:



On Fri, Jul 21, 2017 at 11:05 AM, Harshal Dhumal <
harshal.dhu...@enterprisedb.com> wrote:

> Hi,
>
> On Fri, Jul 21, 2017 at 2:24 PM, Dave Page  wrote:
>
>> Hi
>>
>> On Fri, Jul 21, 2017 at 9:38 AM, Harshal Dhumal <
>> harshal.dhu...@enterprisedb.com> wrote:
>>
>>> Hi,
>>>
>>> Currently if user clicks on empty white area below history entry then
>>> arrow navigation does not work.
>>> Attached patch fixes this issue.
>>>
>>
>> This fixes that problem, but stops it working if I *don't* click on the
>> white area first. We need both I think :-)
>>
> Yes we need both. Please find updated patch
>

Why all the changes to yarn.lock?

Are there? I can't see any changes.

Sent from my mobile


-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

[RM2579][pgAdmin4] Set default file listing layout as list instead of grid

[Harshal Dhumal]

Hi,

Please find attached patch to set default layout of file listing as a list
in file manager.
Also replaced alertify with out custom alertifywrapper in file manager
utils.js


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/misc/file_manager/templates/file_manager/index.html b/web/pgadmin/misc/file_manager/templates/file_manager/index.html
index 1933895..80c85fb 100755
--- a/web/pgadmin/misc/file_manager/templates/file_manager/index.html
+++ b/web/pgadmin/misc/file_manager/templates/file_manager/index.html
@@ -26,8 +26,8 @@
 
 
-
-
+
+
 
 
 
diff --git a/web/pgadmin/misc/file_manager/templates/file_manager/js/file_manager_config.json b/web/pgadmin/misc/file_manager/templates/file_manager/js/file_manager_config.json
index f74430d..fcc242d 100644
--- a/web/pgadmin/misc/file_manager/templates/file_manager/js/file_manager_config.json
+++ b/web/pgadmin/misc/file_manager/templates/file_manager/js/file_manager_config.json
@@ -2,7 +2,7 @@
  "options": {
   "culture": "en",
   "lang": "py",
-  "defaultViewMode": "grid",
+  "defaultViewMode": "list",
   "autoload": true,
   "showFullPath": false,
   "dialog_type": "{{data.dialog_type}}",
diff --git a/web/pgadmin/misc/file_manager/templates/file_manager/js/utility.js b/web/pgadmin/misc/file_manager/templates/file_manager/js/utility.js
index fd8f25c..56be16a 100755
--- a/web/pgadmin/misc/file_manager/templates/file_manager/js/utility.js
+++ b/web/pgadmin/misc/file_manager/templates/file_manager/js/utility.js
@@ -1601,7 +1601,8 @@ pgAdmin.FileUtils = {
 
   // create a new folder
   var getFolderName = function(value) {
-var fname = value;
+var fname = value,
+alertifyWrapper = new AlertifyWrapper();
 
 if (fname != '') {
   foldername = fname;
@@ -1609,14 +1610,17 @@ pgAdmin.FileUtils = {
   $.getJSON(pgAdmin.FileUtils.fileConnector + '?mode=addfolder&path=' + $('.currentpath').val() + '&name=' + foldername, function(resp) {
 var result = resp.data.result;
 if (result.Code === 1) {
-  alertify.success(lg.successful_added_folder);
+  alertifyWrapper.success(lg.successful_added_folder);
   getFolderInfo(result.Parent);
 } else {
-  alertify.error(result.Error);
+  alertifyWrapper.error(result.Error);
 }
+  }).
+  fail(function (resp) {
+alertifyWrapper.error(gettext('Error occurred while saving folder.'));
   });
 } else {
-  alertify.error(lg.no_foldername);
+  alertifyWrapper.error(lg.no_foldername);
 }
   };
 

Re: [pgAdmin4][Patch]: Allow user to cancel long running queries from dashboard

[Harshal Dhumal]

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Jul 24, 2017 at 8:11 PM, Dave Page  wrote:

>
>
> On Mon, Jul 24, 2017 at 3:28 PM, Shirley Wang  wrote:
>
>> 2-3 days is a lot of valuable engineering time. Is this a 'drop
>> everything now' kind of feature or can this wait for some user validation
>> on a mock up first?
>>
>
> Most of the time will likely be on the infrastructure to change the
> display to a subnode control. If you have some cycles to mockup potential
> layouts for the subnode view and have them validated, please feel free,
> however, that seems like an awful lot of work to me to display some missing
> SQL using a standard control.
>
Regarding SQL display: Developing simple control to show codemirror in
disabled state (for now) wont take that much time.


>
>
>>
>> On Jul 24, 2017, at 20:41, Murtuza Zabuawala <
>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>
>> Sure.
>>
>> RM created: https://redmine.postgresql.org/issues/2597.
>>
>> On Mon, Jul 24, 2017 at 6:04 PM, Dave Page  wrote:
>>
>>>
>>>
>>> On Mon, Jul 24, 2017 at 1:12 PM, Murtuza Zabuawala <
>>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>>
>>>> Hi Dave,
>>>>
>>>> On Mon, Jul 24, 2017 at 4:44 PM, Dave Page  wrote:
>>>>
>>>>> Thanks, applied.
>>>>>
>>>>> How much effort do you think it would take to turn the table into a
>>>>> subnode control so we can show the SQL query and other missing fields from
>>>>> pg_stat_activity on that tab?
>>>>>
>>>> I think around 2-3 days would be needed.
>>>>
>>>
>>> OK - can you add a Redmine and work on that please? It's been a known
>>> deficiency for quite a while, and whilst working on this patch I realised
>>> just how badly we really need to see the SQL there.
>>>
>>> Thanks.
>>>
>>>
>>>>
>>>>>
>>>> On Mon, Jul 24, 2017 at 11:45 AM, Murtuza Zabuawala <
>>>>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>>>>
>>>>>> Hi Dave,
>>>>>>
>>>>>> Please find updated patch.
>>>>>>
>>>>>> On Tue, Jul 18, 2017 at 8:05 PM, Murtuza Zabuawala <
>>>>>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>>>>>
>>>>>>> Hi Shirley,
>>>>>>>
>>>>>>> On Tue, Jul 18, 2017 at 1:21 AM, Shirley Wang 
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi!
>>>>>>>>
>>>>>>>> I can't seem to get the patch to completely work on my computer,
>>>>>>>> only the close icon shows up plus the dialog and success/error 
>>>>>>>> messages,
>>>>>>>> but here are some comments:
>>>>>>>>
>>>>>>>> Because  we are just cancelling the active running query, so if the
>>>>>>> start of the session is 'Active' when you cancel it will simply goto 
>>>>>>> 'Idle'
>>>>>>> stat.
>>>>>>>
>>>>>>>> +1 to Dave's comment about refreshing after the cancel operation
>>>>>>>>
>>>>>>>> I'll fix this.
>>>>>>>
>>>>>>>> - We're working on a patch for updating alerts in the Dashboard tab
>>>>>>>> which updates the grays in the Database activities panel and changes 
>>>>>>>> the
>>>>>>>> border around the refresh button and search bar to 1px. This hasn't 
>>>>>>>> been
>>>>>>>> submitted yet but just a heads up as you work on the alignment.
>>>>>>>>
>>>>>>>> - Something to consider is how a super user will identify which
>>>>>>>> session should be closed. Is that information there?
>>>>>>>>
>>>>>>>> I think super user can cancel everything except main connection
>>>>>>> session & as Dave mentioned in previous email that background workers in
>>>>>>> PG10.
>>>>>>>
>>>>>>>> - Are there sessions that should never be clos

Re: [RM2579][pgAdmin4] Set default file listing layout as list instead of grid

[Harshal Dhumal]

Hi,

On Mon, Jul 24, 2017 at 4:23 PM, Dave Page  wrote:

> Hi
>
> On Mon, Jul 24, 2017 at 11:34 AM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi,
>>
>> Please find attached patch to set default layout of file listing as a
>> list in file manager.
>> Also replaced alertify with out custom alertifywrapper in file manager
>> utils.js
>>
>
> This isn't a bad idea on the face of it, but there are some things to
> fix/consider:
>
> - The HTML file seems to be missing translation markers. Can you add them
> throughout please?
> - We should save the users preference in the config database.
> - The grid view seems to underline the file size and for no apparent
> reason change the mouse cursor to ? on mouseover. Let's change the text
> style to be consistent and get rid of the mouseover.
>
Please find updated patch


>
> Thanks!
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
diff --git a/web/pgadmin/misc/file_manager/__init__.py b/web/pgadmin/misc/file_manager/__init__.py
index 5d7ef93..81c9041 100644
--- a/web/pgadmin/misc/file_manager/__init__.py
+++ b/web/pgadmin/misc/file_manager/__init__.py
@@ -26,6 +26,7 @@ from flask_security import login_required
 from pgadmin.utils import PgAdminModule
 from pgadmin.utils import get_storage_directory
 from pgadmin.utils.ajax import make_json_response
+from pgadmin.utils.preferences import Preferences
 
 # Checks if platform is Windows
 if _platform == "win32":
@@ -172,6 +173,13 @@ class FileManagerModule(PgAdminModule):
 gettext("Last directory visited"), 'text', '/',
 category_label=gettext('Options')
 )
+self.grid_layout_view = self.preference.register(
+'options', 'grid_layout_view',
+gettext("Grid layout view"), 'options', 'list',
+category_label=gettext('Options'),
+options=[{'label': gettext('List'), 'value': 'list'},
+ {'label': gettext('Grid'), 'value': 'grid'}]
+)
 
 
 # Initialise the module
@@ -232,9 +240,13 @@ def file_manager_config(trans_id):
 """render the required json"""
 # trans_id = Filemanager.create_new_transaction()
 data = Filemanager.get_trasaction_selection(trans_id)
+pref = Preferences.module('file_manager')
+grid_layout_view = pref.preference('grid_layout_view').get()
+
 return Response(response=render_template(
 "file_manager/js/file_manager_config.json", _=gettext,
-data=data),
+data=data,
+grid_layout_view=grid_layout_view),
 status=200,
 mimetype="application/json")
 
diff --git a/web/pgadmin/misc/file_manager/templates/file_manager/index.html b/web/pgadmin/misc/file_manager/templates/file_manager/index.html
index 1933895..e0341ee 100755
--- a/web/pgadmin/misc/file_manager/templates/file_manager/index.html
+++ b/web/pgadmin/misc/file_manager/templates/file_manager/index.html
@@ -6,9 +6,9 @@
 
 
 
-
+
 
-
 
 
@@ -17,17 +17,17 @@
 
 
 
-
-
+
+
 
-
+
 
-
+
 
-
-
-
+
+
 
 
 
@@ -39,17 +39,17 @@
 
 
 
-Are you sure you want to delete this item ?
+{{ _('Are you sure you want to delete this item ?') }}
 
-  YES
-  NO
+  {{ _('YES') }}
+  {{ _('NO') }}
 
 
 
-Are you sure you want to replace this file ?
+{{ _('Are you sure you want to replace this file?') }}
 
-  YES
-  NO
+  {{ _('YES') }}
+  {{ _('NO') }}
 
 
 
diff --git a/web/pgadmin/misc/file_manager/templates/file_manager/js/file_manager_config.json b/web/pgadmin/misc/file_manager/templates/file_manager/js/file_manager_config.json
index f74430d..ed8efcf 100644
--- a/web/pgadmin/misc/file_manager/templates/file_manager/js/file_manager_config.json
+++ b/web/pgadmin/misc/file_manager/templates/file_manager/js/file_manager_config.json
@@ -2,7 +2,7 @@
  "options": {
   "culture": "en",
   "lang": "py",
-  "defaultViewMode": "grid&qu

Re: [RM2579][pgAdmin4] Set default file listing layout as list instead of grid

[Harshal Dhumal]

Hi Dave,

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Wed, Jul 26, 2017 at 5:41 PM, Dave Page  wrote:

>
>
> On Wed, Jul 26, 2017 at 11:06 AM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi,
>>
>> On Mon, Jul 24, 2017 at 4:23 PM, Dave Page  wrote:
>>
>>> Hi
>>>
>>> On Mon, Jul 24, 2017 at 11:34 AM, Harshal Dhumal <
>>> harshal.dhu...@enterprisedb.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Please find attached patch to set default layout of file listing as a
>>>> list in file manager.
>>>> Also replaced alertify with out custom alertifywrapper in file manager
>>>> utils.js
>>>>
>>>
>>> This isn't a bad idea on the face of it, but there are some things to
>>> fix/consider:
>>>
>>> - The HTML file seems to be missing translation markers. Can you add
>>> them throughout please?
>>> - We should save the users preference in the config database.
>>> - The grid view seems to underline the file size and for no apparent
>>> reason change the mouse cursor to ? on mouseover. Let's change the text
>>> style to be consistent and get rid of the mouseover.
>>>
>> Please find updated patch
>>
>
> Thanks, committed with some naming tweaks... however, I do think the
> setting should be updated when the user toggles it within the file
> dialogue. Can you add code to set that config option when the user toggles
> the mode on the fly please?
>
Yes... and patch attached.


>
> Thanks.
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
diff --git a/web/pgadmin/misc/file_manager/__init__.py b/web/pgadmin/misc/file_manager/__init__.py
index 3d4fb53..843ec61 100644
--- a/web/pgadmin/misc/file_manager/__init__.py
+++ b/web/pgadmin/misc/file_manager/__init__.py
@@ -155,7 +155,8 @@ class FileManagerModule(PgAdminModule):
 'file_manager.index',
 'file_manager.get_trans_id',
 'file_manager.delete_trans_id',
-'file_manager.save_last_dir'
+'file_manager.save_last_dir',
+'file_manager.save_file_dialog_view'
 ]
 
 def get_file_size_preference(self):
@@ -288,6 +289,17 @@ def save_last_directory_visited(trans_id):
 data={'status': True}
 )
 
+@blueprint.route(
+"/save_file_dialog_view/", methods=["POST"],
+endpoint='save_file_dialog_view'
+)
+@login_required
+def save_file_dialog_view(trans_id):
+blueprint.file_dialog_view.set(req.json['view'])
+return make_json_response(
+data={'status': True}
+)
+
 
 class Filemanager(object):
 """FileManager Class."""
diff --git a/web/pgadmin/misc/file_manager/templates/file_manager/js/utility.js b/web/pgadmin/misc/file_manager/templates/file_manager/js/utility.js
index 2d3fcdd..4a2754c 100755
--- a/web/pgadmin/misc/file_manager/templates/file_manager/js/utility.js
+++ b/web/pgadmin/misc/file_manager/templates/file_manager/js/utility.js
@@ -60,6 +60,17 @@ var setViewButtonsFor = function(viewMode) {
   }
 };
 
+var save_file_dialog_view = function(view, trans_id) {
+  return $.ajax({
+url: url_for('file_manager.save_file_dialog_view', {'trans_id': trans_id}),
+type: 'POST',
+async: true,
+data: JSON.stringify({'view':view}),
+contentType: 'application/json'
+  });
+};
+
+
 /*
  * preg_replace
  */
@@ -1066,6 +1077,7 @@ pgAdmin.FileUtils = {
 config;
 
 this.fileConnector = fileConnector;
+this.transId = t_id;
 // load user configuration file
 if (cfg.readyState == 4) {
   this.config = config = JSON.parse(cfg.responseText);
@@ -1235,6 +1247,7 @@ pgAdmin.FileUtils = {
   $('.fileinfo').data('view', 'grid');
   enable_disable_btn();
   getFolderInfo($('.currentpath').val());
+  save_file_dialog_view('grid', pgAdmin.FileUtils.transId);
 });
 
 // Show list mode
@@ -1243,6 +1256,7 @@ pgAdmin.FileUtils = {
   $('.fileinfo').data('view', 'list');
   enable_disable_btn();
   getFolderInfo($('.currentpath').val());
+  save_file_dialog_view('list', pgAdmin.FileUtils.transId);
 });
 
 // Provide initial values for upload form, status, etc.

Re: pgAdmin 4 commit: Fix test assertion.

[Harshal Dhumal]

Well I'm able run all feature test cases on python 2.7 with latest pull.


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Jul 31, 2017 at 5:43 PM, Robert Eckhardt 
wrote:

> It fails because the test was broken and has never been fixed.
>
> Rob
>
> On Jul 31, 2017 8:03 PM, "Murtuza Zabuawala"  enterprisedb.com> wrote:
>
>> Hi Robert,
>>
>> I have the latest pull and still this test is fails every time on my
>> machine, not sure if it is because of Python3.6.
>>
>> On Mon, Jul 31, 2017 at 5:27 PM, Robert Eckhardt 
>> wrote:
>>
>>> This was broken when the clear button and the edit button were changed
>>> to drop downs.  Happened a few weeks ago.
>>>
>>> Rob
>>>
>>> On Jul 31, 2017 6:51 PM, "Murtuza Zabuawala" <
>>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>>
>>> Hi All,
>>>
>>> When I run the feature tests, the gives test always fails (
>>> 'query_tool_journey_test.py'),
>>>
>>> Python: 3.6 (64-bit)
>>> Platform: OS X (10.10.5)
>>>
>>> Am I the only one who gets this error?
>>>
>>>
>>> ==
>>> ERROR: runTest (pgadmin.feature_tests.query_t
>>> ool_journey_test.QueryToolJourneyTest)
>>> Tests the path through the query tool
>>> --
>>> Traceback (most recent call last):
>>>   File "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_
>>> tests/query_tool_journey_test.py", line 46, in runTest
>>> self._test_history_tab()
>>>   File "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_
>>> tests/query_tool_journey_test.py", line 104, in _test_history_tab
>>> query_we_need_to_scroll_to
>>>   File "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_
>>> tests/query_tool_journey_test.py", line 141, in
>>> _assert_not_clickable_because_out_of_view
>>> self.assertRaises(self.page.click_element(element))
>>>   File 
>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/case.py",
>>> line 728, in assertRaises
>>> return context.handle('assertRaises', args, kwargs)
>>>   File 
>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/case.py",
>>> line 158, in handle
>>> (name, self._base_type_str))
>>> TypeError: assertRaises() arg 1 must be an exception type or tuple of
>>> exception types
>>>
>>> --
>>> Ran 10 tests in 359.301s
>>>
>>> FAILED (errors=1, skipped=1)
>>>
>>> ==
>>> Test Result Summary
>>> ==
>>>
>>> PG_95:
>>>
>>> 8 tests passed
>>> 1 test failed:
>>> QueryToolJourneyTest (Tests the path through the query tool)
>>> 1 test skipped:
>>> CheckDebuggerForXssFeatureTest (Tests to check if Debugger is vulnerable
>>> to XSS)
>>>
>>>
>>> --
>>> Regards,
>>> Murtuza Zabuawala
>>> EnterpriseDB: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>>> On Fri, Jul 21, 2017 at 8:46 PM, Dave Page  wrote:
>>>
>>>> Fix test assertion.
>>>>
>>>> Branch
>>>> --
>>>> master
>>>>
>>>> Details
>>>> ---
>>>> https://git.postgresql.org/gitweb?p=pgadmin4.git;a=commitdif
>>>> f;h=2ebb40819534d79376a577a149a8a5db124b
>>>> Author: Sarah McAlear 
>>>>
>>>> Modified Files
>>>> --
>>>> web/pgadmin/feature_tests/query_tool_journey_test.py | 2 +-
>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>>
>>>
>>>
>>

[pgAdmin4][RM2586] Cleanup feature test

[Harshal Dhumal]

Hi,

Please find attached patch to improve feature test execution time.
Now on my machine overall execution time is cut down to 280 seconds from
400+ seconds

Changes:

1. Removed fixed python time.sleeps where ever possible.
2. Removed connect to server test cases.
3. Query tool test cases:
 i. Merged 3 test cases On demand result on scroll, grid select all and
column select all.
 ii. Merged 3 test cases Explain query, Explain query with verbose and
Explain query with cost.
 iii. Merged 3 test cases Explain analyze query, Explain analyze with
buffers and Explain analyze with timing.
4. Improved debugger XSS test case execution time.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/feature_tests/connect_to_server_feature_test.py b/web/pgadmin/feature_tests/connect_to_server_feature_test.py
deleted file mode 100644
index 97e96f2..000
--- a/web/pgadmin/feature_tests/connect_to_server_feature_test.py
+++ /dev/null
@@ -1,84 +0,0 @@
-##
-#
-# pgAdmin 4 - PostgreSQL Tools
-#
-# Copyright (C) 2013 - 2017, The pgAdmin Development Team
-# This software is released under the PostgreSQL Licence
-#
-##
-
-import time
-from selenium.webdriver import ActionChains
-
-import config as app_config
-from regression.feature_utils.base_feature_test import BaseFeatureTest
-from regression.python_test_utils import test_utils
-
-
-class ConnectsToServerFeatureTest(BaseFeatureTest):
-"""
-Tests that a database connection can be created from the UI
-"""
-scenarios = [
-("Test database connection", dict())
-]
-
-def before(self):
-connection = test_utils.get_db_connection(self.server['db'],
-  self.server['username'],
-  self.server['db_password'],
-  self.server['host'],
-  self.server['port'],
-  self.server['sslmode'])
-test_utils.drop_database(connection, "acceptance_test_db")
-test_utils.create_database(self.server, "acceptance_test_db")
-test_utils.create_table(self.server, "acceptance_test_db", "test_table")
-
-def runTest(self):
-"""This function tests that a database connection can be created from
-the UI"""
-self.assertEqual(app_config.APP_NAME, self.page.driver.title)
-self.page.wait_for_spinner_to_disappear()
-
-self._connects_to_server()
-self._tables_node_expandable()
-
-def after(self):
-self.page.remove_server(self.server)
-
-connection = test_utils.get_db_connection(self.server['db'],
-  self.server['username'],
-  self.server['db_password'],
-  self.server['host'],
-  self.server['port'],
-  self.server['sslmode'])
-test_utils.drop_database(connection, "acceptance_test_db")
-
-def _connects_to_server(self):
-self.page.find_by_xpath("//*[@class='aciTreeText' and .='Servers']").click()
-time.sleep(2)
-self.page.driver.find_element_by_link_text("Object").click()
-ActionChains(self.page.driver) \
-.move_to_element(self.page.driver.find_element_by_link_text("Create")) \
-.perform()
-self.page.find_by_partial_link_text("Server...").click()
-
-server_config = self.server
-self.page.fill_input_by_field_name("name", server_config['name'])
-self.page.find_by_partial_link_text("Connection").click()
-self.page.fill_input_by_field_name("host", server_config['host'])
-self.page.fill_input_by_field_name("port", server_config['port'])
-self.page.fill_input_by_field_name("username", server_config['username'])
-self.page.fill_input_by_field_name("password", server_config['db_password'])
-self.page.find_by_xpath("//button[contains(.,'Save')]").click()
-
-def _tables_node_expandable(self):
-self.page.toggle_open_server(self.server['name'])
-self.p

Re: [pgadmin4] unable to pull remote

[Harshal Dhumal]

hi

Change below url in your git config
from *git://git.postgresql.org/git/pgadmin4.git
<http://git.postgresql.org/git/pgadmin4.git>* to
*http://git.postgresql.org/git/pgadmin4.git
<http://git.postgresql.org/git/pgadmin4.git>*



-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Wed, Aug 9, 2017 at 2:39 PM, Sarah McAlear  wrote:

> Hi Hackers!
>
> We noticed that as of some time today (it worked this morning) we are
> unable to pull git.postgresql.org/git/pgadmin4.git. Is this a known
> issue? Our other repos seem to work fine still.
>
> Thanks,
> Sarah
>

Re: [pgAdmin4][Patch]: Fixed RM #2603 - Import/Export File issues

[Harshal Dhumal]

On Fri, Aug 4, 2017 at 6:32 PM, Ashesh Vashi 
wrote:

> Harshal,
>
> On Fri, Jul 28, 2017 at 4:00 PM, Dave Page  wrote:
>
>> Ashesh, can you deal with this please? Thanks.
>>
>> On Thu, Jul 27, 2017 at 5:08 AM, Khushboo Vashi <
>> khushboo.va...@enterprisedb.com> wrote:
>>
>>> Hi,
>>>
>>> Please find the attached patch to fix the RM #2603: Import/Export File
>>> issues.
>>>
>>
>>> Fixed Issues:
>>>  1. Couldn't click on the File Control once gets an error
>>>  2. The encoding is not in alphabetical order
>>>
>> Please review it.
>

It looks good to me.

thanks,


>
> -- Thanks, Ashesh
>
>>
>>> Thanks,
>>> Khsuhboo
>>>
>>
>>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>
>

Re: [pgAdmin4][Patch]: Use the correct resultset type for Type module

[Harshal Dhumal]

On Wed, Aug 9, 2017 at 8:57 AM, Ashesh Vashi 
wrote:

> On Thu, Aug 3, 2017 at 2:41 PM, Murtuza Zabuawala  enterprisedb.com> wrote:
>
>> Hi,
>>
>> PFA minor patch to use fetch dict based data instead of 2d-array based in
>> Type module as we are adding keys on the fly.
>> There is no RM just a minor enhancement.
>>
> Harshal,
>
> Please review this one.
>
I tested this and it's working fine.



>
> --
>
> Thanks & Regards,
>
> Ashesh Vashi
> EnterpriseDB INDIA: Enterprise PostgreSQL Company
> 
>
>
> *http://www.linkedin.com/in/asheshvashi*
> 
>
>>
>> --
>> Regards,
>> Murtuza Zabuawala
>> EnterpriseDB: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>
>

Re: [PATCH] Work with newer Flask-Security.

[Harshal Dhumal]

Hi,

I tested this change with Flask-Security 3.0.0 and 1.7.5 (we ship pgAdmin4
with 1.7.5) and with 2.7 and 3.0+.
This is working as expected.

Apart from this I also found some other issues with User management.
I have created RM2641 <https://redmine.postgresql.org/issues/2641> for
these issue and also attached patch from same.


Thanks


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Aug 7, 2017 at 7:08 PM, Ashesh Vashi 
wrote:

> Harshal,
>
> Please take a look at it.
>
> --
>
> Thanks & Regards,
>
> Ashesh Vashi
> EnterpriseDB INDIA: Enterprise PostgreSQL Company
> <http://www.enterprisedb.com>
>
>
> *http://www.linkedin.com/in/asheshvashi*
> <http://www.linkedin.com/in/asheshvashi>
>
> On Mon, Aug 7, 2017 at 6:46 PM, Wouter van Kesteren 
> wrote:
>
>> I installed it on my system and i got the following error:
>>
>> Traceback (most recent call last):
>>   File "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask/app.py",
>> line 1997, in __call__
>> return self.wsgi_app(environ, start_response)
>>   File "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask/app.py",
>> line 1985, in wsgi_app
>> response = self.handle_exception(e)
>>   File "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask/app.py",
>> line 1540, in handle_exception
>> reraise(exc_type, exc_value, tb)
>>   File 
>> "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask/_compat.py",
>> line 33, in reraise
>> raise value
>>   File "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask/app.py",
>> line 1982, in wsgi_app
>> response = self.full_dispatch_request()
>>   File "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask/app.py",
>> line 1614, in full_dispatch_request
>> rv = self.handle_user_exception(e)
>>   File "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask/app.py",
>> line 1517, in handle_user_exception
>> reraise(exc_type, exc_value, tb)
>>   File 
>> "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask/_compat.py",
>> line 33, in reraise
>> raise value
>>   File "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask/app.py",
>> line 1610, in full_dispatch_request
>> rv = self.preprocess_request()
>>   File "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask/app.py",
>> line 1831, in preprocess_request
>> rv = func()
>>   File 
>> "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask_principal.py",
>> line 477, in _on_before_request
>> identity = loader()
>>   File 
>> "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask_security/core.py",
>> line 245, in _identity_loader
>> if not isinstance(current_user._get_current_object(),
>> AnonymousUserMixin):
>>   File 
>> "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/werkzeug/local.py",
>> line 306, in _get_current_object
>> return self.__local()
>>   File 
>> "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask_login/utils.py",
>> line 26, in 
>> current_user = LocalProxy(lambda: _get_user())
>>   File 
>> "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask_login/utils.py",
>> line 302, in _get_user
>> current_app.login_manager._load_user()
>>   File 
>> "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask_login/login_manager.py",
>> line 316, in _load_user
>> return self.reload_user()
>>   File 
>> "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask_login/login_manager.py",
>> line 278, in reload_user
>> user = self.user_callback(user_id)
>>   File 
>> "/usr/x86_64-pc-linux-gnu/lib/python3.4/site-packages/flask_security/core.py",
>> line 221, in _user_loader
>> return _security.datastore.find_user(id=user_id)
>> AttributeError: 'NoneType' object has no attribute 'find_user'
>>
>> My system has Flask-Security 3.0.0 which contains a patch to
>> Security.init_app
>>
>> -if not self.datastore:
>> -self.datastore = datastore
>> -datastore = self.datastore
>> +self.app = app
>> +self.datastore = datastore
>>
>> https://github.com/mattupstate/flask-security/commit/3701b91
>> d4d3e41036872a3f35db51e44e0681109
>>

Re: [pgAdmin4][RM2586] Cleanup feature test

[Harshal Dhumal]

Hi,

Please find attached updated patch. In this patch I have removed unused
imports.
Regarding ajax calls on slow network/computer I have already taken care of
those.
Before sending first version of patch I tested this on Murtuza's machine
and also
tested on slow machine as well.


Thanks,
Harshal

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Aug 7, 2017 at 8:55 AM, Sarah McAlear  wrote:

> Hi Harshal!
>
> There are a few files in which there are leftover imports that are not
> needed anymore:
>
>- PGDataypeFeatureTest
>- PgadminPage
>- CheckForXssFeatureTest
>- CheckDebuggerForXssFeatureTest
>
>
> We also noticed that there were quite a few time.sleep functions that
> were removed. This seems great overall, but we think that some of them were
> in place because of varying network and computer speeds. So for example, in
> QueryToolFeatureTest, there is an ajax call that was followed by a
> time.sleep to ensure that it had finished executing before continuing to
> execute. Removing this may reintroduce some flakiness. If there are no
> issues with flakiness after this patch, it seems like a great idea. We ran
> the tests a few times and didn't notice any flakiness, but we're unsure if
> it will be a problem on a different system.
>
> Thanks!
> Wenlin & Sarah
>
>
> On Wed, Aug 2, 2017 at 9:32 PM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi,
>>
>> Please find attached patch to improve feature test execution time.
>> Now on my machine overall execution time is cut down to 280 seconds from
>> 400+ seconds
>>
>> Changes:
>>
>> 1. Removed fixed python time.sleeps where ever possible.
>> 2. Removed connect to server test cases.
>> 3. Query tool test cases:
>>  i. Merged 3 test cases On demand result on scroll, grid select all
>> and column select all.
>>  ii. Merged 3 test cases Explain query, Explain query with verbose
>> and Explain query with cost.
>>  iii. Merged 3 test cases Explain analyze query, Explain analyze with
>> buffers and Explain analyze with timing.
>> 4. Improved debugger XSS test case execution time.
>>
>> --
>> *Harshal Dhumal*
>> *Sr. Software Engineer*
>>
>> EnterpriseDB India: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>
>
diff --git a/web/pgadmin/feature_tests/connect_to_server_feature_test.py b/web/pgadmin/feature_tests/connect_to_server_feature_test.py
deleted file mode 100644
index 97e96f2..000
--- a/web/pgadmin/feature_tests/connect_to_server_feature_test.py
+++ /dev/null
@@ -1,84 +0,0 @@
-##
-#
-# pgAdmin 4 - PostgreSQL Tools
-#
-# Copyright (C) 2013 - 2017, The pgAdmin Development Team
-# This software is released under the PostgreSQL Licence
-#
-##
-
-import time
-from selenium.webdriver import ActionChains
-
-import config as app_config
-from regression.feature_utils.base_feature_test import BaseFeatureTest
-from regression.python_test_utils import test_utils
-
-
-class ConnectsToServerFeatureTest(BaseFeatureTest):
-"""
-Tests that a database connection can be created from the UI
-"""
-scenarios = [
-("Test database connection", dict())
-]
-
-def before(self):
-connection = test_utils.get_db_connection(self.server['db'],
-  self.server['username'],
-  self.server['db_password'],
-  self.server['host'],
-  self.server['port'],
-  self.server['sslmode'])
-test_utils.drop_database(connection, "acceptance_test_db")
-test_utils.create_database(self.server, "acceptance_test_db")
-test_utils.create_table(self.server, "acceptance_test_db", "test_table")
-
-def runTest(self):
-"""This function tests that a database connection can be created from
-the UI"""
-self.assertEqual(app_config.APP_NAME, self.page.driver.title)
-self.page.wait_for_spinner_to_disappear()
-
-self._connects_to_server()
-self._tables_node_expandable()
-
-def after(self):
-self.page.remove_server(self.server)
-
-connection = test_utils.get_db_connection(self.server['db'],
-

Re: [pgAdmin4][RM2586] Cleanup feature test

[Harshal Dhumal]

Hi,

Last week Sarah, Matt and I had call regarding some issues with feature
test cases.
For example in auto commit disable test scenario from query tool test cases
by removing code line
*self.page.find_by_id("btn-auto-commit").click() *the test case scenario
was not falling.
This issue was due to previously on going database transaction. This issue
was also with test scenarios like
auto commit enable and auto rollback enabled. The attached patch fixes this
issue.

@Sarah and Matt can you please give this patch a try to check if issues we
discussed last week are fixed.

Thanks,
Harshal




-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, Aug 11, 2017 at 11:55 AM, Sarah McAlear  wrote:

> Great job on cleaning up the whole feature test suite! This is really
> going to benefit all of the developers a lot.
>
> Thanks,
> Matt & Sarah
>
> On Fri, Aug 11, 2017 at 12:24 PM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi,
>>
>> Please find attached updated patch. In this patch I have removed unused
>> imports.
>> Regarding ajax calls on slow network/computer I have already taken care
>> of those.
>> Before sending first version of patch I tested this on Murtuza's machine
>> and also
>> tested on slow machine as well.
>>
>>
>> Thanks,
>> Harshal
>>
>> --
>> *Harshal Dhumal*
>> *Sr. Software Engineer*
>>
>> EnterpriseDB India: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>> On Mon, Aug 7, 2017 at 8:55 AM, Sarah McAlear 
>> wrote:
>>
>>> Hi Harshal!
>>>
>>> There are a few files in which there are leftover imports that are not
>>> needed anymore:
>>>
>>>- PGDataypeFeatureTest
>>>- PgadminPage
>>>- CheckForXssFeatureTest
>>>- CheckDebuggerForXssFeatureTest
>>>
>>>
>>> We also noticed that there were quite a few time.sleep functions that
>>> were removed. This seems great overall, but we think that some of them were
>>> in place because of varying network and computer speeds. So for example, in
>>> QueryToolFeatureTest, there is an ajax call that was followed by a
>>> time.sleep to ensure that it had finished executing before continuing
>>> to execute. Removing this may reintroduce some flakiness. If there are no
>>> issues with flakiness after this patch, it seems like a great idea. We ran
>>> the tests a few times and didn't notice any flakiness, but we're unsure if
>>> it will be a problem on a different system.
>>>
>>> Thanks!
>>> Wenlin & Sarah
>>>
>>>
>>> On Wed, Aug 2, 2017 at 9:32 PM, Harshal Dhumal <
>>> harshal.dhu...@enterprisedb.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Please find attached patch to improve feature test execution time.
>>>> Now on my machine overall execution time is cut down to 280 seconds
>>>> from 400+ seconds
>>>>
>>>> Changes:
>>>>
>>>> 1. Removed fixed python time.sleeps where ever possible.
>>>> 2. Removed connect to server test cases.
>>>> 3. Query tool test cases:
>>>>  i. Merged 3 test cases On demand result on scroll, grid select all
>>>> and column select all.
>>>>  ii. Merged 3 test cases Explain query, Explain query with verbose
>>>> and Explain query with cost.
>>>>  iii. Merged 3 test cases Explain analyze query, Explain
>>>> analyze with buffers and Explain analyze with timing.
>>>> 4. Improved debugger XSS test case execution time.
>>>>
>>>> --
>>>> *Harshal Dhumal*
>>>> *Sr. Software Engineer*
>>>>
>>>> EnterpriseDB India: http://www.enterprisedb.com
>>>> The Enterprise PostgreSQL Company
>>>>
>>>
>>>
>>
>
diff --git a/web/pgadmin/feature_tests/connect_to_server_feature_test.py b/web/pgadmin/feature_tests/connect_to_server_feature_test.py
deleted file mode 100644
index 97e96f2..000
--- a/web/pgadmin/feature_tests/connect_to_server_feature_test.py
+++ /dev/null
@@ -1,84 +0,0 @@
-##
-#
-# pgAdmin 4 - PostgreSQL Tools
-#
-# Copyright (C) 2013 - 2017, The pgAdmin Development Team
-# This software is released under the PostgreSQL Licence
-#
-##
-
-import t

Re: Next release

[Harshal Dhumal]

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Thu, Aug 24, 2017 at 9:44 PM, Dave Page  wrote:

>
>
> On Thu, Aug 24, 2017 at 10:36 AM, Surinder Kumar <
> surinder.ku...@enterprisedb.com> wrote:
>
>> Hi Dave,
>>
>> On Thu, Aug 24, 2017 at 2:28 PM, Dave Page  wrote:
>>
>>> Anyone object to doing a release on 14th September, wrapping the code on
>>> Monday 11th? This seems like the best option for our QA folks who will be
>>> off for EID somewhen in the two weeks before.
>>>
>>> Assuming not, should this be 1.7 or 2.0?
>>>
>>> If we go with 2.0, it'll be for "safety" given the proposed changes to
>>> path management to allow both server and desktop modes to work out of the
>>> box on Linux.
>>>
>>> If we do that, we also need to ensure that any changes to the config
>>> database are backwards compatible, as a 2.0 release would be a side-by-side
>>> installation. Surinder; was it you that had looked into that?
>>>
>> ​I had looked into this and here are my findings:
>> 1. If we are using newer version of pgAdmin and the go back to older
>> version of pgAdmin, then on running `python pgAdmin4.py`. the
>> flask-migrate(Alembic) try to perform downgrade by one step only(ie. it can
>> switch back to one migration only when we run  `python pgAdmin4.py`). But
>> we have multiple database revisions to be migrated. So migration fails here.
>>
>> 2. When Alebmic downgrade is performed by one step, it looks for
>> downgrade function in that specific database revision, but in our code we
>> didn't written downgrade function. But if we have written downgrade
>> statement, still there is an issue:
>> ie. If we add a new column to a table xyz using ALTER statement like:
>>
>> ​```
>> ​
>> def upgrade():
>> ​​
>> verison = get_version()
>>
>> ​​
>> db.engine.execute(
>> ​​
>> 'ALTER TABLE server ADD COLUMN hostaddr TEXT(1024)'
>> ​ ​
>> )
>>
>> def downgrade():
>> ​​
>> pass
>> ​```​
>> then on downgrade it executes `downgrade` method, so downgrade should
>> have code like
>> `ALTER TABLE server DROP COLUMN hostaddr `
>> but in sqlite DROP COLUMN statements don't work.
>> So, this is a an issue with Sqlite database. However, an alternative way
>> is also given. Here is link
>> <https://stackoverflow.com/questions/5938048/delete-column-from-sqlite-table>
>>
>>
>> Still, I didn't find any other solution on upgrading/downgrading database
>> revisions without errors.
>> It is an issue with Flask-Migrate(Alembic) plugin.
>>
>
>
> Urgh. So I guess the other option is that we version the DB filename as
> well. The downside of that is that users will want to migrate their
> settings - which may be awkward as we'll have no real way of knowing where
> they are.
>
> Thoughts?
>
> Or should we write our own custom backword migrations? For eg. dropping
 column can be achieved by creating another table excluding the columns
which we want to drop then copy data to new table and then drop old table
and rename new table to old name. And also sqlite database schema which we
have in pgAdmin4 is small so writing and maintaining custom migration won
be that hard.

-- 
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

Re: Next release

[Harshal Dhumal]

One more thing that this will only work for future pgAdmin4 versions

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, Aug 25, 2017 at 1:48 PM, Dave Page  wrote:

>
>
> On Fri, Aug 25, 2017 at 9:15 AM, Surinder Kumar <
> surinder.ku...@enterprisedb.com> wrote:
>
>> ​Hi​
>>
>> On Fri, Aug 25, 2017 at 12:21 PM, Ashesh Vashi <
>> ashesh.va...@enterprisedb.com> wrote:
>>
>>> On Fri, Aug 25, 2017 at 12:16 PM, Surinder Kumar <
>>> surinder.ku...@enterprisedb.com> wrote:
>>>
>>>> Hi
>>>> On Fri, Aug 25, 2017 at 1:03 AM, Dave Page  wrote:
>>>>
>>>>>
>>>>>
>>>>> On Thu, Aug 24, 2017 at 8:28 PM, Harshal Dhumal <
>>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Harshal Dhumal*
>>>>>> *Sr. Software Engineer*
>>>>>>
>>>>>> EnterpriseDB India: http://www.enterprisedb.com
>>>>>> The Enterprise PostgreSQL Company
>>>>>>
>>>>>> On Thu, Aug 24, 2017 at 9:44 PM, Dave Page  wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Aug 24, 2017 at 10:36 AM, Surinder Kumar <
>>>>>>> surinder.ku...@enterprisedb.com> wrote:
>>>>>>>
>>>>>>>> Hi Dave,
>>>>>>>>
>>>>>>>> On Thu, Aug 24, 2017 at 2:28 PM, Dave Page 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Anyone object to doing a release on 14th September, wrapping the
>>>>>>>>> code on Monday 11th? This seems like the best option for our QA folks 
>>>>>>>>> who
>>>>>>>>> will be off for EID somewhen in the two weeks before.
>>>>>>>>>
>>>>>>>>> Assuming not, should this be 1.7 or 2.0?
>>>>>>>>>
>>>>>>>>> If we go with 2.0, it'll be for "safety" given the proposed
>>>>>>>>> changes to path management to allow both server and desktop modes to 
>>>>>>>>> work
>>>>>>>>> out of the box on Linux.
>>>>>>>>>
>>>>>>>>> If we do that, we also need to ensure that any changes to the
>>>>>>>>> config database are backwards compatible, as a 2.0 release would be a
>>>>>>>>> side-by-side installation. Surinder; was it you that had looked into 
>>>>>>>>> that?
>>>>>>>>>
>>>>>>>> ​I had looked into this and here are my findings:
>>>>>>>> 1. If we are using newer version of pgAdmin and the go back to
>>>>>>>> older version of pgAdmin, then on running `python pgAdmin4.py`. the
>>>>>>>> flask-migrate(Alembic) try to perform downgrade by one step only(ie. 
>>>>>>>> it can
>>>>>>>> switch back to one migration only when we run  `python pgAdmin4.py`). 
>>>>>>>> But
>>>>>>>> we have multiple database revisions to be migrated. So migration fails 
>>>>>>>> here.
>>>>>>>>
>>>>>>>> 2. When Alebmic downgrade is performed by one step, it looks for
>>>>>>>> downgrade function in that specific database revision, but in our code 
>>>>>>>> we
>>>>>>>> didn't written downgrade function. But if we have written downgrade
>>>>>>>> statement, still there is an issue:
>>>>>>>> ie. If we add a new column to a table xyz using ALTER statement
>>>>>>>> like:
>>>>>>>>
>>>>>>>> ​```
>>>>>>>> ​
>>>>>>>> def upgrade():
>>>>>>>> ​​
>>>>>>>> verison = get_version()
>>>>>>>>
>>>>>>>> ​​
>>>>>>>> db.engine.execute(
>>>>>>>> ​​
>>>>>>>> 'ALTER TABLE server ADD COLUMN hostaddr TEXT(1024)'
>>>>>>>> ​ ​
>>>>>>>

Re: fix keyerror problems

[Harshal Dhumal]

how about this?

default_path = (
config.DEFAULT_BINARY_PATHS.get(st.stype, "")
 )




On Sat, Aug 26, 2017 at 12:24 AM, Dave Cramer  wrote:

> See attached patch
>
>
> Dave Cramer
>

Re: [pgAdmin4][RM2586] Cleanup feature test

[Harshal Dhumal]

Hi,

Please find attached updated patch

On Fri, Aug 25, 2017 at 2:32 PM, Dave Page  wrote:

> Hi
>
> On Thu, Aug 24, 2017 at 11:51 AM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi,
>>
>> Last week Sarah, Matt and I had call regarding some issues with feature
>> test cases.
>> For example in auto commit disable test scenario from query tool test
>> cases by removing code line
>> *self.page.find_by_id("btn-auto-commit").click() *the test case scenario
>> was not falling.
>> This issue was due to previously on going database transaction. This
>> issue was also with test scenarios like
>> auto commit enable and auto rollback enabled. The attached patch fixes
>> this issue.
>>
>> @Sarah and Matt can you please give this patch a try to check if issues
>> we discussed last week are fixed.
>>
>
> I don't think Matt and Sarah are working on pgAdmin any more :-(
>
> I just tried this patch, and whilst the tests all passed, it spent quite
> some time typing "select * from hats" into random places in the query tool,
> such that it ended up running queries like
>
> select * select * select * from hats from hats from hats
>
> I don't remember it doing that before; I think it was a) clearing
> codemirror each time and b) entering the text much more quickly (in fact I
> just tried it, and it enters the text and executes the query so fast the
> screen basically strobes).
>
Fixed.


> Performance-wise, I got 266 seconds with the patch, and 378 without, so
> there's definitely some good improvement here, just a little funkyness with
> the query tool journey test.
>

Recently Wenlin Zhang has fixed this in this commit
<https://git.postgresql.org/gitweb/?p=pgadmin4.git;a=commitdiff;h=5dd375dd2081c126b7b3b9f0dba21976d2a45a1f>.
Here is link
<https://www.postgresql.org/message-id/flat/CAEawo3JKQFuVeSK-xw9wU4fUSY57KXU5GeXKcdCusQeHkjTykA%40mail.gmail.com#caeawo3jkqfuvesk-xw9wu4fusy57kxu5gexkcdcusqehkjt...@mail.gmail.com>
to email thread.


> Thanks.
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
diff --git a/web/pgadmin/feature_tests/connect_to_server_feature_test.py b/web/pgadmin/feature_tests/connect_to_server_feature_test.py
deleted file mode 100644
index 97e96f2..000
--- a/web/pgadmin/feature_tests/connect_to_server_feature_test.py
+++ /dev/null
@@ -1,84 +0,0 @@
-##
-#
-# pgAdmin 4 - PostgreSQL Tools
-#
-# Copyright (C) 2013 - 2017, The pgAdmin Development Team
-# This software is released under the PostgreSQL Licence
-#
-##
-
-import time
-from selenium.webdriver import ActionChains
-
-import config as app_config
-from regression.feature_utils.base_feature_test import BaseFeatureTest
-from regression.python_test_utils import test_utils
-
-
-class ConnectsToServerFeatureTest(BaseFeatureTest):
-"""
-Tests that a database connection can be created from the UI
-"""
-scenarios = [
-("Test database connection", dict())
-]
-
-def before(self):
-connection = test_utils.get_db_connection(self.server['db'],
-  self.server['username'],
-  self.server['db_password'],
-  self.server['host'],
-  self.server['port'],
-  self.server['sslmode'])
-test_utils.drop_database(connection, "acceptance_test_db")
-test_utils.create_database(self.server, "acceptance_test_db")
-test_utils.create_table(self.server, "acceptance_test_db", "test_table")
-
-def runTest(self):
-"""This function tests that a database connection can be created from
-the UI"""
-self.assertEqual(app_config.APP_NAME, self.page.driver.title)
-self.page.wait_for_spinner_to_disappear()
-
-self._connects_to_server()
-self._tables_node_expandable()
-
-def after(self):
-self.page.remove_server(self.server)
-
-connection = test_utils.get_db_connection(self.server['db'],
-  self.server['username'],
-  self.server['db_password'],
-  self.server['host

Re: pgAdmin 4 commit: Fix the feature tests failuers.

[Harshal Dhumal]

Hi Ashesh,

I ran feature test case on mac couple times however I didn't get failure
which Dave and you got.
In my case feature test cases mostly falling in xxs_debugger test case.

This was because as when we configure database sever to debug functions it
adds 21 its own functions in database and when we run this feature test, it
adds one more function (with name test_function) to actually debug it.
But due to its name (test_function) it appears at bottom in visible area of
browser and some times
selenium fails to click on that element. To overcome this I have renamed
test function name from *test_function*
to *a_test_function *so that it will appear at top position under function
collection node (patch attached).


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, Aug 25, 2017 at 9:11 PM, Ashesh Vashi  wrote:

> On Fri, Aug 25, 2017 at 8:26 PM, Dave Page  wrote:
>
>> QueryToolFeatureTest seems to be failing on both Jenkins and my test mac
>> system since this was applied. The other one below seems more intermittent.
>>
>> ==
>> ERROR: runTest (pgadmin.feature_tests.query_t
>> ool_tests.QueryToolFeatureTest)
>> Query tool feature test
>> --
>> Traceback (most recent call last):
>>   File 
>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py",
>> line 84, in runTest
>> self._query_tool_explain_cost()
>>   File 
>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py",
>> line 361, in _query_tool_explain_cost
>> canvas.find_element_by_xpath("//*[contains(string(),'Total Cost')]")
>>   File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa
>> ges/selenium/webdriver/remote/webelement.py", line 260, in
>> find_element_by_xpath
>> return self.find_element(by=By.XPATH, value=xpath)
>>   File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa
>> ges/selenium/webdriver/remote/webelement.py", line 510, in find_element
>> {"using": by, "value": value})['value']
>>   File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa
>> ges/selenium/webdriver/remote/webelement.py", line 493, in _execute
>> return self._parent.execute(command, params)
>>   File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa
>> ges/selenium/webdriver/remote/webdriver.py", line 249, in execute
>> self.error_handler.check_response(response)
>>   File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa
>> ges/selenium/webdriver/remote/errorhandler.py", line 193, in
>> check_response
>> raise exception_class(message, screen, stacktrace)
>> NoSuchElementException: Message: no such element: Unable to locate
>> element: {"method":"xpath","selector":"//*[contains(string(),'Total
>> Cost')]"}
>>
>   (Session info: chrome=60.0.3112.101)
>>   (Driver info: chromedriver=2.29.461585 
>> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac
>> OS X 10.11.6 x86_64)
>>
>>
>> ==
>> ERROR: runTest (pgadmin.feature_tests.view_da
>> ta_dml_queries.CheckForViewDataTest)
>> Validate Insert, Update operations in View/Edit data with given test data
>> --
>> Traceback (most recent call last):
>>   File 
>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
>> line 107, in runTest
>> self._verify_row_data(True)
>>   File 
>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
>> line 291, in _verify_row_data
>> result_row = self.page.find_by_xpath(xpath)
>>   File 
>> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>> line 148, in find_by_xpath
>> return self.wait_for_element(lambda driver:
>> driver.find_element_by_xpath(xpath))
>>   File 
>> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>> line 232, in wait_for_element
>> return self._wait_for("element to exist", element_if_it_exists)
>>   File 
>> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>> line 282, in _wait_for
>> "Time

Fix for RM2670

[Harshal Dhumal]

Hi,

Please find minor patch to improve datamodel validations for default
Validator if user (developer) do not implement validate function in
datamodel.
Without this default validation error messages  were suppressing if no
validation function was provided.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/browser/static/js/datamodel.js b/web/pgadmin/browser/static/js/datamodel.js
index c7c5dd6..dfc803f 100644
--- a/web/pgadmin/browser/static/js/datamodel.js
+++ b/web/pgadmin/browser/static/js/datamodel.js
@@ -358,14 +358,14 @@ function(_, S, pgAdmin, $, Backbone) {
 if (!msg) {
   msg = self.validate(_.keys(attrs));
 }
-
-/*
- * If any parent present, we will need to inform the parent - that
- * I have some issues/fixed the issue.
- *
- * If not parent found, we will raise the issue
- */
-if (_.size(self.errorModel.attributes) == 0) {
+  }
+  /*
+   * If any parent present, we will need to inform the parent - that
+   * I have some issues/fixed the issue.
+   *
+   * If not parent found, we will raise the issue
+   */
+  if (_.size(self.errorModel.attributes) == 0) {
   if (self.collection || self.handler) {
 (self.collection || self.handler).trigger(
 'pgadmin-session:model:valid', self, (self.collection || self.handler)
@@ -383,7 +383,6 @@ function(_, S, pgAdmin, $, Backbone) {
 self.trigger('pgadmin-session:invalid', msg, self);
   }
 }
-  }
 
   return res;
 }

Array representation [pgAdmin4]

[Harshal Dhumal]

Hi Dave,

Regarding RM2641 <https://redmine.postgresql.org/issues/2671> how should we
represent array data types; With curly braces (like pgAdminIII) or without
curly braces (only comma separated)?
Currently in pgAdmin4 curly braces do not appear for text and int array
data types however they do appear for real, bigint, float, etc array data
types.

Also additional efforts will required If we plan to represent like
pgAdminIII as we'll need to update/add slickgrid cell editors/formatter for
array data types.




-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: Build failed in Jenkins: pgadmin4-master-python27-feature #11

[Harshal Dhumal]

Hi Dave,

Please attached patch to fixed Querytool feature test cases (RM2681
<https://redmine.postgresql.org/issues/2681>).


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Sep 4, 2017 at 9:02 PM, pgAdmin 4 Jenkins 
wrote:

> See <https://jenkins.pgadmin.org/job/pgadmin4-master-python27-
> feature/11/display/redirect?page=changes>
>
> Changes:
>
> [Dave Page] Allow queries to be cancelled from the dashboard, and display
> additional
>
> [Ashesh Vashi] Define the utility path lable, and help message for the
> Greenplum
>
> [Ashesh Vashi] Using the proper python syntax to fetch the default binary
> path for
>
> [akshay.joshi] Fixed issue where 'backend_type' column is exist for
> database server
>
> [Dave Page] Cleanup feature tests. Fixes #2586
>
> [Dave Page] Allow pgAdmin to run with config database versions from the
> future.
>
> [Dave Page] Rename the debugger XSS test function to 'a_test_function' to
> ensure it
>
> [Dave Page] GPDB compatibility fixes:
>
> [Dave Page] Fix RE-SQL for triggers with a single arg. Fixes #2668
>
> [Dave Page] Improve datamodel validations for default Validator if user
> (developer)
>
> [Dave Page] Fix display of types in GPDB.
>
> [Ashesh Vashi] Define the proper NODE_ENV environment during running the
> webpack. Using
>
> [akshay.joshi] 1) Fixed error in alertify.pgNotifier when server
> connection is lost. 2)
>
> [Dave Page] Fix Copy so it still works after query results have been
> copied. Fixes
>
> --
> [...truncated 79.49 KB...]
>   File "<https://jenkins.pgadmin.org/job/pgadmin4-master-python27-
> feature/ws/pgadmin-venv/lib/python2.7/site-packages/
> selenium/webdriver/remote/errorhandler.py",> line 193, in check_response
> raise exception_class(message, screen, stacktrace)
> NoSuchElementException: Message: no such element: Unable to locate
> element: {"method":"xpath","selector":"//*[contains(string(),'Total
> Cost')]"}
>   (Session info: chrome=59.0.3071.115)
>   (Driver info: chromedriver=2.29.461571 
> (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5),platform=Linux
> 3.10.0-514.16.1.el7.x86_64 x86_64)
>
>
> --
> Ran 9 tests in 154.798s
>
> FAILED (errors=1)
>
> =Running the test cases for 'EDB Postgres AS 9.4'=
> runTest (pgadmin.feature_tests.copy_selected_query_results_feature_test.
> CopySelectedQueryResultsFeatureTest)
> Copy rows, column using button and keyboard shortcut ... ok
> runTest (pgadmin.feature_tests.pg_datatype_validation_test.
> PGDataypeFeatureTest)
> Test checks for PG data-types output ... ok
> runTest (pgadmin.feature_tests.query_tool_journey_test.
> QueryToolJourneyTest)
> Tests the path through the query tool ... ok
> runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest)
> Query tool feature test ...
> On demand query result...
> On demand result set on scrolling... OK.
> On demand result set on grid select all... OK.
> On demand result set on column select all... OK.
> Explain query with verbose and cost... OK.
> Explain analyze query with buffers and timing... OK.
> Auto commit disabled... OK.
> Auto commit enabled... OK.
> Auto rollback enabled... OK.
> Cancel query... OK.
> ok
> runTest (pgadmin.feature_tests.table_ddl_feature_test.TableDdlFeatureTest)
> Test table DDL generation ... ok
> runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest)
> Validate Insert, Update operations in View/Edit data with given test data
> ... ok
> runTest (pgadmin.feature_tests.xss_checks_panels_and_query_tool_
> test.CheckForXssFeatureTest)
> Test XSS check for panels and query tool ... ok
> runTest (pgadmin.feature_tests.xss_checks_pgadmin_debugger_test.
> CheckDebuggerForXssFeatureTest)
> Tests to check if Debugger is vulnerable to XSS ... ok
> runTest (pgadmin.feature_tests.xss_checks_roles_control_test.
> CheckRoleMembershipControlFeatureTest)
> Tests to check if Role membership control is vulnerable to XSS ... ok
>
> --
> Ran 9 tests in 225.902s
>
> OK
>
> =Running the test cases for 'EDB Postgres AS 9.5'=
> runTest (pgadmin.feature_tests.copy_selected_query_results_feature_test.
> CopySelectedQueryResultsFeatureTest)
> Copy rows, column using button and keyboard shortcut ... ok
> runTest (pgadmin.feature_tests.pg_datatype_validation_test.
> PGDataypeFeatu

Array representation and save array data from datagrid [RM2671][pgAdmin4]

[Harshal Dhumal]

Hi,

Please find attached patch to represent array data in datagrid with curly
braces and
also allow user to save array with values like  , , '
*""*' and "*''*"

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/feature_tests/test_data.json b/web/pgadmin/feature_tests/test_data.json
index a2457c2..ac00b0b 100644
--- a/web/pgadmin/feature_tests/test_data.json
+++ b/web/pgadmin/feature_tests/test_data.json
@@ -7,12 +7,19 @@
   "4": ["", "Hello World", "text"],
   "5": ["\"\"", "", "text", "Two double quotes"],
   "6": ["\\\"\\\"", "\"\"", "text", "double backslash followed by a double quote"],
-  "7": ["\"\"", "\\\"\\\"", "text", "double backslash followed by a double quote"],
+  "7": ["\"\"", "\"\"", "text", "double backslash followed by a double quote"],
   "8": ["", "[null]", "text"],
   "9": ["", "[51,52]", "json"],
   "10": ["[61,62]", "[61,62]", "json"],
   "11": ["", "true", "bool"],
-  "12": ["", "[null]", "bool"]
+  "12": ["", "[null]", "bool"],
+  "13": ["", "[null]", "text[]"],
+  "14": ["{}", "{}", "text[]"],
+  "15": ["{data,,'',\"\",\\'\\',\\\"\\\"}", "{data,[null],,,'',\"\"}", "text[]"],
+  "16": ["{}", "{}", "int[]"],
+  "17": ["{123,,456}", "{123,[null],456}", "int[]"],
+  "18": ["", "[null]", "boolean[]"],
+  "19": ["{false,,true}", "{false,[null],true}", "boolean[]"]
 }
   }
-}
\ No newline at end of file
+}
diff --git a/web/pgadmin/feature_tests/view_data_dml_queries.py b/web/pgadmin/feature_tests/view_data_dml_queries.py
index 0b2da0c..153d796 100644
--- a/web/pgadmin/feature_tests/view_data_dml_queries.py
+++ b/web/pgadmin/feature_tests/view_data_dml_queries.py
@@ -67,6 +67,13 @@ CREATE TABLE public.defaults
 json_null json,
 boolean_defaults boolean DEFAULT true,
 boolean_null boolean,
+text_arr text[],
+text_arr_empty text[],
+text_arr_null text[],
+int_arr integer[],
+int_arr_empty integer[],
+boolean_arr boolean[],
+boolean_arr_null boolean[],
 CONSTRAINT defaults_pkey PRIMARY KEY (id)
 )
 """
@@ -169,17 +176,16 @@ CREATE TABLE public.defaults
 ActionChains(self.driver).move_to_element(cell_el).double_click(
 cell_el
 ).perform()
-
 cell_type = data[2]
 value = data[0]
 
-if cell_type == 'int':
+if cell_type in ['int', 'int[]']:
 if value == 'clear':
 cell_el.find_element_by_css_selector('input').clear()
 else:
 ActionChains(self.driver).send_keys(value).perform()
 
-elif cell_type in ['text', 'json']:
+elif cell_type in ['text', 'json', 'text[]', 'boolean[]']:
 self.page.find_by_xpath(
 "//*[contains(@class, 'pg_textarea')]").click()
 ActionChains(self.driver).send_keys(value).perform()
@@ -231,7 +237,6 @@ CREATE TABLE public.defaults
 self.page.find_by_xpath(row0_cell0_xpath).click()
 self.page.find_by_xpath("//*[@id='btn-copy-row']").click()
 self.page.find_by_xpath("//*[@id='btn-paste-row']").click()
-
 # Update primary key of copied cell
 self._update_cell(row1_cell1_xpath, [2, "", "int"])
 self.page.find_by_xpath(
@@ -261,13 +266,12 @@ CREATE TABLE public.defaults
 self._verify_row_data(False)
 
 def _add_row(self):
-for idx in range(1, len(config_data.keys())):
+for idx in range(1, len(config_data.keys()) + 1):
 cell_xpath = CheckForViewDataTest._get_cell_xpath(
 'r'+str(idx), 1
 )
 time.sleep(0.2)
 self._update_cell(cell_xpath, config_data[str(idx)])
-
 self.page.find_by_id("btn-save")

Re: Array representation and save array data from datagrid [RM2671][pgAdmin4]

[Harshal Dhumal]

Thanks, I'll look into it.

On Sep 14, 2017 5:31 PM, "Akshay Joshi" 
wrote:

> Hi Harshal
>
> I have reviewed your patch it looks good to me. Only one issue I have
> found with char[], when I store {'a','b','c'} it gets stored and when I
> refreshed it shows {',','}. Please look into this issue.
>
> On Thu, Sep 14, 2017 at 3:54 PM, Akshay Joshi <
> akshay.jo...@enterprisedb.com> wrote:
>
>> It works, sorry for the noise.
>>
>> On Thu, Sep 14, 2017 at 3:48 PM, Akshay Joshi <
>> akshay.jo...@enterprisedb.com> wrote:
>>
>>> Harshal,
>>>
>>> Can you please rebase the patch, it's not applied with the latest code.
>>>
>>> On Tue, Sep 12, 2017 at 2:37 PM, Akshay Joshi <
>>> akshay.jo...@enterprisedb.com> wrote:
>>>
>>>>
>>>>
>>>> On Tue, Sep 12, 2017 at 2:34 PM, Dave Page  wrote:
>>>>
>>>>> Adding Akshay...
>>>>>
>>>>
>>>> Sure.
>>>>
>>>>>
>>>>> On Tue, Sep 12, 2017 at 10:04 AM, Dave Page  wrote:
>>>>>
>>>>>> Akshay, can you review/commit this please?
>>>>>>
>>>>>> On Mon, Sep 11, 2017 at 3:40 PM, Harshal Dhumal <
>>>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Please find attached patch to represent array data in datagrid with
>>>>>>> curly braces and
>>>>>>> also allow user to save array with values like  , >>>>>> string>, '*""*' and "*''*"
>>>>>>>
>>>>>>> --
>>>>>>> *Harshal Dhumal*
>>>>>>> *Sr. Software Engineer*
>>>>>>>
>>>>>>> EnterpriseDB India: http://www.enterprisedb.com
>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Dave Page
>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>> Twitter: @pgsnake
>>>>>>
>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>> The Enterprise PostgreSQL Company
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Dave Page
>>>>> Blog: http://pgsnake.blogspot.com
>>>>> Twitter: @pgsnake
>>>>>
>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>> The Enterprise PostgreSQL Company
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Akshay Joshi*
>>>> *Principal Software Engineer *
>>>>
>>>>
>>>>
>>>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246*
>>>>
>>>
>>>
>>>
>>> --
>>> *Akshay Joshi*
>>> *Principal Software Engineer *
>>>
>>>
>>>
>>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246*
>>>
>>
>>
>>
>> --
>> *Akshay Joshi*
>> *Principal Software Engineer *
>>
>>
>>
>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246*
>>
>
>
>
> --
> *Akshay Joshi*
> *Principal Software Engineer *
>
>
>
> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246*
>

Re: Array representation and save array data from datagrid [RM2671][pgAdmin4]

[Harshal Dhumal]

Hi Akshay,

It's not issue; Try without quotes. As char data type stores only one
character so it's storing only leading quote (') and
truncating remaining characters (a').
Only exception is when you want to store empty string (char) use two quotes
(either single or double)
they will translated before sending to server.

Apart from this I found that data types "char" and "char"[] (alias to
character and character[] respectively)
were not taken into consideration in sqleditor.js. I have attached updated
patch which fixes this issue.





-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Thu, Sep 14, 2017 at 5:31 PM, Akshay Joshi  wrote:

> Hi Harshal
>
> I have reviewed your patch it looks good to me. Only one issue I have
> found with char[], when I store {'a','b','c'} it gets stored and when I
> refreshed it shows {',','}. Please look into this issue.
>
> On Thu, Sep 14, 2017 at 3:54 PM, Akshay Joshi <
> akshay.jo...@enterprisedb.com> wrote:
>
>> It works, sorry for the noise.
>>
>> On Thu, Sep 14, 2017 at 3:48 PM, Akshay Joshi <
>> akshay.jo...@enterprisedb.com> wrote:
>>
>>> Harshal,
>>>
>>> Can you please rebase the patch, it's not applied with the latest code.
>>>
>>> On Tue, Sep 12, 2017 at 2:37 PM, Akshay Joshi <
>>> akshay.jo...@enterprisedb.com> wrote:
>>>
>>>>
>>>>
>>>> On Tue, Sep 12, 2017 at 2:34 PM, Dave Page  wrote:
>>>>
>>>>> Adding Akshay...
>>>>>
>>>>
>>>> Sure.
>>>>
>>>>>
>>>>> On Tue, Sep 12, 2017 at 10:04 AM, Dave Page  wrote:
>>>>>
>>>>>> Akshay, can you review/commit this please?
>>>>>>
>>>>>> On Mon, Sep 11, 2017 at 3:40 PM, Harshal Dhumal <
>>>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Please find attached patch to represent array data in datagrid with
>>>>>>> curly braces and
>>>>>>> also allow user to save array with values like  , >>>>>> string>, '*""*' and "*''*"
>>>>>>>
>>>>>>> --
>>>>>>> *Harshal Dhumal*
>>>>>>> *Sr. Software Engineer*
>>>>>>>
>>>>>>> EnterpriseDB India: http://www.enterprisedb.com
>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Dave Page
>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>> Twitter: @pgsnake
>>>>>>
>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>> The Enterprise PostgreSQL Company
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Dave Page
>>>>> Blog: http://pgsnake.blogspot.com
>>>>> Twitter: @pgsnake
>>>>>
>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>> The Enterprise PostgreSQL Company
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Akshay Joshi*
>>>> *Principal Software Engineer *
>>>>
>>>>
>>>>
>>>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246*
>>>>
>>>
>>>
>>>
>>> --
>>> *Akshay Joshi*
>>> *Principal Software Engineer *
>>>
>>>
>>>
>>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246*
>>>
>>
>>
>>
>> --
>> *Akshay Joshi*
>> *Principal Software Engineer *
>>
>>
>>
>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246*
>>
>
>
>
> --
> *Akshay Joshi*
> *Principal Software Engineer *
>
>
>
> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246*
>
diff --git a/web/pgadmin/feature_tests/test_data.json b/web/pgadmin/feature_tests/test_data.json
index a2457c2..ac00b0b 100644
--- a/web/pgadmin/feature_tests/test_data.json
+++ b/web/pgadmin/feature_tests/test_data.json
@@ -7,12 +7,19 @@
   "4": ["", "Hello World", "text"],
   "5": ["\"\"", &

Re: [pgAdmin4][Patch]: Properly handle builtins.SERVER_MODE variable when SERVER_MODE is not set

[Harshal Dhumal]

I think this is similar issue..
https://redmine.postgresql.org/issues/2134

On Mon, Sep 18, 2017 at 11:16 AM, Surinder Kumar <
surinder.ku...@enterprisedb.com> wrote:

> Hi
>
> While configuring pgAdmin4 with Apache mod_wsgi, it fails to run apache
> server due to python error.
>
> [mod_wsgi (pid=12596): Target WSGI script 
> '/opt/pgadmin4_2/virtualenv/lib/python2.7/site-packages/pgadmin4/pgAdmin4.wsgi'
>  cannot be loaded as Python module.
> mod_wsgi (pid=12596): Exception occurred processing WSGI script 
> '/opt/pgadmin4_2/virtualenv/lib/python2.7/site-packages/pgadmin4/pgAdmin4.wsgi'.
>  Traceback (most recent call last):
>File 
> "/opt/pgadmin4_2/virtualenv/lib/python2.7/site-packages/pgadmin4/pgAdmin4.wsgi",
>  line 20, in 
>  import config
>File 
> "/opt/pgadmin4_2/virtualenv/lib/python2.7/site-packages/pgadmin4/config.py", 
> line 118, in 
>  if builtins.SERVER_MODE is None:
>  AttributeError: 'module' object has no attribute 'SERVER_MODE'
>
> ​Reported by: ​
> Gabriel Sánchez
>
> Please find an attached patch.
>
> Thanks,
> Surinder
> ​
>

Fix for Exception when the server is restarted [RM2713][pgAdmin4]

[Harshal Dhumal]

Hi,

This issue was caused because we recently added session_write_delay in
session.
So session won't be written/updated to disk from memory until specified
seconds are elapsed.

However we must forcefully write/update session to disk if user loges in or
out irrespective of session_write_delay to keep sessions from memory and
disk in sync as user logged in status is
kept in session.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/__init__.py b/web/pgadmin/__init__.py
index b204967..61a9beb 100644
--- a/web/pgadmin/__init__.py
+++ b/web/pgadmin/__init__.py
@@ -17,7 +17,7 @@ from importlib import import_module
 from flask import Flask, abort, request, current_app, session, url_for
 from flask_babel import Babel, gettext
 from flask_htmlmin import HTMLMIN
-from flask_login import user_logged_in
+from flask_login import user_logged_in, user_logged_out
 from flask_security import Security, SQLAlchemyUserDatastore
 from flask_mail import Mail
 from flask_security.utils import login_user
@@ -368,7 +368,6 @@ def create_app(app_name=None):
 ##
 @user_logged_in.connect_via(app)
 def on_user_logged_in(sender, user):
-
 # Keep hold of the user ID
 user_id = user.id
 
@@ -497,6 +496,10 @@ def create_app(app_name=None):
 except:
 pass
 
+@user_logged_in.connect_via(app)
+@user_logged_out.connect_via(app)
+def force_session_write(app, user):
+session.force_write = True
 
 ##
 # Load plugin modules
diff --git a/web/pgadmin/utils/session.py b/web/pgadmin/utils/session.py
index e4ff765..8f2ae61 100644
--- a/web/pgadmin/utils/session.py
+++ b/web/pgadmin/utils/session.py
@@ -58,6 +58,7 @@ class ManagedSession(CallbackDict, SessionMixin):
 self.modified = False
 self.randval = randval
 self.last_write = None
+self.force_write = False
 self.hmac_digest = hmac_digest
 
 def sign(self, secret):
@@ -219,12 +220,13 @@ class FileBackedSessionManager(SessionManager):
 current_time = time.time()
 if not session.hmac_digest:
 session.sign(self.secret)
-else:
+elif not session.force_write:
 if session.last_write is not None \
 and (current_time - float(session.last_write)) < self.disk_write_delay:
 return
 
 session.last_write = current_time
+session.force_write = False
 fname = os.path.join(self.path, session.sid)
 with open(fname, 'wb') as f:
 dump(

Re: pgAdmin 4 commit: Update release notes.

[Harshal Dhumal]

Hi Dave,

I think RM2671 is missing in release notes.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Thu, Sep 28, 2017 at 2:51 PM, Dave Page  wrote:

> Update release notes.
>
> Branch
> --
> master
>
> Details
> ---
> https://git.postgresql.org/gitweb?p=pgadmin4.git;a=commitdiff;h=
> 50a78abf48795173c2e4cbd5f49157339db92bf5
>
> Modified Files
> --
> docs/en_US/release_notes_2_0.rst | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
>

Re: pgadmin4 exception: session['_id'] not in self.managers

[Harshal Dhumal]

Hi Dave,

Please find attached patch (old RM2713).
Also for server group and server module login required checks were missing,
I have added them in same patch

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Oct 10, 2017 at 11:39 AM, Harshal Dhumal <
harshal.dhu...@enterprisedb.com> wrote:

> Hi David,
>
> Thanks for you input. Session was not invalidated (otherwise execution
> would not have reached to connection manager); Only value of '_id' was
> changed for session.
>
> If we look at code
> <https://github.com/maxcountryman/flask-login/blob/master/flask_login/utils.py#L333>
> how '_id' is generated then we can see it uses remote address and user-agent
> to generate it. I thing we should use another session identifier (sid -
> session id) to map user connection from connection manager.
>
> --
> *Harshal Dhumal*
> *Sr. Software Engineer*
>
> EnterpriseDB India: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
> On Mon, Oct 9, 2017 at 10:18 PM, David Gilman 
> wrote:
>
>> You can probably stand down on this one. The issue was that my PC was
>> switching back and forth between IPv4 and IPv6 for whatever reason and when
>> that happened the cookie and my session would get invalidated. Maybe it is
>> worth making code changes so you don't get Python tracebacks dumped to the
>> error logs every time this happens but I have been able to resolve the
>> issue by turning off IPv6 temporarily. That was an interesting one to
>> troubleshoot!
>>
>> On Mon, Oct 9, 2017 at 4:30 AM, Harshal Dhumal <
>> harshal.dhu...@enterprisedb.com> wrote:
>>
>>> sure Dave
>>>
>>> --
>>> *Harshal Dhumal*
>>> *Sr. Software Engineer*
>>>
>>> EnterpriseDB India: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>>> On Mon, Oct 9, 2017 at 1:16 PM, Dave Page  wrote:
>>>
>>>> Harshal, can you help with this please?
>>>>
>>>> On Sun, Oct 8, 2017 at 12:39 AM, David Gilman 
>>>> wrote:
>>>>
>>>>> I'm trying out pgadmin4 v2.0 for the first time.  It seems that after
>>>>> only a few minutes (maybe even less than five) my pgadmin4 session will 
>>>>> get
>>>>> logged out and I'll need to log in again and reopen everything from
>>>>> scratch.  This exception is thrown in the mod_wsgi logs:
>>>>>
>>>>> mod_wsgi (pid=5965): Exception occurred processing WSGI script
>>>>> '/home/pgadmin/venv/lib/python2.7/site-packages/pgadmin4/pgA
>>>>> dmin4.wsgi'.
>>>>> Traceback (most recent call last):
>>>>>   File 
>>>>> "/home/pgadmin/venv/local/lib/python2.7/site-packages/flask/app.py",
>>>>> line 2000, in __call__
>>>>> return self.wsgi_app(environ, start_response)
>>>>>   File 
>>>>> "/home/pgadmin/venv/local/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1991, in wsgi_app
>>>>> response = self.make_response(self.handle_exception(e))
>>>>>   File 
>>>>> "/home/pgadmin/venv/local/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1567, in handle_exception
>>>>> reraise(exc_type, exc_value, tb)
>>>>>   File 
>>>>> "/home/pgadmin/venv/local/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1988, in wsgi_app
>>>>> response = self.full_dispatch_request()
>>>>>   File 
>>>>> "/home/pgadmin/venv/local/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1641, in full_dispatch_request
>>>>> rv = self.handle_user_exception(e)
>>>>>   File 
>>>>> "/home/pgadmin/venv/local/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1544, in handle_user_exception
>>>>> reraise(exc_type, exc_value, tb)
>>>>>   File 
>>>>> "/home/pgadmin/venv/local/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1639, in full_dispatch_request
>>>>> rv = self.dispatch_request()
>>>>>   File 
>>>>> "/home/pgadmin/venv/local/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1625, in dispatch_request
>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>

Re: [pgAdmin4][Patch]: To handle long numbers for Rows (estimated) field in Table's properties

[Harshal Dhumal]

Hi Murtuza,

I think for tables having row count more than preference count it should
show preference count with plus '+' sign instead empty.
eg. 2000+ (for table with rows more than 2000 and for preference count of
2000.)


[image: Inline image 1]



-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Wed, Nov 1, 2017 at 7:38 PM, Murtuza Zabuawala <
murtuza.zabuaw...@enterprisedb.com> wrote:

> Hi Dave,
>
> Please find updated patch.
>
> ​-- Murtuza​
>
>
> On Wed, Nov 1, 2017 at 6:50 PM, Murtuza Zabuawala  enterprisedb.com> wrote:
>
>> Thanks Dave, I'll check the implementation in pgAdmin3 & send new patch
>> with suggested changes.
>>
>>
>> On Wed, Nov 1, 2017 at 6:41 PM, Dave Page  wrote:
>>
>>> Hi
>>>
>>> On Wed, Nov 1, 2017 at 11:58 AM, Murtuza Zabuawala <
>>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> PFA patch to fix the issue where "Rows (estimated)" field was not
>>>> displaying long number properly.
>>>> RM#2386
>>>>
>>>> I have also added the code to count actual rows in a table [ "Rows
>>>> (counted)" filed ] which was missing.
>>>>
>>>
>>> Oops. Well the idea is good, but unfortunately we want to be careful
>>> about when we count the rows, as we could have just clicked on a 10B row
>>> table which would take a while. To avoid that, pgAdmin III would only count
>>> the rows in the table if the estimated number of rows was below a specific
>>> value (see File -> Preferences -> Browser -> Properties -> "Count rows if
>>> estimated less than").
>>>
>>> I think we need to do the same here - add a preferences option, and only
>>> count if estimated is greater than the specified value (the default for
>>> which was 2000 in pgAdmin III I believe).
>>>
>>> --
>>> Dave Page
>>> Blog: http://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EnterpriseDB UK: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>>
>>
>

Fix for minor issue RM2868

[Harshal Dhumal]

Hi,

Path to fix minor issue "local variable 'password' referenced before
assignment."


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/browser/server_groups/servers/__init__.py b/web/pgadmin/browser/server_groups/servers/__init__.py
index 6d972ee..25ec28a 100644
--- a/web/pgadmin/browser/server_groups/servers/__init__.py
+++ b/web/pgadmin/browser/server_groups/servers/__init__.py
@@ -694,21 +694,18 @@ class ServerNode(PGChildNodeView):
 manager.update(server)
 conn = manager.connection()
 
-have_password =  False
+have_password = False
+password = None
+passfile = None
 if 'password' in data and data["password"] != '':
 # login with password
 have_password = True
-passfile = None
 password = data['password']
 password = encrypt(password, current_user.password)
 elif 'passfile' in data and data["passfile"] != '':
 passfile = data['passfile']
 setattr(server, 'passfile', passfile)
 db.session.commit()
-else:
-# Attempt password less login
-password = None
-passfile = None
 
 status, errmsg = conn.connect(
 password=password,

Fix for issue RM2760

[Harshal Dhumal]

Hi,

Please find attached patch to fix RM2760 and RM2867



-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/misc/file_manager/static/js/file_manager.js b/web/pgadmin/misc/file_manager/static/js/file_manager.js
index a88ed67..fefa1b0 100644
--- a/web/pgadmin/misc/file_manager/static/js/file_manager.js
+++ b/web/pgadmin/misc/file_manager/static/js/file_manager.js
@@ -159,6 +159,7 @@ define('misc.file_manager', [
 $(innerbody).find('*').off();
 innerbody.remove();
 removeTransId(trans_id);
+pgAdmin.Browser.Events.trigger('pgadmin-storage:cancel_btn:storage_dialog');
   }
 },
 build: function() {
@@ -285,6 +286,7 @@ define('misc.file_manager', [
 $(innerbody).find('*').off();
 innerbody.remove();
 removeTransId(trans_id);
+pgAdmin.Browser.Events.trigger('pgadmin-storage:cancel_btn:select_file');
   }
 },
 build: function() {
@@ -409,6 +411,7 @@ define('misc.file_manager', [
 $(innerbody).find('*').off();
 innerbody.remove();
 removeTransId(trans_id);
+pgAdmin.Browser.Events.trigger('pgadmin-storage:cancel_btn:select_folder');
   }
 },
 build: function() {
@@ -632,6 +635,7 @@ define('misc.file_manager', [
 $(innerbody).find('*').off();
 innerbody.remove();
 removeTransId(trans_id);
+pgAdmin.Browser.Events.trigger('pgadmin-storage:cancel_btn:create_file');
   }
 },
 build: function() {
diff --git a/web/pgadmin/static/js/backform.pgadmin.js b/web/pgadmin/static/js/backform.pgadmin.js
index acde87b..7d446fe 100644
--- a/web/pgadmin/static/js/backform.pgadmin.js
+++ b/web/pgadmin/static/js/backform.pgadmin.js
@@ -2163,9 +2163,6 @@
 },
 initialize: function(){
   Backform.InputControl.prototype.initialize.apply(this, arguments);
-
-  // Listen click events of Storage Manager dialog buttons
-  pgAdmin.Browser.Events.on('pgadmin-storage:finish_btn:'+this.field.get('dialog_type'), this.storage_dlg_hander, this);
 },
 template: _.template([
   '<%=label%>',
@@ -2199,15 +2196,30 @@
 
   pgAdmin.FileManager.init();
   pgAdmin.FileManager.show_dialog(params);
+  // Listen click events of Storage Manager dialog buttons
+  this.listen_file_dlg_events();
 },
 storage_dlg_hander: function(value) {
   var field = _.defaults(this.field.toJSON(), this.defaults),
   attrArr = this.field.get("name").split('.'),
   name = attrArr.shift();
 
+  this.remove_file_dlg_event_listeners();
+
   // Set selected value into the model
   this.model.set(name, decodeURI(value));
 },
+storage_close_dlg_hander: function() {
+  this.remove_file_dlg_event_listeners();
+},
+listen_file_dlg_events: function() {
+  pgAdmin.Browser.Events.on('pgadmin-storage:finish_btn:'+this.field.get('dialog_type'), this.storage_dlg_hander, this);
+  pgAdmin.Browser.Events.on('pgadmin-storage:cancel_btn:'+this.field.get('dialog_type'), this.storage_close_dlg_hander, this);
+},
+remove_file_dlg_event_listeners: function() {
+  pgAdmin.Browser.Events.off('pgadmin-storage:finish_btn:'+this.field.get('dialog_type'), this.storage_dlg_hander, this);
+  pgAdmin.Browser.Events.off('pgadmin-storage:cancel_btn:'+this.field.get('dialog_type'), this.storage_close_dlg_hander, this);
+},
 clearInvalid: function() {
   Backform.InputControl.prototype.clearInvalid.apply(this, arguments);
   this.$el.removeClass("pgadmin-file-has-error");

Fix for RM2845: Make "Save changes" prompt configurable

[Harshal Dhumal]

Hi,

Please find attached patch to fix RM2845

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/tools/datagrid/__init__.py b/web/pgadmin/tools/datagrid/__init__.py
index c37e5d3..eaa1850 100644
--- a/web/pgadmin/tools/datagrid/__init__.py
+++ b/web/pgadmin/tools/datagrid/__init__.py
@@ -216,6 +216,11 @@ def panel(trans_id, is_query_tool, editor_title):
 else:
 new_browser_tab = 'false'
 
+if is_query_tool == 'true':
+prompt_save_changes = pref.preference('prompt_save_query_changes').get()
+else:
+prompt_save_changes = pref.preference('prompt_save_data_changes').get()
+
 # Fetch the server details
 #
 bgcolor = None
@@ -243,7 +248,10 @@ def panel(trans_id, is_query_tool, editor_title):
 server_type=server_type,
 client_platform=user_agent.platform,
 bgcolor=bgcolor,
-fgcolor=fgcolor
+fgcolor=fgcolor,
+# convert python boolean value to equivalent js boolean literal before
+# passing it to html template.
+prompt_save_changes='true' if prompt_save_changes else 'false'
 )
 
 
diff --git a/web/pgadmin/tools/datagrid/templates/datagrid/index.html b/web/pgadmin/tools/datagrid/templates/datagrid/index.html
index 05e0986..ff4368d 100644
--- a/web/pgadmin/tools/datagrid/templates/datagrid/index.html
+++ b/web/pgadmin/tools/datagrid/templates/datagrid/index.html
@@ -369,6 +369,6 @@
 
 // Start the query tool.
 sqlEditorController.start({{ is_query_tool }}, "{{ editor_title }}",
-script_sql, {{ is_new_browser_tab }}, "{{ server_type }}");
+script_sql, {{ is_new_browser_tab }}, "{{ server_type }}", {{ prompt_save_changes }});
 });
 {% endblock %}
diff --git a/web/pgadmin/tools/sqleditor/__init__.py b/web/pgadmin/tools/sqleditor/__init__.py
index d360d91..a4b83d4 100644
--- a/web/pgadmin/tools/sqleditor/__init__.py
+++ b/web/pgadmin/tools/sqleditor/__init__.py
@@ -235,6 +235,26 @@ class SqlEditorModule(PgAdminModule):
 )
 )
 
+self.show_prompt_save_query_changes = self.preference.register(
+'Options', 'prompt_save_query_changes',
+gettext("Prompt to save unsaved query changes?"), 'boolean', True,
+category_label=gettext('Options'),
+help_str=gettext(
+'Specifies whether or not to prompt user to save unsaved '
+'query on query tool exit.'
+)
+)
+
+self.show_prompt_save_data_changes = self.preference.register(
+'Options', 'prompt_save_data_changes',
+gettext("Prompt to save unsaved data changes?"), 'boolean', True,
+category_label=gettext('Options'),
+help_str=gettext(
+'Specifies whether or not to prompt user to save unsaved '
+'data on data grid exit.'
+)
+)
+
 self.csv_quoting = self.preference.register(
 'CSV_output', 'csv_quoting',
 gettext("CSV quoting"), 'options', 'strings',
diff --git a/web/pgadmin/tools/sqleditor/static/js/sqleditor.js b/web/pgadmin/tools/sqleditor/static/js/sqleditor.js
index ba0ce67..68f953a 100644
--- a/web/pgadmin/tools/sqleditor/static/js/sqleditor.js
+++ b/web/pgadmin/tools/sqleditor/static/js/sqleditor.js
@@ -243,25 +243,28 @@ define('tools.querytool', [
 // Listen on the panel closed event and notify user to save modifications.
 _.each(window.top.pgAdmin.Browser.docker.findPanels('frm_datagrid'), function (p) {
   if (p.isVisible()) {
-p.on(wcDocker.EVENT.CLOSING, function () {
-  // Only if we can edit data then perform this check
-  var notify = false, msg;
-  if (self.handler.can_edit) {
-var data_store = self.handler.data_store;
-if (data_store && (_.size(data_store.added) ||
-  _.size(data_store.updated))) {
-  msg = gettext("The data has changed. Do you want to save changes?");
+if(self.handler.prompt_save_changes) {
+  p.on(wcDocker.EVENT.CLOSING, function () {
+// Only if we can edit data then perform this check
+var notify = false, msg;
+if (self.handler.can_edit) {
+  var data_store = self.handler.data_store;
+  if (data_store && (_.size(data_store.added) ||
+_.size(data_store.updated))) {
+msg = gettext("The data has changed. Do you want to

Fix for RM2811

[Harshal Dhumal]

Hi,

Please find attached patch to fix RM2811

Issue was caused due to assumption made when current position in log file
while reading it reaches to last line then
we were assuming process is finished. However this is not the case.
Background process may be busy performing
some other task and logs might not be logged to file immediately. So we
should also check process exit code along
with above condition.

Apart from above this patch also includes minor fix related to status text
colour.

--
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/misc/bgprocess/processes.py b/web/pgadmin/misc/bgprocess/processes.py
index 0b94775..209c2a1 100644
--- a/web/pgadmin/misc/bgprocess/processes.py
+++ b/web/pgadmin/misc/bgprocess/processes.py
@@ -365,7 +365,7 @@ class BatchProcess(object):
 if enc is None or enc == 'ascii':
 enc = 'utf-8'
 
-def read_log(logfile, log, pos, ctime):
+def read_log(logfile, log, pos, ctime, ecode=None):
 completed = True
 idx = 0
 c = re.compile(r"(\d+),(.*$)")
@@ -376,6 +376,9 @@ class BatchProcess(object):
 with open(logfile, 'rb') as f:
 eofs = os.fstat(f.fileno()).st_size
 f.seek(pos, 0)
+if pos == eofs and ecode is None:
+completed = False
+
 while pos < eofs:
 idx += 1
 line = f.readline()
@@ -394,15 +397,12 @@ class BatchProcess(object):
 completed = False
 break
 if pos == eofs:
-completed = True
+if ecode is None:
+completed = False
 break
 
 return pos, completed
 
-if process_output:
-out, out_completed = read_log(self.stdout, stdout, out, ctime)
-err, err_completed = read_log(self.stderr, stderr, err, ctime)
-
 j = Process.query.filter_by(
 pid=self.id, user_id=current_user.id
 ).first()
@@ -423,11 +423,11 @@ class BatchProcess(object):
 
 execution_time = (etime - stime).total_seconds()
 
-if process_output and self.ecode is not None and (
-len(stdout) + len(stderr) < 1024
-):
-out, out_completed = read_log(self.stdout, stdout, out, ctime)
-err, err_completed = read_log(self.stderr, stderr, err, ctime)
+if process_output:
+out, out_completed = read_log(self.stdout, stdout, out, ctime,
+  self.ecode)
+err, err_completed = read_log(self.stderr, stderr, err, ctime,
+  self.ecode)
 else:
 out_completed = err_completed = False
 
diff --git a/web/pgadmin/misc/bgprocess/static/js/bgprocess.js b/web/pgadmin/misc/bgprocess/static/js/bgprocess.js
index a4506b5..bc61733 100644
--- a/web/pgadmin/misc/bgprocess/static/js/bgprocess.js
+++ b/web/pgadmin/misc/bgprocess/static/js/bgprocess.js
@@ -301,9 +301,16 @@ define('misc.bgprocess', [
   ).append(
 $('').text(' ' + gettext('seconds'))
   );
-  self.container.find('.pg-bg-status').empty().append(
+  var $status_bar = $(self.container.find('.pg-bg-status'));
+  $status_bar.empty().append(
 self.curr_status
   );
+
+  if (self.exit_code === 0) {
+$status_bar.addClass('bg-success');
+  } else if (self.exit_code == 1){
+$status_bar.addClass('bg-failed');
+  }
 } else {
   self.show_detailed_view.apply(self)
 }

Re: Fix for RM2811

[Harshal Dhumal]

Hi Dave,

Please find updated patch.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Nov 27, 2017 at 5:28 PM, Dave Page  wrote:

> Hi
>
> On Mon, Nov 27, 2017 at 11:23 AM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi,
>>
>> Please find attached patch to fix RM2811
>>
>> Issue was caused due to assumption made when current position in log file
>> while reading it reaches to last line then
>> we were assuming process is finished. However this is not the case.
>> Background process may be busy performing
>> some other task and logs might not be logged to file immediately. So we
>> should also check process exit code along
>> with above condition.
>>
>> Apart from above this patch also includes minor fix related to status
>> text colour.
>>
>
> Seems to work nicely, except that the text scrolls out of view almost
> immediately. Can you fix it to jump to the end of the text when more is
> appended please?
>
> Fixed.


> Thanks.
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
diff --git a/web/pgadmin/misc/bgprocess/processes.py b/web/pgadmin/misc/bgprocess/processes.py
index 0b94775..209c2a1 100644
--- a/web/pgadmin/misc/bgprocess/processes.py
+++ b/web/pgadmin/misc/bgprocess/processes.py
@@ -365,7 +365,7 @@ class BatchProcess(object):
 if enc is None or enc == 'ascii':
 enc = 'utf-8'
 
-def read_log(logfile, log, pos, ctime):
+def read_log(logfile, log, pos, ctime, ecode=None):
 completed = True
 idx = 0
 c = re.compile(r"(\d+),(.*$)")
@@ -376,6 +376,9 @@ class BatchProcess(object):
 with open(logfile, 'rb') as f:
 eofs = os.fstat(f.fileno()).st_size
 f.seek(pos, 0)
+if pos == eofs and ecode is None:
+completed = False
+
 while pos < eofs:
 idx += 1
 line = f.readline()
@@ -394,15 +397,12 @@ class BatchProcess(object):
 completed = False
 break
 if pos == eofs:
-completed = True
+if ecode is None:
+completed = False
 break
 
 return pos, completed
 
-if process_output:
-out, out_completed = read_log(self.stdout, stdout, out, ctime)
-err, err_completed = read_log(self.stderr, stderr, err, ctime)
-
 j = Process.query.filter_by(
 pid=self.id, user_id=current_user.id
 ).first()
@@ -423,11 +423,11 @@ class BatchProcess(object):
 
 execution_time = (etime - stime).total_seconds()
 
-if process_output and self.ecode is not None and (
-len(stdout) + len(stderr) < 1024
-):
-out, out_completed = read_log(self.stdout, stdout, out, ctime)
-err, err_completed = read_log(self.stderr, stderr, err, ctime)
+if process_output:
+out, out_completed = read_log(self.stdout, stdout, out, ctime,
+  self.ecode)
+err, err_completed = read_log(self.stderr, stderr, err, ctime,
+  self.ecode)
 else:
 out_completed = err_completed = False
 
diff --git a/web/pgadmin/misc/bgprocess/static/js/bgprocess.js b/web/pgadmin/misc/bgprocess/static/js/bgprocess.js
index a4506b5..be4ac15 100644
--- a/web/pgadmin/misc/bgprocess/static/js/bgprocess.js
+++ b/web/pgadmin/misc/bgprocess/static/js/bgprocess.js
@@ -159,8 +159,12 @@ define('misc.bgprocess', [
 while (ie < err.length) {
   res.push('' + escapeHTML(err[ie++][1]) + '');
 }
+
 if (res.length) {
   self.logs.append(res.join(''));
+  setTimeout(function() {
+self.logs[0].scrollTop = self.logs[0].scrollHeight;
+  });
 }
 
 if (self.stime) {
@@ -301,9 +305,16 @@ define('misc.bgprocess', [
   ).append(
 $('').text(' ' + gettext('seconds'))
   );
-  self.container.find('.pg-bg-status').empty().append(
+  var $status_bar = $(self.container.find('.pg-bg-status'));
+  $status_bar.empty().append(
 self.curr_status
   );
+
+  if (self.exit_code === 0) {
+$status_bar.addClass('bg-success');
+  } else if (

Re: Fix for RM2811

[Harshal Dhumal]

On Mon, Nov 27, 2017 at 6:32 PM, Dave Page  wrote:

> Hi
>
> On Mon, Nov 27, 2017 at 12:29 PM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi Dave,
>>
>> Please find updated patch.
>>
>> --
>> *Harshal Dhumal*
>> *Sr. Software Engineer*
>>
>> EnterpriseDB India: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>> On Mon, Nov 27, 2017 at 5:28 PM, Dave Page  wrote:
>>
>>> Hi
>>>
>>> On Mon, Nov 27, 2017 at 11:23 AM, Harshal Dhumal <
>>> harshal.dhu...@enterprisedb.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Please find attached patch to fix RM2811
>>>>
>>>> Issue was caused due to assumption made when current position in log
>>>> file while reading it reaches to last line then
>>>> we were assuming process is finished. However this is not the case.
>>>> Background process may be busy performing
>>>> some other task and logs might not be logged to file immediately. So we
>>>> should also check process exit code along
>>>> with above condition.
>>>>
>>>> Apart from above this patch also includes minor fix related to status
>>>> text colour.
>>>>
>>>
>>> Seems to work nicely, except that the text scrolls out of view almost
>>> immediately. Can you fix it to jump to the end of the text when more is
>>> appended please?
>>>
>>> Fixed.
>>
>
> Looks good - though when I tested with a large database, it got to a point
> and then just stopped adding more data to the logs (in fact, even the err
> file in the process_log directory stopped receiving updates). However, the
> database carried on being dumped, and it had definitely moved past the
> point it was up to in the logs. I've attached all the output received - it
> literally stops at the 'c'.
>
> I've committed the patch, but can you see if you can reproduce this please?
>
Sure Dave. I'll have a look at this once.

>
> Thanks!
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

Re: Fix for RM2811

[Harshal Dhumal]

Hi Dave,
-- 
On Mon, Nov 27, 2017 at 9:20 PM, Harshal Dhumal 
wrote:

>
>
> On Mon, Nov 27, 2017 at 6:32 PM, Dave Page  wrote:
>
>> Hi
>>
>> On Mon, Nov 27, 2017 at 12:29 PM, Harshal Dhumal <
>> harshal.dhu...@enterprisedb.com> wrote:
>>
>>> Hi Dave,
>>>
>>> Please find updated patch.
>>>
>>> --
>>> *Harshal Dhumal*
>>> *Sr. Software Engineer*
>>>
>>> EnterpriseDB India: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>>> On Mon, Nov 27, 2017 at 5:28 PM, Dave Page  wrote:
>>>
>>>> Hi
>>>>
>>>> On Mon, Nov 27, 2017 at 11:23 AM, Harshal Dhumal <
>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Please find attached patch to fix RM2811
>>>>>
>>>>> Issue was caused due to assumption made when current position in log
>>>>> file while reading it reaches to last line then
>>>>> we were assuming process is finished. However this is not the case.
>>>>> Background process may be busy performing
>>>>> some other task and logs might not be logged to file immediately. So
>>>>> we should also check process exit code along
>>>>> with above condition.
>>>>>
>>>>> Apart from above this patch also includes minor fix related to status
>>>>> text colour.
>>>>>
>>>>
>>>> Seems to work nicely, except that the text scrolls out of view almost
>>>> immediately. Can you fix it to jump to the end of the text when more is
>>>> appended please?
>>>>
>>>> Fixed.
>>>
>>
>> Looks good - though when I tested with a large database, it got to a
>> point and then just stopped adding more data to the logs (in fact, even the
>> err file in the process_log directory stopped receiving updates). However,
>> the database carried on being dumped, and it had definitely moved past the
>> point it was up to in the logs. I've attached all the output received - it
>> literally stops at the 'c'.
>>
>> I've committed the patch, but can you see if you can reproduce this
>> please?
>>
> Sure Dave. I'll have a look at this once.
>

I tried backing up server with size ~1.3 GB multiple times and was able see
log till the end.

Thanks!
>>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>
>

[pgAdmin4][RM2892] Handle password changes properly if error occurs during sending email

[Harshal Dhumal]

Hi,

Unlike flask login-manager flask-security does not provide facility to pass
custom view
function to any of callbacks like change/reset/forgot password. So we cannot
handle any exceptions occurred during changing/resetting password.
Only way we can handle such exceptions is writing our own routes for these
callbacks and
add addition code to handle such exceptions.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/__init__.py b/web/pgadmin/__init__.py
index a1d10b8..0196852 100644
--- a/web/pgadmin/__init__.py
+++ b/web/pgadmin/__init__.py
@@ -174,6 +174,23 @@ def create_app(app_name=None):
 if not app_name:
 app_name = config.APP_NAME
 
+# Only enable password related functionality in server mode.
+if config.SERVER_MODE is True:
+# Some times we need to access these config params where application
+# context is not available (we can't use current_app.config in those
+# cases even with current_app.app_context())
+# So update these params in config itself.
+# And also these updated config values will picked up by application
+# since we are updating config before the application instance is
+# created.
+
+config.SECURITY_RECOVERABLE = True
+config.SECURITY_CHANGEABLE = True
+# Now we'll open change password page in alertify dialog
+# we don't want it to redirect to main page after password
+# change operation so we will open the same password change page again.
+config.SECURITY_POST_CHANGE_VIEW = 'browser.change_password'
+
 """Create the Flask application, startup logging and dynamically load
 additional modules (blueprints) that are found in this directory."""
 app = PgAdmin(__name__, static_url_path='/static')
@@ -276,18 +293,6 @@ def create_app(app_name=None):
 getattr(config, 'SQLITE_TIMEOUT', 500)
 )
 
-# Only enable password related functionality in server mode.
-if config.SERVER_MODE is True:
-# TODO: Figure out how to disable /logout and /login
-app.config['SECURITY_RECOVERABLE'] = True
-app.config['SECURITY_CHANGEABLE'] = True
-# Now we'll open change password page in alertify dialog
-# we don't want it to redirect to main page after password
-# change operation so we will open the same password change page again.
-app.config.update(
-dict(SECURITY_POST_CHANGE_VIEW='security.change_password')
-)
-
 # Create database connection object and mailer
 db.init_app(app)
 
diff --git a/web/pgadmin/browser/__init__.py b/web/pgadmin/browser/__init__.py
index a70a751..b54441b 100644
--- a/web/pgadmin/browser/__init__.py
+++ b/web/pgadmin/browser/__init__.py
@@ -8,19 +8,32 @@
 ##
 
 import json
+import logging
 from abc import ABCMeta, abstractmethod, abstractproperty
-
 import six
+from socket import error as SOCKETErrorException
+from smtplib import SMTPConnectError, SMTPResponseException,\
+SMTPServerDisconnected, SMTPDataError,SMTPHeloError, SMTPException, \
+SMTPAuthenticationError, SMTPSenderRefused, SMTPRecipientsRefused
 from flask import current_app, render_template, url_for, make_response, flash,\
-Response
+Response, request, after_this_request, redirect
 from flask_babel import gettext
-from flask_login import current_user
-from flask_security import login_required
+from flask_login import current_user, login_required
+from flask_security.decorators import anonymous_user_required
 from flask_gravatar import Gravatar
 from pgadmin.settings import get_setting
 from pgadmin.utils import PgAdminModule
 from pgadmin.utils.ajax import make_json_response
 from pgadmin.utils.preferences import Preferences
+from werkzeug.datastructures import MultiDict
+from flask_security.views import _security, _commit, _render_json, _ctx
+from flask_security.changeable import change_user_password
+from flask_security.recoverable import reset_password_token_status, \
+generate_reset_password_token, update_password
+from flask_security.utils import config_value, do_flash, get_url, get_message,\
+slash_url_suffix, login_user, send_mail
+from flask_security.signals import reset_password_instructions_sent
+
 
 import config
 from pgadmin import current_blueprint
@@ -528,6 +541,7 @@ def index():
 
 return response
 
+
 @blueprint.route("/js/utils.js")
 @login_required
 def utils():
@@ -677,3 +691,188 @@ def get_nodes():
 nodes.extend(submodule.get_nodes())
 
 return make_json_response(data=nodes)
+
+# Only register route if SECURITY_CHANGEABLE is set to True
+# We can't access app context here so cannot
+# use app.config['SECURITY_CHA

[pgAdmin4][RM2922] Login desktop user only once in runtime

[Harshal Dhumal]

Hi,

Please find attached patch to fix issue where desktop user was logged in
each request
in runtime.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/__init__.py b/web/pgadmin/__init__.py
index a1d10b8..48c1b56 100644
--- a/web/pgadmin/__init__.py
+++ b/web/pgadmin/__init__.py
@@ -535,7 +535,9 @@ def create_app(app_name=None):
 ):
 abort(401)
 
-if not config.SERVER_MODE:
+if not config.SERVER_MODE:
+@app.before_first_request
+def before_first_request():
 user = user_datastore.get_user(config.DESKTOP_USER)
 
 # Throw an error if we failed to find the desktop user, to give

Re: [pgAdmin4][RM2892] Handle password changes properly if error occurs during sending email

[Harshal Dhumal]

sure Dave, looking in it now.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Thu, Nov 30, 2017 at 5:01 PM, Dave Page  wrote:

> Oh, nuts - turns out this breaks the regression tests (the Python API
> tests) if they're run with SERVER_MODE = True. Can you take a look ASAP
> please?
>
> Traceback (most recent call last):
>   File "regression/runtests.py", line 325, in 
> test_utils.login_tester_account(test_client)
>   File 
> "/Users/dpage/git/pgadmin4/web/regression/python_test_utils/test_utils.py",
> line 53, in login_tester_account
> follow_redirects=True)
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
> line 772, in post
> return self.open(*args, **kw)
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/testing.py",
> line 113, in open
> follow_redirects=follow_redirects)
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
> line 751, in open
> environ, buffered=buffered)
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
> line 691, in resolve_redirect
> buffered=buffered)
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/testing.py",
> line 113, in open
> follow_redirects=follow_redirects)
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
> line 736, in open
> response = self.run_wsgi_app(environ, buffered=buffered)
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
> line 659, in run_wsgi_app
> rv = run_wsgi_app(self.application, environ, buffered=buffered)
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
> line 855, in run_wsgi_app
> app_iter = app(environ, start_response)
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 2000, in __call__
> return self.wsgi_app(environ, start_response)
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 1991, in wsgi_app
> response = self.make_response(self.handle_exception(e))
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 1567, in handle_exception
> reraise(exc_type, exc_value, tb)
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 1988, in wsgi_app
> response = self.full_dispatch_request()
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 1641, in full_dispatch_request
> rv = self.handle_user_exception(e)
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 1544, in handle_user_exception
> reraise(exc_type, exc_value, tb)
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 1639, in full_dispatch_request
> rv = self.dispatch_request()
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 1625, in dispatch_request
> return self.view_functions[rule.endpoint](**req.view_args)
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py",
> line 792, in decorated_view
> return func(*args, **kwargs)
>   File "/Users/dpage/git/pgadmin4/web/pgadmin/browser/__init__.py", line
> 527, in index
> _=gettext
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/templating.py",
> line 134, in render_template
> context, ctx.app)
>   File 
> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/templating.py",
> line 116, in _render
> rv = template.render(context)
>   File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-
> packages/jinja2/environment.py", line 969, in render
> return self.environment.handle_exception(exc_info, True)
>   File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-
> packages/jinja2/environment.py", line 742, in handle_exception
> reraise(exc_type, exc_value, tb)
>   File 
> "/Users/dpage/git/pgadmin4/web/pgadmin/browser/templates/browser/index.html",
> line 1, in top-level template code
> {% extends "base.html" %}
>   File "

Re: [pgAdmin4][RM2892] Handle password changes properly if error occurs during sending email

[Harshal Dhumal]

Hi Dave,

Please find patch to fix testsuite.

Changes:
1. Set config parameters SECURITY_RECOVERABLE and SECURITY_CHANGEABLE to
True if SERVER_MODE is True
while running testsuite (Though we have set these parameters to True in
create_app function but regression testsuite
initialises (imports) browser module before the app instance is created.)

2. Updated new URL references in testsuite code.

3. Also I have changed URL */browser/forgot_password* to
*/browser/reset_password* to make it consistent
with flask-security URL (as there is no such URL /forgot in flask-security)


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Thu, Nov 30, 2017 at 5:09 PM, Harshal Dhumal <
harshal.dhu...@enterprisedb.com> wrote:

> sure Dave, looking in it now.
>
> --
> *Harshal Dhumal*
> *Sr. Software Engineer*
>
> EnterpriseDB India: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
> On Thu, Nov 30, 2017 at 5:01 PM, Dave Page  wrote:
>
>> Oh, nuts - turns out this breaks the regression tests (the Python API
>> tests) if they're run with SERVER_MODE = True. Can you take a look ASAP
>> please?
>>
>> Traceback (most recent call last):
>>   File "regression/runtests.py", line 325, in 
>> test_utils.login_tester_account(test_client)
>>   File 
>> "/Users/dpage/git/pgadmin4/web/regression/python_test_utils/test_utils.py",
>> line 53, in login_tester_account
>> follow_redirects=True)
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>> line 772, in post
>> return self.open(*args, **kw)
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/testing.py",
>> line 113, in open
>> follow_redirects=follow_redirects)
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>> line 751, in open
>> environ, buffered=buffered)
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>> line 691, in resolve_redirect
>> buffered=buffered)
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/testing.py",
>> line 113, in open
>> follow_redirects=follow_redirects)
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>> line 736, in open
>> response = self.run_wsgi_app(environ, buffered=buffered)
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>> line 659, in run_wsgi_app
>> rv = run_wsgi_app(self.application, environ, buffered=buffered)
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>> line 855, in run_wsgi_app
>> app_iter = app(environ, start_response)
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 2000, in __call__
>> return self.wsgi_app(environ, start_response)
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 1991, in wsgi_app
>> response = self.make_response(self.handle_exception(e))
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 1567, in handle_exception
>> reraise(exc_type, exc_value, tb)
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 1988, in wsgi_app
>> response = self.full_dispatch_request()
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 1641, in full_dispatch_request
>> rv = self.handle_user_exception(e)
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 1544, in handle_user_exception
>> reraise(exc_type, exc_value, tb)
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 1639, in full_dispatch_request
>> rv = self.dispatch_request()
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 1625, in dispatch_request
>> return self.view_functions[rule.endpoint](**req.view_args)
>>   File 
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_lo

Re: [pgAdmin4][RM2922] Login desktop user only once in runtime

[Harshal Dhumal]

Dave,

On Thu, Nov 30, 2017 at 3:38 PM, Dave Page  wrote:

> Hi
>
> Does this resolve the suggestion in https://redmine.postgresql.
> org/issues/2919 about creating new sessions?
>

Do you think we should allow endpoint /misc/ping to be accessible outside
of Qt application
as long as user passes PGADMIN_KEY with request? (this will require new
session to be created for each /misc/ping request)


>
> On Thu, Nov 30, 2017 at 7:58 AM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi,
>>
>> Please find attached patch to fix issue where desktop user was logged in
>> each request
>> in runtime.
>>
>> --
>> *Harshal Dhumal*
>> *Sr. Software Engineer*
>>
>> EnterpriseDB India: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

[pgAdmin4][RM2782][RM2822] fix array representation in query tool and data grid

[Harshal Dhumal]

Hi,

Attached patch fixes array representation issue as well as multidimensional
array representation issue.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/feature_tests/test_data.json b/web/pgadmin/feature_tests/test_data.json
index 6c53cc8..03aca9a 100644
--- a/web/pgadmin/feature_tests/test_data.json
+++ b/web/pgadmin/feature_tests/test_data.json
@@ -16,11 +16,11 @@
   "13": ["", "false", "bool"],
   "14": ["", "[null]", "text[]"],
   "15": ["{}", "{}", "text[]"],
-  "16": ["{data,,'',\"\",\\'\\',\\\"\\\"}", "{data,[null],,,'',\"\"}", "text[]"],
+  "16": ["{data,NULL,'',\"\"}", "{data,NULL,'',\"\"}", "text[]"],
   "17": ["{}", "{}", "int[]"],
-  "18": ["{123,,456}", "{123,[null],456}", "int[]"],
+  "18": ["{123,,456}", "{123,NULL,456}", "int[]"],
   "19": ["", "[null]", "boolean[]"],
-  "20": ["{false,,true}", "{false,[null],true}", "boolean[]"]
+  "20": ["{false,null,true}", "{f,NULL,t}", "boolean[]"]
 }
   }
 }
diff --git a/web/pgadmin/static/js/slickgrid/editors.js b/web/pgadmin/static/js/slickgrid/editors.js
index 0b38da5..518ab7f 100644
--- a/web/pgadmin/static/js/slickgrid/editors.js
+++ b/web/pgadmin/static/js/slickgrid/editors.js
@@ -210,28 +210,13 @@
   $input.select();
 }
   } else {
-var data = [];
-for (var k in item[args.column.field]) {
-  if (_.isUndefined(item[args.column.field][k]) || _.isNull(item[args.column.field][k])) {
-data.push('');
-  } else if (item[args.column.field][k] === "") {
-data.push("''");
-  } else if (item[args.column.field][k] === "''") {
-data.push("\\'\\'");
-  } else if (item[args.column.field][k] === '""') {
-data.push('\\"\\"');
-  } else {
-data.push(item[args.column.field][k]);
-$input.select();
-  }
-}
-defaultValue = data;
-$input.val('{' + data.join() +'}');
-
+$input.val(defaultValue = item[args.column.field]);
+$input.select();
   }
 };
 
 this.serializeValue = function () {
+
   var value = $input.val();
   // If empty return null
   if (value === "") {
@@ -249,31 +234,7 @@
   return value;
 }
   } else {
-
-// Remove leading { and trailing }.
-// Also remove leading and trailing whitespaces.
-var value = $.trim(value.slice(1, -1));
-
-if(value == '') {
-  return [];
-}
-
-var data = [];
-value = value.split(',');
-for (var k in value) {
-  if (value[k] == "") {
-data.push(null);  //empty string from editor is null value.
-  } else if (value[k] === "''" || value[k] === '""') {
-data.push('');// double quote from editor is blank string;
-  } else if (value[k] === "\\'\\'") {
-data.push("''");
-  } else if (value[k] === '\\"\\"') {
-data.push('""');
-  } else {
-data.push(value[k]);
-  }
-}
-return data;
+return $.trim(value);
   }
 };
 
@@ -943,14 +904,16 @@
 };
 
 this.serializeValue = function () {
-  if ($input.val() === "") {
+  var value = $input.val();
+
+  if (value === "") {
 return null;
   }
 
   if(args.column.is_array) {
 // Remove leading { and trailing }.
 // Also remove leading and trailing whitespaces.
-var val = $.trim($input.val().slice(1, -1));
+var val = $.trim(value.slice(1, -1));
 
 if(val == '') {
   return [];
@@ -964,7 +927,7 @@
 return val;
   }
 
-  return $input.val();
+  return value;
 };
 
 this.applyValue = function (item, state) {
diff --git a/web/pgadmin/static/js/slickgrid/formatters.js b/web/pgadmin/static/js/slickgrid/formatters.js
index 5de2dce..31dbbac 100644
--- a/web/pgadmin/static/js/slickgrid/formatters.js
+++ b/web/pgadmin/static/js/slickg

Re: [pgAdmin4][Patch]: Do not render security URL in templates if running in Desktop mode

[Harshal Dhumal]

Hi Murtuza,

Moving login related code from under  decorator @app.before_first_request
to @app.before_request
will cause runtime user to login on each request. I think we need to find
some better way.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Thu, Dec 14, 2017 at 2:02 PM, Murtuza Zabuawala <
murtuza.zabuaw...@enterprisedb.com> wrote:

> Hi,
>
> PFA patch to fix the issue where we were rendering Security URL's like
> change password, reset password in HTML templates while running in Desktop
> mode, we only register these security blueprints in Server mode hence it
> was failing with BuildError.
> RM#2952
>
> Also reverted RM#2922 because it not working as expected and causing
> runtime to render login screen.
>
>
> Thanks to Neel for helping me in testing the patch with latest runtime
> code.
>
>
> --
> Regards,
> Murtuza Zabuawala
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
>

Re: Enums printout - possible bug

[Harshal Dhumal]

+pgadmin-hackers

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Thu, Dec 14, 2017 at 5:26 PM, Harshal Dhumal <
harshal.dhu...@enterprisedb.com> wrote:

> Hi,
>
> Please find attached patch for various data type test cases.
>
>
> --
> *Harshal Dhumal*
> *Sr. Software Engineer*
>
> EnterpriseDB India: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
> On Mon, Dec 11, 2017 at 5:24 PM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>>
>> --
>> *Harshal Dhumal*
>> *Sr. Software Engineer*
>>
>> EnterpriseDB India: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>> On Sat, Dec 9, 2017 at 12:06 PM, Khushboo Vashi <
>> khushboo.va...@enterprisedb.com> wrote:
>>
>>>
>>>
>>> On Sat, Dec 9, 2017 at 11:50 AM, Harshal Dhumal <
>>> harshal.dhu...@enterprisedb.com> wrote:
>>>
>>>>
>>>> On Sat, Dec 9, 2017 at 11:30 AM, Dave Page  wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> Yeah, that one is in my list to work on. Have you specifically tested
>>>>> it with enum types (not enum[])?
>>>>>
>>>>
>>>> Yes.
>>>>
>>>
>>> @Harshal,
>>>
>>> Can you add multidimensional array, enum types into feature test
>>> (pg_datatype_validation_test.py) ?
>>>
>>
>> I think I need to add test cases for all array data types not just enum
>> types since fix was not targeted for any particular data type.
>>
>>
>>>
>>> [image: Inline image 3]
>>>>
>>>>
>>>>
>>>>>
>>>>> On Sat, Dec 9, 2017 at 5:55 AM, Harshal Dhumal <
>>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>>
>>>>>> Hi Dave,
>>>>>>
>>>>>> Recently I sent a patch
>>>>>> <https://www.postgresql.org/message-id/CAFiP3vzj6t2QuhkWy-sHnpcQB4tiq%2BK6gqLOVQkkcOuBtksvfw%40mail.gmail.com>
>>>>>> regarding multidimensional array representation issue.
>>>>>> In that patch I have reworked about how multidimensional (1 dimension
>>>>>> to n dimension)
>>>>>> array data should be represented in grid.
>>>>>> Also this patch covers almost all the array data types including
>>>>>> composite array data types like
>>>>>> int8range[], enum[], inet[], cidr[], macaddr[], uuid[], xml[], bit[],
>>>>>> varbit[] and so on.
>>>>>>
>>>>>> Please review the patch and let me know if any thing needs to be
>>>>>> included in this patch.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Harshal Dhumal*
>>>>>> *Sr. Software Engineer*
>>>>>>
>>>>>> EnterpriseDB India: http://www.enterprisedb.com
>>>>>> The Enterprise PostgreSQL Company
>>>>>>
>>>>>> On Sat, Dec 9, 2017 at 10:45 AM, Dave Page  wrote:
>>>>>>
>>>>>>> Murtuza, can you investigate please?
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> On Fri, Dec 8, 2017 at 8:08 PM, Bartosz Dmytrak 
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> According to documentation: https://www.postgresql.org/doc
>>>>>>>> s/current/static/functions-enum.html when I execute SELECT
>>>>>>>> enum_range(null::rainbow) then output should look like this:
>>>>>>>> {red,orange,yellow,green,blue,purple}
>>>>>>>>
>>>>>>>> But in pgAdmin output looks like this:
>>>>>>>> {{,r,e,d,,,o,r,a,n,g,e,,,y,e,l,l,o,w,,,g,r,e,e,n,,,b,l,u,e,,,p,u,r,p,l,e,}}.
>>>>>>>> When you click on the field, popup window shows correct value. I’ve 
>>>>>>>> double
>>>>>>>> checked it in psql, and output is correct. I think it could be 
>>>>>>>> considered
>>>>>>>> as bug in pgAdmin
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> PgAdmin details:
>>>>>>>>
>>>>>>>> *Version *2.0
>>>>>>>>
>>>>>>>> *Python Version *2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016,
>>>>>>>> 20:42:59) [MSC v.1500 32 bit (Intel)]
>>>>>>>>
>>>>>>>> *Flask Version *0.12.2
>>>>>>>>
>>>>>>>> *Application Mode *Desktop
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Pg Version: 10.1
>>>>>>>>
>>>>>>>> Regardless this one, many thanks for your great job pgAdmin Team!
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>>
>>>>>>>> *Bartosz Dmytrak*
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Dave Page
>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>> Twitter: @pgsnake
>>>>>>>
>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Dave Page
>>>>> Blog: http://pgsnake.blogspot.com
>>>>> Twitter: @pgsnake
>>>>>
>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>> The Enterprise PostgreSQL Company
>>>>>
>>>>
>>>>
>>>
>>
>

Re: [pgAdmin4][Patch]: Adding connection status in Query tool

[Harshal Dhumal]

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Dec 19, 2017 at 7:47 PM, Murtuza Zabuawala <
murtuza.zabuaw...@enterprisedb.com> wrote:

>
>
> On Tue, Dec 19, 2017 at 7:24 PM, Dave Page  wrote:
>
>> Hi
>>
>> Interesting. A few thoughts:
>>
>> - The pulsating icon is very off-putting. I think we need to make it only
>> flash a couple of times when we actually need to attract the attention of
>> the user.
>>
> ​As per my discussion with Chethana, In his opinion user tends to notice
> things that way more quickly.
> Are you sure you wish to flash only couple of times on error?
>
>>
>> - We shouldn't really use tooltips like this, as it may confuse folks
>> with screen readers. Should we make the icon clickable (which should have a
>> visual hint)? Maybe a drop-down status panel.
>> ​
>>
>>
> ​Sure let me check.
> ​
>
>>
>> - Do we need to poll separately for the status? Instead, why not update
>> it whenever polling for results, or executing something?
>>
> ​Then user won't be able to know the current connection status prior to
> query execution, the purpose of the feature is to make user aware of
> current connection status even if there is no query running, As most user
> tends to leave open their query tool window after their work it will be
> useful when flask session gets expired and connection to server gets closed
> after that.
>

- Status Ideal and Active can be piggy backed (when we are polling query
result).

- In case of connection closed then we are reconnecting seamlessly (as per
your last comment on RM2475).
  So letting user know in advance that connection is closed does not
provide any benefit.
  Eventually user will look of any reconnect button/option in query tool to
reconnect if we tell connection status in advance.

- If flask session expires then (irrespective of connection status) status
poll http request will be redirected to login page.
  So we won't get actual connection status in this case.


>
>> Thanks!
>>
>> On Tue, Dec 19, 2017 at 11:42 AM, Murtuza Zabuawala <
>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>
>>> Hi,
>>>
>>> PFA patch to add the connection status
>>> <http://initd.org/psycopg/docs/extensions.html#transaction-status-constants>
>>> in query tool, this feature will allow user to check the database
>>> connection status in query tool itself, it will also provide the detailed
>>> status as a tooltip when user hovers on it, the most benefit of the feature
>>> will be when user open query tool in new Browser Tab where Browser tree is
>>> not visible to user, user can also configure the status polling time using
>>> preference dialog.
>>> RM#2475
>>>
>>> Apart from that I have also removed the ..sqleditor/static/css/sqledit
>>> or.css reference from ../datagrid/templates/datagrid/index.html file
>>> because we are already bundling the "sqleditor.css" file in main
>>> "style.css" file.
>>>
>>>
>>> Thanks to Chethana for his UI related inputs and to Surinder for helping
>>> me on html alignment issues.
>>>
>>>
>>> --
>>> Regards,
>>> Murtuza Zabuawala
>>> EnterpriseDB: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>>>
>>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>
>

RM2815: Relogin to pgAdmin from sqleditor/datadrid if session exprires

[Harshal Dhumal]

Hi,

Plz find attached patch to fix session expired issue
from sqleditor/datadrid.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/__init__.py b/web/pgadmin/__init__.py
index db676c5..220ea6b 100644
--- a/web/pgadmin/__init__.py
+++ b/web/pgadmin/__init__.py
@@ -25,7 +25,7 @@ from flask_paranoid import Paranoid
 
 from pgadmin.utils import PgAdminModule, driver
 from pgadmin.utils.versioned_template_loader import VersionedTemplateLoader
-from pgadmin.utils.session import create_session_interface
+from pgadmin.utils.session import create_session_interface, pga_unauthorised
 from werkzeug.local import LocalProxy
 from werkzeug.utils import find_modules
 
@@ -344,6 +344,9 @@ def create_app(app_name=None):
 
 security.init_app(app, user_datastore)
 
+# register custom unauthorised handler.
+app.login_manager.unauthorized_handler(pga_unauthorised)
+
 app.session_interface = create_session_interface(app)
 
 # Make the Session more secure against XSS & CSRF when running in web mode
diff --git a/web/pgadmin/browser/server_groups/servers/__init__.py b/web/pgadmin/browser/server_groups/servers/__init__.py
index 5330942..626526c 100644
--- a/web/pgadmin/browser/server_groups/servers/__init__.py
+++ b/web/pgadmin/browser/server_groups/servers/__init__.py
@@ -202,6 +202,9 @@ class ServerModule(sg.ServerGroupPluginModule):
 """
 ServerType.register_preferences()
 
+def get_exposed_url_endpoints(self):
+return ['NODE-server.connect_id']
+
 
 class ServerMenuItem(MenuItem):
 def __init__(self, **kwargs):
diff --git a/web/pgadmin/tools/datagrid/__init__.py b/web/pgadmin/tools/datagrid/__init__.py
index 6b1d699..75f63d7 100644
--- a/web/pgadmin/tools/datagrid/__init__.py
+++ b/web/pgadmin/tools/datagrid/__init__.py
@@ -17,7 +17,6 @@ import random
 from flask import Response, url_for, session, request, make_response
 from werkzeug.useragents import UserAgent
 from flask import current_app as app
-from flask_babel import gettext
 from flask_security import login_required
 from pgadmin.tools.sqleditor.command import *
 from pgadmin.utils import PgAdminModule
@@ -27,6 +26,9 @@ from pgadmin.utils.ajax import make_json_response, bad_request, \
 from config import PG_DEFAULT_DRIVER
 from pgadmin.utils.preferences import Preferences
 from pgadmin.model import Server
+from pgadmin.utils.driver import get_driver
+from pgadmin.utils.exception import ConnectionLost
+
 
 class DataGridModule(PgAdminModule):
 """
@@ -90,11 +92,11 @@ def show_filter():
 
 
 @blueprint.route(
-'/initialize/datagrid/',
+'/initialize/datagrid//',
 methods=["PUT", "POST"], endpoint="initialize_datagrid"
 )
 @login_required
-def initialize_datagrid(cmd_type, obj_type, sid, did, obj_id):
+def initialize_datagrid(cmd_type, obj_type, sgid, sid, did, obj_id):
 """
 This method is responsible for creating an asynchronous connection.
 After creating the connection it will instantiate and initialize
@@ -104,6 +106,7 @@ def initialize_datagrid(cmd_type, obj_type, sid, did, obj_id):
 Args:
 cmd_type: Contains value for which menu item is clicked.
 obj_type: Contains type of selected object for which data grid to be render
+sgid: Server group Id
 sid: Server Id
 did: Database Id
 obj_id: Id of currently selected object
@@ -118,15 +121,26 @@ def initialize_datagrid(cmd_type, obj_type, sid, did, obj_id):
 conn_id = str(random.randint(1, 999))
 try:
 manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(sid)
+# default_conn is same connection which is created when user connect to
+# database from tree
+default_conn = manager.connection(did=did)
 conn = manager.connection(did=did, conn_id=conn_id,
   use_binary_placeholder=True,
   array_to_string=True)
+except ConnectionLost as e:
+raise
 except Exception as e:
+app.logger.error(e)
 return internal_server_error(errormsg=str(e))
 
-# Connect the Server
+status, msg = default_conn.connect()
+if not status:
+app.logger.error(msg)
+return internal_server_error(errormsg=str(msg))
+
 status, msg = conn.connect()
 if not status:
+app.logger.error(msg)
 return internal_server_error(errormsg=str(msg))
 
 try:
@@ -135,10 +149,13 @@ def initialize_datagrid(cmd_type, obj_type, sid, did, obj_id):
 obj_type = 'table'
 
 # Get the object as per the object type
-command_obj = ObjectRegistry.get_object(obj_type, conn_id=conn_id, sid=sid,
-did=did, obj_id=obj_id, cmd_type=cmd_type,
+command_obj

Re: RM2815: Relogin to pgAdmin from sqleditor/datadrid if session exprires

[Harshal Dhumal]

Further details:

1. If session is expired and user performs any action from sqleditor that
makes ajax call
then in ajax error call back user can check and handle login related error
using code snippet.

if (pgAdmin.Browser.UserManagement.is_pga_login_required(xhr)) {
  return pgAdmin.Browser.UserManagement.pga_login();
}

Where is xhr is standard xhr or jqxhr object.

2. Similarly for connection lost (only maintenance db connection as we can
recover or reconnect other
connections if maintenance db connection is alive). It will attempt to
create/reconnect connection without
asking password (to handle passwordless connection, or saveed password  or
password from pgpass file)
If connection to database still fails then it'll prompt for password.

Code snippet:
SqlEditorController.handle_connection_lost();
 once connection lost is detected one can call handle_connection_lost() to
reconnect.

3. We maintain some additional data in session to maintain affinity between
each sqleditor/datagrid instance to backend database connection. However if
session expires and user
re-loggins then we need to first restore affinity between sqleditor to
backend database before we can start
using query tool.

Code snippet:

if(is_new_transaction_required(xhr)) {
  SqlEditorController.init_transaction();
}




-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, Jan 5, 2018 at 11:44 AM, Harshal Dhumal <
harshal.dhu...@enterprisedb.com> wrote:

> Hi,
>
> Plz find attached patch to fix session expired issue
> from sqleditor/datadrid.
>
> --
> *Harshal Dhumal*
> *Sr. Software Engineer*
>
> EnterpriseDB India: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

Re: RM2815: Relogin to pgAdmin from sqleditor/datadrid if session exprires

[Harshal Dhumal]

On Mon, Jan 8, 2018 at 4:34 PM, Dave Page  wrote:

> Hi
>
> On Fri, Jan 5, 2018 at 7:50 AM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Further details:
>>
>> 1. If session is expired and user performs any action from sqleditor that
>> makes ajax call
>> then in ajax error call back user can check and handle login related
>> error using code snippet.
>>
>> if (pgAdmin.Browser.UserManagement.is_pga_login_required(xhr)) {
>>   return pgAdmin.Browser.UserManagement.pga_login();
>> }
>>
>> Where is xhr is standard xhr or jqxhr object.
>>
>> 2. Similarly for connection lost (only maintenance db connection as we
>> can recover or reconnect other
>> connections if maintenance db connection is alive). It will attempt to
>> create/reconnect connection without
>> asking password (to handle passwordless connection, or saveed password
>>  or password from pgpass file)
>> If connection to database still fails then it'll prompt for password.
>>
>> Code snippet:
>> SqlEditorController.handle_connection_lost();
>>  once connection lost is detected one can call handle_connection_lost()
>> to reconnect.
>>
>> 3. We maintain some additional data in session to maintain affinity
>> between
>> each sqleditor/datagrid instance to backend database connection. However
>> if session expires and user
>> re-loggins then we need to first restore affinity between sqleditor to
>> backend database before we can start
>> using query tool.
>>
>> Code snippet:
>>
>> if(is_new_transaction_required(xhr)) {
>>   SqlEditorController.init_transaction();
>> }
>>
>> (note: I haven't looked at the code yet)
>
> How does this handle re-establishment of the connection mid-transaction,
> or, if the user has modified any session variables?
>
> ServeManager and Connection Manager are written in a such way that if any
connection is lost except maintenance db connection
then we can re-connect or create new connection without prompting for
database password and if maintenance db connection is lost
then It prompts for password.

Regarding session variables as long as flask session is not expired we uses
same session variables. But in case of user logout (due to
flask session expire) we create new transaction id and sets new session
variables for that particular Sql editor /datagrid instance.




> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

Re: RM2815: Relogin to pgAdmin from sqleditor/datadrid if session exprires

[Harshal Dhumal]

On Mon, Jan 8, 2018 at 5:15 PM, Dave Page  wrote:

> HI
>
> On Mon, Jan 8, 2018 at 11:41 AM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> On Mon, Jan 8, 2018 at 4:34 PM, Dave Page  wrote:
>>
>>> Hi
>>>
>>> On Fri, Jan 5, 2018 at 7:50 AM, Harshal Dhumal <
>>> harshal.dhu...@enterprisedb.com> wrote:
>>>
>>>> Further details:
>>>>
>>>> 1. If session is expired and user performs any action from sqleditor
>>>> that makes ajax call
>>>> then in ajax error call back user can check and handle login related
>>>> error using code snippet.
>>>>
>>>> if (pgAdmin.Browser.UserManagement.is_pga_login_required(xhr)) {
>>>>   return pgAdmin.Browser.UserManagement.pga_login();
>>>> }
>>>>
>>>> Where is xhr is standard xhr or jqxhr object.
>>>>
>>>> 2. Similarly for connection lost (only maintenance db connection as we
>>>> can recover or reconnect other
>>>> connections if maintenance db connection is alive). It will attempt to
>>>> create/reconnect connection without
>>>> asking password (to handle passwordless connection, or saveed password
>>>>  or password from pgpass file)
>>>> If connection to database still fails then it'll prompt for password.
>>>>
>>>> Code snippet:
>>>> SqlEditorController.handle_connection_lost();
>>>>  once connection lost is detected one can call handle_connection_lost()
>>>> to reconnect.
>>>>
>>>> 3. We maintain some additional data in session to maintain affinity
>>>> between
>>>> each sqleditor/datagrid instance to backend database connection.
>>>> However if session expires and user
>>>> re-loggins then we need to first restore affinity between sqleditor to
>>>> backend database before we can start
>>>> using query tool.
>>>>
>>>> Code snippet:
>>>>
>>>> if(is_new_transaction_required(xhr)) {
>>>>   SqlEditorController.init_transaction();
>>>> }
>>>>
>>>> (note: I haven't looked at the code yet)
>>>
>>> How does this handle re-establishment of the connection mid-transaction,
>>> or, if the user has modified any session variables?
>>>
>>> ServeManager and Connection Manager are written in a such way that if
>> any connection is lost except maintenance db connection
>> then we can re-connect or create new connection without prompting for
>> database password and if maintenance db connection is lost
>> then It prompts for password.
>>
>
> Right.
>
>
>>
>> Regarding session variables as long as flask session is not expired we
>> uses same session variables. But in case of user logout (due to
>> flask session expire) we create new transaction id and sets new session
>> variables for that particular Sql editor /datagrid instance.
>>
>
> I mean DB session variables (and related things). For example, if the user
> executed queries such as the following, then they absolutely need to know
> if the session got reset:
>

Ok.
Then in this case can we notify user about the same. That we're unable to
restore old database connection and created new one and therefore
any DB session variables were set/modified (like SET CLIENT_ENCODING...,
SET DATESTYLE...) are lost (or similar message).



> CREATE TEMPORARY TABLE 
> SET ROLE ...
> SET [various other options]
>
> If the user has done any of those things (or similar things that I haven't
> thought of), then we cannot just blindly reset the database connection.
>

We only create new connection if flask session was expired as we lost
transaction id associated with Sql editor/datagrid and therefore unique
connection id
given to connection which was associated to Sql editor/datagrid. In this
case we can notify about same (as stated above).


-- 
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

Re: RM2815: Relogin to pgAdmin from sqleditor/datadrid if session exprires

[Harshal Dhumal]

On Mon, Jan 8, 2018 at 6:11 PM, Dave Page  wrote:

>
>
> On Mon, Jan 8, 2018 at 12:37 PM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> On Mon, Jan 8, 2018 at 5:15 PM, Dave Page  wrote:
>>
>>> HI
>>>
>>> On Mon, Jan 8, 2018 at 11:41 AM, Harshal Dhumal <
>>> harshal.dhu...@enterprisedb.com> wrote:
>>>
>>>> On Mon, Jan 8, 2018 at 4:34 PM, Dave Page  wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> On Fri, Jan 5, 2018 at 7:50 AM, Harshal Dhumal <
>>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>>
>>>>>> Further details:
>>>>>>
>>>>>> 1. If session is expired and user performs any action from sqleditor
>>>>>> that makes ajax call
>>>>>> then in ajax error call back user can check and handle login related
>>>>>> error using code snippet.
>>>>>>
>>>>>> if (pgAdmin.Browser.UserManagement.is_pga_login_required(xhr)) {
>>>>>>   return pgAdmin.Browser.UserManagement.pga_login();
>>>>>> }
>>>>>>
>>>>>> Where is xhr is standard xhr or jqxhr object.
>>>>>>
>>>>>> 2. Similarly for connection lost (only maintenance db connection as
>>>>>> we can recover or reconnect other
>>>>>> connections if maintenance db connection is alive). It will attempt
>>>>>> to create/reconnect connection without
>>>>>> asking password (to handle passwordless connection, or saveed
>>>>>> password  or password from pgpass file)
>>>>>> If connection to database still fails then it'll prompt for password.
>>>>>>
>>>>>> Code snippet:
>>>>>> SqlEditorController.handle_connection_lost();
>>>>>>  once connection lost is detected one can call
>>>>>> handle_connection_lost() to reconnect.
>>>>>>
>>>>>> 3. We maintain some additional data in session to maintain affinity
>>>>>> between
>>>>>> each sqleditor/datagrid instance to backend database connection.
>>>>>> However if session expires and user
>>>>>> re-loggins then we need to first restore affinity between sqleditor
>>>>>> to backend database before we can start
>>>>>> using query tool.
>>>>>>
>>>>>> Code snippet:
>>>>>>
>>>>>> if(is_new_transaction_required(xhr)) {
>>>>>>   SqlEditorController.init_transaction();
>>>>>> }
>>>>>>
>>>>>> (note: I haven't looked at the code yet)
>>>>>
>>>>> How does this handle re-establishment of the connection
>>>>> mid-transaction, or, if the user has modified any session variables?
>>>>>
>>>>> ServeManager and Connection Manager are written in a such way that if
>>>> any connection is lost except maintenance db connection
>>>> then we can re-connect or create new connection without prompting for
>>>> database password and if maintenance db connection is lost
>>>> then It prompts for password.
>>>>
>>>
>>> Right.
>>>
>>>
>>>>
>>>> Regarding session variables as long as flask session is not expired we
>>>> uses same session variables. But in case of user logout (due to
>>>> flask session expire) we create new transaction id and sets new session
>>>> variables for that particular Sql editor /datagrid instance.
>>>>
>>>
>>> I mean DB session variables (and related things). For example, if the
>>> user executed queries such as the following, then they absolutely need to
>>> know if the session got reset:
>>>
>>
>> Ok.
>> Then in this case can we notify user about the same. That we're unable to
>> restore old database connection and created new one and therefore
>> any DB session variables were set/modified (like SET CLIENT_ENCODING...,
>> SET DATESTYLE...) are lost (or similar message).
>>
>>
>>
>>> CREATE TEMPORARY TABLE 
>>> SET ROLE ...
>>> SET [various other options]
>>>
>>> If the user has done any of those things (or similar things that I
>>> haven't thought of), then we cannot just blindly reset the database
>>> connection.
&

Re: RM2815: Relogin to pgAdmin from sqleditor/datadrid if session exprires

[Harshal Dhumal]

On Mon, Jan 8, 2018 at 7:26 PM, Dave Page  wrote:

>
>
> On Mon, Jan 8, 2018 at 1:18 PM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>>
>> On Mon, Jan 8, 2018 at 6:11 PM, Dave Page  wrote:
>>
>>>
>>>
>>> On Mon, Jan 8, 2018 at 12:37 PM, Harshal Dhumal <
>>> harshal.dhu...@enterprisedb.com> wrote:
>>>
>>>> On Mon, Jan 8, 2018 at 5:15 PM, Dave Page  wrote:
>>>>
>>>>> HI
>>>>>
>>>>> On Mon, Jan 8, 2018 at 11:41 AM, Harshal Dhumal <
>>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>>
>>>>>> On Mon, Jan 8, 2018 at 4:34 PM, Dave Page  wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> On Fri, Jan 5, 2018 at 7:50 AM, Harshal Dhumal <
>>>>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>>>>
>>>>>>>> Further details:
>>>>>>>>
>>>>>>>> 1. If session is expired and user performs any action from
>>>>>>>> sqleditor that makes ajax call
>>>>>>>> then in ajax error call back user can check and handle login
>>>>>>>> related error using code snippet.
>>>>>>>>
>>>>>>>> if (pgAdmin.Browser.UserManagement.is_pga_login_required(xhr)) {
>>>>>>>>   return pgAdmin.Browser.UserManagement.pga_login();
>>>>>>>> }
>>>>>>>>
>>>>>>>> Where is xhr is standard xhr or jqxhr object.
>>>>>>>>
>>>>>>>> 2. Similarly for connection lost (only maintenance db connection as
>>>>>>>> we can recover or reconnect other
>>>>>>>> connections if maintenance db connection is alive). It will attempt
>>>>>>>> to create/reconnect connection without
>>>>>>>> asking password (to handle passwordless connection, or saveed
>>>>>>>> password  or password from pgpass file)
>>>>>>>> If connection to database still fails then it'll prompt for
>>>>>>>> password.
>>>>>>>>
>>>>>>>> Code snippet:
>>>>>>>> SqlEditorController.handle_connection_lost();
>>>>>>>>  once connection lost is detected one can call
>>>>>>>> handle_connection_lost() to reconnect.
>>>>>>>>
>>>>>>>> 3. We maintain some additional data in session to maintain affinity
>>>>>>>> between
>>>>>>>> each sqleditor/datagrid instance to backend database connection.
>>>>>>>> However if session expires and user
>>>>>>>> re-loggins then we need to first restore affinity between sqleditor
>>>>>>>> to backend database before we can start
>>>>>>>> using query tool.
>>>>>>>>
>>>>>>>> Code snippet:
>>>>>>>>
>>>>>>>> if(is_new_transaction_required(xhr)) {
>>>>>>>>   SqlEditorController.init_transaction();
>>>>>>>> }
>>>>>>>>
>>>>>>>> (note: I haven't looked at the code yet)
>>>>>>>
>>>>>>> How does this handle re-establishment of the connection
>>>>>>> mid-transaction, or, if the user has modified any session variables?
>>>>>>>
>>>>>>> ServeManager and Connection Manager are written in a such way that
>>>>>> if any connection is lost except maintenance db connection
>>>>>> then we can re-connect or create new connection without prompting for
>>>>>> database password and if maintenance db connection is lost
>>>>>> then It prompts for password.
>>>>>>
>>>>>
>>>>> Right.
>>>>>
>>>>>
>>>>>>
>>>>>> Regarding session variables as long as flask session is not expired
>>>>>> we uses same session variables. But in case of user logout (due to
>>>>>> flask session expire) we create new transaction id and sets new
>>>>>> session variables for that particular Sql editor /datagrid instance.
>>>>>>
>>>>>
>

Re: RM2815: Relogin to pgAdmin from sqleditor/datadrid if session exprires

[Harshal Dhumal]

Hi Dave,

Please find updated patch where we are notifying user about connection
reset.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Jan 8, 2018 at 7:39 PM, Harshal Dhumal <
harshal.dhu...@enterprisedb.com> wrote:

> On Mon, Jan 8, 2018 at 7:26 PM, Dave Page  wrote:
>
>>
>>
>> On Mon, Jan 8, 2018 at 1:18 PM, Harshal Dhumal <
>> harshal.dhu...@enterprisedb.com> wrote:
>>
>>>
>>> On Mon, Jan 8, 2018 at 6:11 PM, Dave Page  wrote:
>>>
>>>>
>>>>
>>>> On Mon, Jan 8, 2018 at 12:37 PM, Harshal Dhumal <
>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>
>>>>> On Mon, Jan 8, 2018 at 5:15 PM, Dave Page  wrote:
>>>>>
>>>>>> HI
>>>>>>
>>>>>> On Mon, Jan 8, 2018 at 11:41 AM, Harshal Dhumal <
>>>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>>>
>>>>>>> On Mon, Jan 8, 2018 at 4:34 PM, Dave Page  wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> On Fri, Jan 5, 2018 at 7:50 AM, Harshal Dhumal <
>>>>>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>>>>>
>>>>>>>>> Further details:
>>>>>>>>>
>>>>>>>>> 1. If session is expired and user performs any action from
>>>>>>>>> sqleditor that makes ajax call
>>>>>>>>> then in ajax error call back user can check and handle login
>>>>>>>>> related error using code snippet.
>>>>>>>>>
>>>>>>>>> if (pgAdmin.Browser.UserManagement.is_pga_login_required(xhr)) {
>>>>>>>>>   return pgAdmin.Browser.UserManagement.pga_login();
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> Where is xhr is standard xhr or jqxhr object.
>>>>>>>>>
>>>>>>>>> 2. Similarly for connection lost (only maintenance db connection
>>>>>>>>> as we can recover or reconnect other
>>>>>>>>> connections if maintenance db connection is alive). It will
>>>>>>>>> attempt to create/reconnect connection without
>>>>>>>>> asking password (to handle passwordless connection, or saveed
>>>>>>>>> password  or password from pgpass file)
>>>>>>>>> If connection to database still fails then it'll prompt for
>>>>>>>>> password.
>>>>>>>>>
>>>>>>>>> Code snippet:
>>>>>>>>> SqlEditorController.handle_connection_lost();
>>>>>>>>>  once connection lost is detected one can call
>>>>>>>>> handle_connection_lost() to reconnect.
>>>>>>>>>
>>>>>>>>> 3. We maintain some additional data in session to maintain
>>>>>>>>> affinity between
>>>>>>>>> each sqleditor/datagrid instance to backend database connection.
>>>>>>>>> However if session expires and user
>>>>>>>>> re-loggins then we need to first restore affinity between
>>>>>>>>> sqleditor to backend database before we can start
>>>>>>>>> using query tool.
>>>>>>>>>
>>>>>>>>> Code snippet:
>>>>>>>>>
>>>>>>>>> if(is_new_transaction_required(xhr)) {
>>>>>>>>>   SqlEditorController.init_transaction();
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> (note: I haven't looked at the code yet)
>>>>>>>>
>>>>>>>> How does this handle re-establishment of the connection
>>>>>>>> mid-transaction, or, if the user has modified any session variables?
>>>>>>>>
>>>>>>>> ServeManager and Connection Manager are written in a such way that
>>>>>>> if any connection is lost except maintenance db connection
>>>>>>> then we can re-connect or create new connection without prompting
>>>>>>> for database password and if maintenance db connection

Re: RM2815: Relogin to pgAdmin from sqleditor/datadrid if session exprires

[Harshal Dhumal]

Hi,

Please find rebased patch.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jan 9, 2018 at 9:21 PM, Harshal Dhumal <
harshal.dhu...@enterprisedb.com> wrote:

> Hi Dave,
>
> Please find updated patch where we are notifying user about connection
> reset.
>
> --
> *Harshal Dhumal*
> *Sr. Software Engineer*
>
> EnterpriseDB India: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
> On Mon, Jan 8, 2018 at 7:39 PM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> On Mon, Jan 8, 2018 at 7:26 PM, Dave Page  wrote:
>>
>>>
>>>
>>> On Mon, Jan 8, 2018 at 1:18 PM, Harshal Dhumal <
>>> harshal.dhu...@enterprisedb.com> wrote:
>>>
>>>>
>>>> On Mon, Jan 8, 2018 at 6:11 PM, Dave Page  wrote:
>>>>
>>>>>
>>>>>
>>>>> On Mon, Jan 8, 2018 at 12:37 PM, Harshal Dhumal <
>>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>>
>>>>>> On Mon, Jan 8, 2018 at 5:15 PM, Dave Page  wrote:
>>>>>>
>>>>>>> HI
>>>>>>>
>>>>>>> On Mon, Jan 8, 2018 at 11:41 AM, Harshal Dhumal <
>>>>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>>>>
>>>>>>>> On Mon, Jan 8, 2018 at 4:34 PM, Dave Page 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> On Fri, Jan 5, 2018 at 7:50 AM, Harshal Dhumal <
>>>>>>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>>>>>>
>>>>>>>>>> Further details:
>>>>>>>>>>
>>>>>>>>>> 1. If session is expired and user performs any action from
>>>>>>>>>> sqleditor that makes ajax call
>>>>>>>>>> then in ajax error call back user can check and handle login
>>>>>>>>>> related error using code snippet.
>>>>>>>>>>
>>>>>>>>>> if (pgAdmin.Browser.UserManagement.is_pga_login_required(xhr)) {
>>>>>>>>>>   return pgAdmin.Browser.UserManagement.pga_login();
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> Where is xhr is standard xhr or jqxhr object.
>>>>>>>>>>
>>>>>>>>>> 2. Similarly for connection lost (only maintenance db connection
>>>>>>>>>> as we can recover or reconnect other
>>>>>>>>>> connections if maintenance db connection is alive). It will
>>>>>>>>>> attempt to create/reconnect connection without
>>>>>>>>>> asking password (to handle passwordless connection, or saveed
>>>>>>>>>> password  or password from pgpass file)
>>>>>>>>>> If connection to database still fails then it'll prompt for
>>>>>>>>>> password.
>>>>>>>>>>
>>>>>>>>>> Code snippet:
>>>>>>>>>> SqlEditorController.handle_connection_lost();
>>>>>>>>>>  once connection lost is detected one can call
>>>>>>>>>> handle_connection_lost() to reconnect.
>>>>>>>>>>
>>>>>>>>>> 3. We maintain some additional data in session to maintain
>>>>>>>>>> affinity between
>>>>>>>>>> each sqleditor/datagrid instance to backend database connection.
>>>>>>>>>> However if session expires and user
>>>>>>>>>> re-loggins then we need to first restore affinity between
>>>>>>>>>> sqleditor to backend database before we can start
>>>>>>>>>> using query tool.
>>>>>>>>>>
>>>>>>>>>> Code snippet:
>>>>>>>>>>
>>>>>>>>>> if(is_new_transaction_required(xhr)) {
>>>>>>>>>>   SqlEditorController.init_transaction();
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> (note: I haven't looke

Re: [pgAdmin4][Patch]: Adding connection status in Query tool

[Harshal Dhumal]

Murtuza, I think we should only poll if sqleditor/datagrid is visible.
We've *wcDocker.EVENT.VISIBILITY_CHANGED *event when panel visibility
changes.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Wed, Jan 10, 2018 at 1:55 PM, Murtuza Zabuawala <
murtuza.zabuaw...@enterprisedb.com> wrote:

> Hi Dave,
>
> PFA updated patch.
>
> On Tue, Jan 9, 2018 at 7:57 PM, Dave Page  wrote:
>
>> Hi
>>
>> On Tue, Jan 9, 2018 at 6:33 AM, Murtuza Zabuawala <
>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>
>>> Hi Dave,
>>>
>>> Please find updated patch.
>>>
>>
>> I turned off the status option, but polling is still happening. This
>> should definitely stop! :-)
>>
> ​Fixed typo in variable.
>
> Can you also reverse the enable/disable switch and the interval setting on
>> the preferences page? I think Enable/Disable should be at the top, and be
>> followed by the interval.
>> ​
>>
>  Fixed.
> ​
> But just a heads up we won't be able to put '?' after 'Connection
> status'​  eg: '
> Connection status
> ​?'​
> ​
> ​Then string sorting again puts it after 'Connection status refresh rate'
> ​:)
>
>> ​
>>
>> Thanks.
>>
>>
>>>
>>> On Mon, Jan 8, 2018 at 7:21 PM, Dave Page  wrote:
>>>
>>>>
>>>>
>>>> On Mon, Jan 8, 2018 at 1:24 PM, Murtuza Zabuawala <
>>>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>>>
>>>>> Hi Dave,
>>>>>
>>>>> PFA updated patch.
>>>>>
>>>>> On Mon, Jan 8, 2018 at 5:11 PM, Dave Page  wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> On Fri, Jan 5, 2018 at 8:49 AM, Murtuza Zabuawala <
>>>>>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>>>>>
>>>>>>> Hi Dave,
>>>>>>>
>>>>>>> PFA updated patch,
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jan 3, 2018 at 10:44 PM, Dave Page 
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> On Thu, Dec 28, 2017 at 9:38 AM, Murtuza Zabuawala <
>>>>>>>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> PFA updated patch based on new design suggested by Chethana.
>>>>>>>>> The patch also includes some misc fixes related to object
>>>>>>>>> validation.
>>>>>>>>> RM#2475
>>>>>>>>>
>>>>>>>>
>>>>>>>> This seems much nicer, but I still think there are some tweaks to
>>>>>>>> make:
>>>>>>>>
>>>>>>>> 1) If I open a query tool, and then stop the application server,
>>>>>>>> the icon is updated to show the broken connection. However, unless I
>>>>>>>> execute a query in the query tool before the server is shut down, the
>>>>>>>> connection status won't recover when the server is restarted. If I do 
>>>>>>>> run a
>>>>>>>> query first (SELECT 1; will do), then the connection status will 
>>>>>>>> recover.
>>>>>>>>
>>>>>>>
>>>>>>> I have logged​
>>>>>>> ​
>>>>>>> https://redmine.postgresql.org/issues/2983
>>>>>>>
>>>>>>>
>>>>>>>> 2) I think the "connected" icon should be in $primary-blue
>>>>>>>> (#2c76b4). The green is ugly and not overly easy to read. It's also
>>>>>>>> distracting as it catches the eye, which the default, non-error state
>>>>>>>> should not do.
>>>>>>>>
>>>>>>> ​Fixed​
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> Much better.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> 3) I'm not overly happy with the the status text popover. After
>>>>

Re: [pgAdmin4][Patch]: Adding connection status in Query tool

[Harshal Dhumal]

On Thu, Jan 11, 2018 at 12:06 PM, Murtuza Zabuawala <
murtuza.zabuaw...@enterprisedb.com> wrote:

> User can open Query tool in new browser window where we'll not have
> wcDocker panel.
>

In that case we can use window onfocus and onblur events
<http://www.thefutureoftheweb.com/blog/detect-browser-window-focus>.
In sqleditor there are cases where we've written conditional code based on
whether sqleditor is opened in new window or not.


On Thu, Jan 11, 2018 at 12:00 PM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Murtuza, I think we should only poll if sqleditor/datagrid is visible.
>> We've *wcDocker.EVENT.VISIBILITY_CHANGED *event when panel visibility
>> changes.
>>
>> --
>> *Harshal Dhumal*
>> *Sr. Software Engineer*
>>
>> EnterpriseDB India: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>> On Wed, Jan 10, 2018 at 1:55 PM, Murtuza Zabuawala <
>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>
>>> Hi Dave,
>>>
>>> PFA updated patch.
>>>
>>> On Tue, Jan 9, 2018 at 7:57 PM, Dave Page  wrote:
>>>
>>>> Hi
>>>>
>>>> On Tue, Jan 9, 2018 at 6:33 AM, Murtuza Zabuawala <
>>>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>>>
>>>>> Hi Dave,
>>>>>
>>>>> Please find updated patch.
>>>>>
>>>>
>>>> I turned off the status option, but polling is still happening. This
>>>> should definitely stop! :-)
>>>>
>>> ​Fixed typo in variable.
>>>
>>> Can you also reverse the enable/disable switch and the interval setting
>>>> on the preferences page? I think Enable/Disable should be at the top, and
>>>> be followed by the interval.
>>>> ​
>>>>
>>>  Fixed.
>>> ​
>>> But just a heads up we won't be able to put '?' after 'Connection
>>> status'​  eg: '
>>> Connection status
>>> ​?'​
>>> ​
>>> ​Then string sorting again puts it after 'Connection status refresh
>>> rate' ​:)
>>>
>>>> ​
>>>>
>>>> Thanks.
>>>>
>>>>
>>>>>
>>>>> On Mon, Jan 8, 2018 at 7:21 PM, Dave Page  wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Jan 8, 2018 at 1:24 PM, Murtuza Zabuawala <
>>>>>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>>>>>
>>>>>>> Hi Dave,
>>>>>>>
>>>>>>> PFA updated patch.
>>>>>>>
>>>>>>> On Mon, Jan 8, 2018 at 5:11 PM, Dave Page  wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> On Fri, Jan 5, 2018 at 8:49 AM, Murtuza Zabuawala <
>>>>>>>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Dave,
>>>>>>>>>
>>>>>>>>> PFA updated patch,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jan 3, 2018 at 10:44 PM, Dave Page 
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi
>>>>>>>>>>
>>>>>>>>>> On Thu, Dec 28, 2017 at 9:38 AM, Murtuza Zabuawala <
>>>>>>>>>> murtuza.zabuaw...@enterprisedb.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> PFA updated patch based on new design suggested by Chethana.
>>>>>>>>>>> The patch also includes some misc fixes related to object
>>>>>>>>>>> validation.
>>>>>>>>>>> RM#2475
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This seems much nicer, but I still think there are some tweaks to
>>>>>>>>>> make:
>>>>>>>>>>
>>>>>>>>>> 1) If I open a query tool, and then stop the application server,
>>>>>>>>>> the icon is updated to show the broken connection. However, unless I
>>>>>>>>>> execu

Fix for RM3052: Alter column error

[Harshal Dhumal]

Hi,.

Issue was caused due to we were considering internal datatype length (eg.
smallint 2 bytes, int4 4 bytes, integer 8 bytes) as user configurable
length.
However we cannot change internal datatype lengths of some primitive
datatypes.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/utils.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/utils.py
index f152392..f911bbb 100644
--- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/utils.py
+++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/utils.py
@@ -1690,10 +1690,8 @@ class BaseTableView(PGChildNodeView):
 old_data['isdup'], old_data['attndims'], old_data['atttypmod']
 )
 
-length = False
-precision = False
-
-# If the column data type has not changed then fetch old length and precision
+# If the column data type has not changed then fetch
+# old length and precision
 if 'elemoid' in old_data and 'cltype' not in c:
 length, precision, typeval = \
 self.get_length_precision(old_data['elemoid'])
@@ -1714,6 +1712,23 @@ class BaseTableView(PGChildNodeView):
 c['attlen'] = None
 c['attprecision'] = None
 
+if 'cltype' in c:
+typename = c['cltype']
+if 'hasSqrBracket' in c and c['hasSqrBracket']:
+typename += '[]'
+length, precision, typeval = \
+self.get_length_precision(typename)
+
+# if new datatype does not have length or precision
+# then we cannot apply length or precision of old
+# datatype to new one.
+
+if not length:
+old_data['attlen'] = -1
+
+if not precision:
+old_data['attprecision'] = None
+
 old_data['cltype'] = DataTypeReader.parse_type_name(
 old_data['cltype']
 )
diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/types/templates/type/macros/get_full_type_sql_format.macros b/web/pgadmin/browser/server_groups/servers/databases/schemas/types/templates/type/macros/get_full_type_sql_format.macros
index 9e7276f..1520232 100644
--- a/web/pgadmin/browser/server_groups/servers/databases/schemas/types/templates/type/macros/get_full_type_sql_format.macros
+++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/types/templates/type/macros/get_full_type_sql_format.macros
@@ -59,5 +59,5 @@ time({{ data.attlen }}) with time zone {% endif %}{% if o_data.hasSqrBracket %}[
 ({{ data.attlen }}{% elif o_data.attlen and o_data.attlen != 'None' %}({{ o_data.attlen }}{% endif %}{% if data.attprecision and data.attprecision != 'None' %}
 , {{ data.attprecision }}){% elif o_data.attprecision and o_data.attprecision != 'None' %}, {{ o_data.attprecision }}){% else %}){% endif %}
 {% endif %}
-{% endif %}
+{% endif %}{% if o_data.hasSqrBracket %}[]{% endif %}
 {% endmacro %}
diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/utils.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/utils.py
index bba36ed..ddebd5e 100644
--- a/web/pgadmin/browser/server_groups/servers/databases/schemas/utils.py
+++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/utils.py
@@ -15,6 +15,7 @@ from flask import render_template
 from pgadmin.browser.collection import CollectionNodeModule
 from pgadmin.utils.ajax import internal_server_error
 
+
 class SchemaChildModule(CollectionNodeModule):
 """
 Base class for the schema child node.
@@ -149,20 +150,35 @@ class DataTypeReader:
 return True, res
 
 @staticmethod
-def get_length_precision(elemoid):
+def get_length_precision(elemoid_or_name):
 precision = False
 length = False
 typeval = ''
 
-# Check against PGOID for specific type
-if elemoid:
-if elemoid in (1560, 1561, 1562, 1563, 1042, 1043,
-  1014, 1015):
+# Check against PGOID/typename for specific type
+if elemoid_or_name:
+if elemoid_or_name in (1560, 'bit&#

RM3080 Fix for alignment issue in keyboard preferences

[Harshal Dhumal]

Hi,

Please find attached patch to fix alignment issue in keyboard preferences.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/browser/__init__.py b/web/pgadmin/browser/__init__.py
index 04065f5..1d0374a 100644
--- a/web/pgadmin/browser/__init__.py
+++ b/web/pgadmin/browser/__init__.py
@@ -214,10 +214,10 @@ class BrowserModule(PgAdminModule):
 category_label=gettext('Properties')
 )
 fields = [
-{'name': 'alt', 'type': 'checkbox', 'label': gettext('Alt / Option')},
+{'name': 'key', 'type': 'keyCode', 'label': gettext('Key')},
 {'name': 'shift', 'type': 'checkbox', 'label': gettext('Shift')},
 {'name': 'control', 'type': 'checkbox', 'label': gettext('Ctrl')},
-{'name': 'key', 'type': 'keyCode', 'label': gettext('Key')}
+{'name': 'alt', 'type': 'checkbox', 'label': gettext('Alt/Option')}
 ]
 
 self.preference.register(
diff --git a/web/pgadmin/tools/debugger/__init__.py b/web/pgadmin/tools/debugger/__init__.py
index e2703ee..305c152 100644
--- a/web/pgadmin/tools/debugger/__init__.py
+++ b/web/pgadmin/tools/debugger/__init__.py
@@ -84,25 +84,24 @@ class DebuggerModule(PgAdminModule):
 
 shortcut_fields = [
 {
-'name': 'alt',
-'type': 'checkbox',
-'label': gettext('Alt/Option')
+'name': 'key',
+'type': 'keyCode',
+'label': gettext('Key')
 },
 {
 'name': 'shift',
 'type': 'checkbox',
 'label': gettext('Shift')
 },
-
 {
 'name': 'control',
 'type': 'checkbox',
 'label': gettext('Ctrl')
 },
 {
-'name': 'key',
-'type': 'keyCode',
-'label': gettext('Key')
+'name': 'alt',
+'type': 'checkbox',
+'label': gettext('Alt/Option')
 }
 ]
 

RM2898 Keyboard navigation in dialog tabs (nav tabs)

[Harshal Dhumal]

Hi,

Please find attached patch to enable keyboard navigation in dialog.

To allow navigation from one tab pane (bootstrap tab pane) to another one
I have added two new shortcut preferences *Dialog tab previous *(
shift+control+[ ) and *Dialog tab next* ( shift+control+] ) for backward
and forward tab navigation.

Also all dialog controls (within same tab pane) can be navigated using TAB
key.


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/docs/en_US/keyboard_shortcuts.rst b/docs/en_US/keyboard_shortcuts.rst
index 1142975..f7bf402 100644
--- a/docs/en_US/keyboard_shortcuts.rst
+++ b/docs/en_US/keyboard_shortcuts.rst
@@ -41,6 +41,21 @@ When using main browser window, the following keyboard shortcuts are available:
 | Alt+Shift+G   | Direct debugging   |
 +---++
 
+
+**Dialog tab shortcuts**
+
+When any dialog which has bootstrap tabs (nav tabs) below shortcuts are
+available to navigate within them:
+
++---++
+| Shortcut for all platform | Function   |
++===++
+| Control+Shift+[   | Dialog tab previous|
++---++
+| Control+Shift+]   | Dialog tab next|
++---++
+
+
 **SQL Editors**
 
 When using the syntax-highlighting SQL editors, the following shortcuts are available:
diff --git a/web/pgadmin/browser/__init__.py b/web/pgadmin/browser/__init__.py
index 04065f5..1e74e96 100644
--- a/web/pgadmin/browser/__init__.py
+++ b/web/pgadmin/browser/__init__.py
@@ -430,6 +430,36 @@ class BrowserModule(PgAdminModule):
 fields=fields
 )
 
+self.preference.register(
+'keyboard_shortcuts',
+'dialog_tab_next',
+gettext('Dialog tab next'),
+'keyboardshortcut',
+{
+'alt': False,
+'shift': True,
+'control': True,
+'key': {'key_code': 93, 'char': ']'}
+},
+category_label=gettext('Keyboard shortcuts'),
+fields=fields
+)
+
+self.preference.register(
+'keyboard_shortcuts',
+'dialog_tab_previous',
+gettext('Dialog tab previous'),
+'keyboardshortcut',
+{
+'alt': False,
+'shift': True,
+'control': True,
+'key': {'key_code': 91, 'char': '['}
+},
+category_label=gettext('Keyboard shortcuts'),
+fields=fields
+)
+
 def get_exposed_url_endpoints(self):
 """
 Returns:
diff --git a/web/pgadmin/browser/static/js/browser.js b/web/pgadmin/browser/static/js/browser.js
index 13af4ea..e166293 100644
--- a/web/pgadmin/browser/static/js/browser.js
+++ b/web/pgadmin/browser/static/js/browser.js
@@ -1951,6 +1951,29 @@ define('pgadmin.browser', [
   brace_matching: pgBrowser.utils.braceMatching,
   indent_with_tabs: pgBrowser.utils.is_indent_with_tabs,
 },
+find_and_set_focus: function(container) {
+  setTimeout(function() {
+var first_el = container
+  .find('button.fa-plus:first');
+
+if (first_el.length == 0) {
+  first_el = container.find('.pgadmin-controls:first')
+  .find('input,.select2-selection,.CodeMirror-scroll');
+}
+
+// don't club textarea with input,select2,codemirror,
+// as it has least precedence.
+
+if (first_el.length == 0) {
+  first_el = container.find('.pgadmin-controls:first').find('textarea');
+}
+
+if(first_el.length > 0) {
+  first_el[0].focus();
+}
+
+  }, 500);
+},
 
   });
 
diff --git a/web/pgadmin/browser/static/js/keyboard.js b/web/pgadmin/browser/static/js/keyboard.js
index 95b77db..ea775d7 100644
--- a/web/pgadmin/browser/static/js/keyboard.js
+++ b/web/pgadmin/browser/static/js/keyboard.js
@@ -27,7 +27,9 @@ function(_, S, pgAdmin, $, Mousetrap) {
   'sub_menu_create': getShortcut(pgBrowser.get_preference('browser', 'sub_menu_create').value),
   'sub_menu_delete&

RM2983: Fix transient error after application restart

[Harshal Dhumal]

Hi,

Please find patch to fix specified_version_number ValueError issue.

Now we try to restore server details like server version number and version
string
from flask session in-case if there was a application restart.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/utils/driver/psycopg2/__init__.py b/web/pgadmin/utils/driver/psycopg2/__init__.py
index 5acfb32..81442e4 100644
--- a/web/pgadmin/utils/driver/psycopg2/__init__.py
+++ b/web/pgadmin/utils/driver/psycopg2/__init__.py
@@ -1752,6 +1752,8 @@ class ServerManager(object):
 
 res = dict()
 res['sid'] = self.sid
+res['ver'] = self.ver
+res['sversion'] = self.sversion
 if hasattr(self, 'password') and self.password:
 # If running under PY2
 if hasattr(self.password, 'decode'):
@@ -1861,6 +1863,20 @@ WHERE db.oid = {0}""".format(did))
 Helps restoring to reconnect the auto-connect connections smoothly on
 reload/restart of the app server..
 """
+# restore server version from flask session if flask server was
+# restarted. As we need server version to resolve sql template paths.
+
+self.ver = data.get('ver', None)
+self.sversion = data.get('sversion', None)
+
+if self.ver and not self.server_type:
+from pgadmin.browser.server_groups.servers.types import ServerType
+for st in ServerType.types():
+if st.instanceOf(self.ver):
+self.server_type = st.stype
+self.server_cls = st
+break
+
 # Hmm.. we will not honour this request, when I already have
 # connections
 if len(self.connections) != 0:
@@ -1967,6 +1983,7 @@ WHERE db.oid = {0}""".format(did))
 else:
 managers[self.sid] = updated_mgr
 session['__pgsql_server_managers'] = managers
+session.force_write = True
 
 def utility(self, operation):
 """

Re: RM2898 Keyboard navigation in dialog tabs (nav tabs)

[Harshal Dhumal]

Hi,

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Feb 20, 2018 at 10:34 PM, Dave Page  wrote:

> Hi
>
> On Tue, Feb 20, 2018 at 7:22 AM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi,
>>
>> Please find attached patch to enable keyboard navigation in dialog.
>>
>> To allow navigation from one tab pane (bootstrap tab pane) to another one
>> I have added two new shortcut preferences *Dialog tab previous *(
>> shift+control+[ ) and *Dialog tab next* ( shift+control+] ) for backward
>> and forward tab navigation.
>>
>> Also all dialog controls (within same tab pane) can be navigated using
>> TAB key.
>>
>
>

> This seems unreliable to me - for example, it keeps getting stuck on the
> connection tab on the server properties dialog.
>
>

> Also, can we use the same wording as for the tabbed panel navigation
> please? E.g. Next/Previous instead of Forward/Back.
>

I have fixed all of above issues. Please find updated patch.

Thanks,


>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
diff --git a/docs/en_US/keyboard_shortcuts.rst b/docs/en_US/keyboard_shortcuts.rst
index 1142975..7699798 100644
--- a/docs/en_US/keyboard_shortcuts.rst
+++ b/docs/en_US/keyboard_shortcuts.rst
@@ -41,6 +41,21 @@ When using main browser window, the following keyboard shortcuts are available:
 | Alt+Shift+G   | Direct debugging   |
 +---++
 
+
+**Dialog tab shortcuts**
+
+When any dialog which has bootstrap tabs (nav tabs) below shortcuts are
+available to navigate within them:
+
++---++
+| Shortcut for all platform | Function   |
++===++
+| Control+Shift+[   | Dialog tab backward|
++---++
+| Control+Shift+]   | Dialog tab forward |
++---++
+
+
 **SQL Editors**
 
 When using the syntax-highlighting SQL editors, the following shortcuts are available:
diff --git a/web/pgadmin/browser/__init__.py b/web/pgadmin/browser/__init__.py
index 1d0374a..d827a49 100644
--- a/web/pgadmin/browser/__init__.py
+++ b/web/pgadmin/browser/__init__.py
@@ -430,6 +430,36 @@ class BrowserModule(PgAdminModule):
 fields=fields
 )
 
+self.preference.register(
+'keyboard_shortcuts',
+'dialog_tab_forward',
+gettext('Dialog tab forward'),
+'keyboardshortcut',
+{
+'alt': False,
+'shift': True,
+'control': True,
+'key': {'key_code': 93, 'char': ']'}
+},
+category_label=gettext('Keyboard shortcuts'),
+fields=fields
+)
+
+self.preference.register(
+'keyboard_shortcuts',
+'dialog_tab_backward',
+gettext('Dialog tab backward'),
+'keyboardshortcut',
+{
+'alt': False,
+'shift': True,
+'control': True,
+'key': {'key_code': 91, 'char': '['}
+},
+category_label=gettext('Keyboard shortcuts'),
+fields=fields
+)
+
 def get_exposed_url_endpoints(self):
 """
 Returns:
diff --git a/web/pgadmin/browser/static/js/browser.js b/web/pgadmin/browser/static/js/browser.js
index 13af4ea..cf739bb 100644
--- a/web/pgadmin/browser/static/js/browser.js
+++ b/web/pgadmin/browser/static/js/browser.js
@@ -1951,7 +1951,25 @@ define('pgadmin.browser', [
   brace_matching: pgBrowser.utils.braceMatching,
   indent_with_tabs: pgBrowser.utils.is_indent_with_tabs,
 },
+find_and_set_focus: function(container) {
+  if (container.length == 0) {
+return;
+  }
+  setTimeout(function() {
+var first_el = container
+  .find('button.fa-plus:first');
+
+if (first_el.length == 0) {
+  first_el = container.find('.pgadmin-controls:first>input,.CodeM

Re: RM2898 Keyboard navigation in dialog tabs (nav tabs)

[Harshal Dhumal]

Hi,

On Wed, Feb 21, 2018 at 10:59 PM, Joao De Almeida Pereira <
jdealmeidapere...@pivotal.io> wrote:

> Yep I installed the V2 file
>
> On Wed, Feb 21, 2018 at 11:31 AM Dave Page  wrote:
>
>> On Wed, Feb 21, 2018 at 4:22 PM, Joao De Almeida Pereira <
>> jdealmeidapere...@pivotal.io> wrote:
>>
>>> Hello Harshal,
>>>
>>> I passed the patch through our CI and all the tests passed. The changes
>>> do not break previous behavior but because there are no tests on the new
>>> feature  we could not be sure it was really working. So we did some manual
>>> testing and sometimes it doesn't work, like it gets stuck in a place and
>>> you need to press the shortcut again in order for it to move.
>>>
>>
>> It stuck because I have to wait until next tab is completely visible
(fade in effect is completed).
The fade in or fade out transition duration is 150 ms (set by bootstrap).
So I can not set focus back to tab pane
until fade in or fade out transition is completed. May be one improvement I
can do is to reduce wait time to
something 200 ms (currently it's 500 ms).

Note that the original issue reported by Dave is already fixed in updated
patch.


> Was that with the updated patch? It sounds like the issue I saw with the
>> original one.
>>
>>
>>>
>>> Codewise I have some suggestions:
>>>  - dialogTabNavigator looks a nice candidate for a class with its own
>>> file. This way we can test the behavior
>>>
>> Ok I'll move dialogTabNavigator to new file and will add test cases.

>  - There is no difference between a variable called e and a variable error
>>> so for sake of clarity I would love to see variable names that we can
>>> easily read
>>>  - We are also using 2 different types of variable naming camelCase and
>>> snake_case, if we could use only camelCase on Javascript it would make the
>>> code more uniform
>>>
>> Ok I'll do this.


>  - I noticed that there are some linting issues in the Javascript code
>>>
>> I just found that linter was disabled for this file by adding comment  /*
eslint-disable */ at first line. (not sure why we did that)


>>> Summing it up I believe that despite the feature not working 100% of the
>>> time, looks like the code is almost there but needs some refactoring to
>>> make it more readable, instead of comments we could have function calls
>>> that more clearly state what we are looking for something like
>>> DialogTabNavigator.isLastTabOfChild
>>> ​
>>> ​
>>>
>>>
>>> Thanks
>>> Joao
>>>
>>> On Wed, Feb 21, 2018 at 4:32 AM Harshal Dhumal <
>>> harshal.dhu...@enterprisedb.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> --
>>>> *Harshal Dhumal*
>>>> *Sr. Software Engineer*
>>>>
>>>> EnterpriseDB India: http://www.enterprisedb.com
>>>> The Enterprise PostgreSQL Company
>>>>
>>>> On Tue, Feb 20, 2018 at 10:34 PM, Dave Page  wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> On Tue, Feb 20, 2018 at 7:22 AM, Harshal Dhumal <
>>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Please find attached patch to enable keyboard navigation in dialog.
>>>>>>
>>>>>> To allow navigation from one tab pane (bootstrap tab pane) to another
>>>>>> one
>>>>>> I have added two new shortcut preferences *Dialog tab previous *(
>>>>>> shift+control+[ ) and *Dialog tab next* ( shift+control+] ) for backward
>>>>>> and forward tab navigation.
>>>>>>
>>>>>> Also all dialog controls (within same tab pane) can be navigated
>>>>>> using TAB key.
>>>>>>
>>>>>
>>>>>
>>>>
>>>>> This seems unreliable to me - for example, it keeps getting stuck on
>>>>> the connection tab on the server properties dialog.
>>>>>
>>>>>
>>>>
>>>>> Also, can we use the same wording as for the tabbed panel navigation
>>>>> please? E.g. Next/Previous instead of Forward/Back.
>>>>>
>>>>
>>>> I have fixed all of above issues. Please find updated patch.
>>>>
>>>> Thanks,
>>>>
>>>>
>>>>>
>>>>> --
>>>>> Dave Page
>>>>> Blog: http://pgsnake.blogspot.com
>>>>> Twitter: @pgsnake
>>>>>
>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>> The Enterprise PostgreSQL Company
>>>>>
>>>>
>>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>

Re: RM2898 Keyboard navigation in dialog tabs (nav tabs)

[Harshal Dhumal]

Hi,

Please find the updated patch for keyboard navigation.

In this patch I have reduced delay which is required until current tab
navigation is completed.
Extracted class dialogTabNavigator and put it in new file.
Added jasmine test cases.
Fixed linting issues, variable naming convention issues.



-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Wed, Feb 21, 2018 at 11:39 PM, Harshal Dhumal 
wrote:

> Hi,
>
> On Wed, Feb 21, 2018 at 10:59 PM, Joao De Almeida Pereira <
> jdealmeidapere...@pivotal.io> wrote:
>
>> Yep I installed the V2 file
>>
>> On Wed, Feb 21, 2018 at 11:31 AM Dave Page  wrote:
>>
>>> On Wed, Feb 21, 2018 at 4:22 PM, Joao De Almeida Pereira <
>>> jdealmeidapere...@pivotal.io> wrote:
>>>
>>>> Hello Harshal,
>>>>
>>>> I passed the patch through our CI and all the tests passed. The changes
>>>> do not break previous behavior but because there are no tests on the new
>>>> feature  we could not be sure it was really working. So we did some manual
>>>> testing and sometimes it doesn't work, like it gets stuck in a place and
>>>> you need to press the shortcut again in order for it to move.
>>>>
>>>
>>> It stuck because I have to wait until next tab is completely visible
> (fade in effect is completed).
> The fade in or fade out transition duration is 150 ms (set by bootstrap).
> So I can not set focus back to tab pane
> until fade in or fade out transition is completed. May be one improvement
> I can do is to reduce wait time to
> something 200 ms (currently it's 500 ms).
>
> Note that the original issue reported by Dave is already fixed in updated
> patch.
>
>
>> Was that with the updated patch? It sounds like the issue I saw with the
>>> original one.
>>>
>>>
>>>>
>>>> Codewise I have some suggestions:
>>>>  - dialogTabNavigator looks a nice candidate for a class with its own
>>>> file. This way we can test the behavior
>>>>
>>> Ok I'll move dialogTabNavigator to new file and will add test cases.
>
>>  - There is no difference between a variable called e and a variable
>>>> error so for sake of clarity I would love to see variable names that
>>>> we can easily read
>>>>  - We are also using 2 different types of variable naming camelCase and
>>>> snake_case, if we could use only camelCase on Javascript it would make the
>>>> code more uniform
>>>>
>>> Ok I'll do this.
>
>
>>  - I noticed that there are some linting issues in the Javascript code
>>>>
>>> I just found that linter was disabled for this file by adding comment
> /* eslint-disable */ at first line. (not sure why we did that)
>
>
>>>> Summing it up I believe that despite the feature not working 100% of
>>>> the time, looks like the code is almost there but needs some refactoring to
>>>> make it more readable, instead of comments we could have function calls
>>>> that more clearly state what we are looking for something like
>>>> DialogTabNavigator.isLastTabOfChild
>>>> ​
>>>> ​
>>>>
>>>>
>>>> Thanks
>>>> Joao
>>>>
>>>> On Wed, Feb 21, 2018 at 4:32 AM Harshal Dhumal <
>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> --
>>>>> *Harshal Dhumal*
>>>>> *Sr. Software Engineer*
>>>>>
>>>>> EnterpriseDB India: http://www.enterprisedb.com
>>>>> The Enterprise PostgreSQL Company
>>>>>
>>>>> On Tue, Feb 20, 2018 at 10:34 PM, Dave Page  wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> On Tue, Feb 20, 2018 at 7:22 AM, Harshal Dhumal <
>>>>>> harshal.dhu...@enterprisedb.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Please find attached patch to enable keyboard navigation in dialog.
>>>>>>>
>>>>>>> To allow navigation from one tab pane (bootstrap tab pane) to
>>>>>>> another one
>>>>>>> I have added two new shortcut preferences *Dialog tab previous *(
>>>>>>> shift+control+[ ) and *Dialog tab next* ( shift+control+] ) for backward
>>>>>>> and forward tab na

pep-8 fixes

[Harshal Dhumal]

Hi,

Please find patch to fix pep-8 issues for given modules.

1. help (__init__.py)
2. pgadmin (__init__.py)
3. browser (__init__.py, collection.py, utils.py)


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/__init__.py b/web/pgadmin/__init__.py
index 74a3124..90ecf99 100644
--- a/web/pgadmin/__init__.py
+++ b/web/pgadmin/__init__.py
@@ -291,10 +291,10 @@ def create_app(app_name=None):
 # Setup authentication
 ##
 
-app.config['SQLALCHEMY_DATABASE_URI'] = u'sqlite:///{0}?timeout={1}'.format(
-config.SQLITE_PATH.replace(u'\\', u'/'),
-getattr(config, 'SQLITE_TIMEOUT', 500)
-)
+app.config['SQLALCHEMY_DATABASE_URI'] = u'sqlite:///{0}?timeout={1}'\
+.format(config.SQLITE_PATH.replace(u'\\', u'/'),
+getattr(config, 'SQLITE_TIMEOUT', 500)
+)
 
 # Create database connection object and mailer
 db.init_app(app)
@@ -406,7 +406,8 @@ def create_app(app_name=None):
 servergroup_id = servergroup.id
 
 '''Add a server to the config database'''
-def add_server(user_id, servergroup_id, name, superuser, port, discovery_id, comment):
+def add_server(user_id, servergroup_id, name, superuser, port,
+   discovery_id, comment):
 # Create a server object if needed, and store it.
 servers = Server.query.filter_by(
 user_id=user_id,
@@ -437,7 +438,7 @@ def create_app(app_name=None):
 
 try:
 proc_arch64 = os.environ['PROCESSOR_ARCHITEW6432'].lower()
-except:
+except Exception as e:
 proc_arch64 = None
 
 if proc_arch == 'x86' and not proc_arch64:
@@ -467,7 +468,8 @@ def create_app(app_name=None):
 svr_port = winreg.QueryValueEx(inst_key, 'Port')[0]
 svr_discovery_id = inst_id
 svr_comment = gettext(
-"Auto-detected %s installation with the data directory at %s" % (
+"Auto-detected %s installation with the data "
+"directory at %s" % (
 winreg.QueryValueEx(
 inst_key, 'Display Name'
 )[0],
@@ -484,7 +486,7 @@ def create_app(app_name=None):
 )
 
 inst_key.Close()
-except:
+except Exception as e:
 pass
 else:
 # We use the postgres-winreg.ini file on non-Windows
@@ -501,7 +503,8 @@ def create_app(app_name=None):
 
 # Loop the sections, and get the data from any that are PG or PPAS
 for section in sections:
-if section.startswith('PostgreSQL/') or section.startswith('EnterpriseDB/'):
+if section.startswith('PostgreSQL/') \
+or section.startswith('EnterpriseDB/'):
 svr_name = registry.get(section, 'Description')
 svr_superuser = registry.get(section, 'Superuser')
 svr_port = registry.getint(section, 'Port')
@@ -511,14 +514,17 @@ def create_app(app_name=None):
 if hasattr(str, 'decode'):
 description = description.decode('utf-8')
 data_directory = data_directory.decode('utf-8')
-svr_comment = gettext(u"Auto-detected %s installation with the data directory at %s" % (
-description,
-data_directory
-))
+svr_comment = gettext(u"Auto-detected %s installation "
+  u"with the data directory at %s" % (
+description,
+data_directory
+)
+  )
 add_server(user_id, servergroup_id, svr_name,
-   svr_superuser, svr_port, svr_discovery_id, svr_comment)
+   svr_superuser, svr_port, svr_discovery_id,
+   svr_comment)
 
-except:
+except Exception as e:
 pass
 
 @user_logged_in.connect_via(app)
@@ -545,7 +551,8 @@ def

Re: RM2898 Keyboard navigation in dialog tabs (nav tabs)

[Harshal Dhumal]

Hi,

Please find the updated patch.

On Mon, Feb 26, 2018 at 8:03 PM, Dave Page  wrote:

> Hi
>
> On Mon, Feb 26, 2018 at 12:03 PM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi,
>>
>> Please find the updated patch for keyboard navigation.
>>
>> In this patch I have reduced delay which is required until current tab
>> navigation is completed.
>> Extracted class dialogTabNavigator and put it in new file.
>> Added jasmine test cases.
>> Fixed linting issues, variable naming convention issues.
>>
>
> This is still getting stuck on the Connection tab when I test on the
> server dialog.
>
The disabled input field (Host name/address) on connection tab was causing
issue.


>
> BTW, I've updated the documentation a little - please find the attached
> version.
>
Thanks for this. I have included revised version of documentation in
current patch.


>
> Thanks.
>
>
>
>>
>>
>>
>> --
>> *Harshal Dhumal*
>> *Sr. Software Engineer*
>>
>> EnterpriseDB India: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>> On Wed, Feb 21, 2018 at 11:39 PM, Harshal Dhumal <
>> harshaldhuma...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> On Wed, Feb 21, 2018 at 10:59 PM, Joao De Almeida Pereira <
>>> jdealmeidapere...@pivotal.io> wrote:
>>>
>>>> Yep I installed the V2 file
>>>>
>>>> On Wed, Feb 21, 2018 at 11:31 AM Dave Page  wrote:
>>>>
>>>>> On Wed, Feb 21, 2018 at 4:22 PM, Joao De Almeida Pereira <
>>>>> jdealmeidapere...@pivotal.io> wrote:
>>>>>
>>>>>> Hello Harshal,
>>>>>>
>>>>>> I passed the patch through our CI and all the tests passed. The
>>>>>> changes do not break previous behavior but because there are no tests on
>>>>>> the new feature  we could not be sure it was really working. So we did 
>>>>>> some
>>>>>> manual testing and sometimes it doesn't work, like it gets stuck in a 
>>>>>> place
>>>>>> and you need to press the shortcut again in order for it to move.
>>>>>>
>>>>>
>>>>> It stuck because I have to wait until next tab is completely visible
>>> (fade in effect is completed).
>>> The fade in or fade out transition duration is 150 ms (set by
>>> bootstrap). So I can not set focus back to tab pane
>>> until fade in or fade out transition is completed. May be one
>>> improvement I can do is to reduce wait time to
>>> something 200 ms (currently it's 500 ms).
>>>
>>> Note that the original issue reported by Dave is already fixed in
>>> updated patch.
>>>
>>>
>>>> Was that with the updated patch? It sounds like the issue I saw with
>>>>> the original one.
>>>>>
>>>>>
>>>>>>
>>>>>> Codewise I have some suggestions:
>>>>>>  - dialogTabNavigator looks a nice candidate for a class with its own
>>>>>> file. This way we can test the behavior
>>>>>>
>>>>> Ok I'll move dialogTabNavigator to new file and will add test cases.
>>>
>>>>  - There is no difference between a variable called e and a variable
>>>>>> error so for sake of clarity I would love to see variable names that
>>>>>> we can easily read
>>>>>>  - We are also using 2 different types of variable naming camelCase
>>>>>> and snake_case, if we could use only camelCase on Javascript it would 
>>>>>> make
>>>>>> the code more uniform
>>>>>>
>>>>> Ok I'll do this.
>>>
>>>
>>>>  - I noticed that there are some linting issues in the Javascript code
>>>>>>
>>>>> I just found that linter was disabled for this file by adding comment
>>> /* eslint-disable */ at first line. (not sure why we did that)
>>>
>>>
>>>>>> Summing it up I believe that despite the feature not working 100% of
>>>>>> the time, looks like the code is almost there but needs some refactoring 
>>>>>> to
>>>>>> make it more readable, instead of comments we could have function calls
>>>>>> that more clearly state what we are looking for something like
>>>>>> DialogTabNavigator.isLastTabOfChild
&

Re: RM2898 Keyboard navigation in dialog tabs (nav tabs)

[Harshal Dhumal]

Hi,

On Tue, Feb 27, 2018 at 1:08 AM, Dave Page  wrote:

> Hi,
>
> I'm still able to make it get stuck, if I tab back and forth quickly.
>
Quickly switching tabs was causing to switch to next tab before previous
navigation was completed and
this was leading to lose focus on tab pane.
Now I have made changes that it won't process any user navigation events
until current navigation is completed.


> On Mon, Feb 26, 2018 at 6:04 PM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi,
>>
>> Please find the updated patch.
>>
>> On Mon, Feb 26, 2018 at 8:03 PM, Dave Page  wrote:
>>
>>> Hi
>>>
>>> On Mon, Feb 26, 2018 at 12:03 PM, Harshal Dhumal <
>>> harshal.dhu...@enterprisedb.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Please find the updated patch for keyboard navigation.
>>>>
>>>> In this patch I have reduced delay which is required until current tab
>>>> navigation is completed.
>>>> Extracted class dialogTabNavigator and put it in new file.
>>>> Added jasmine test cases.
>>>> Fixed linting issues, variable naming convention issues.
>>>>
>>>
>>> This is still getting stuck on the Connection tab when I test on the
>>> server dialog.
>>>
>> The disabled input field (Host name/address) on connection tab was
>> causing issue.
>>
>>
>>>
>>> BTW, I've updated the documentation a little - please find the attached
>>> version.
>>>
>> Thanks for this. I have included revised version of documentation in
>> current patch.
>>
>>
>>>
>>> Thanks.
>>>
>>>
>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Harshal Dhumal*
>>>> *Sr. Software Engineer*
>>>>
>>>> EnterpriseDB India: http://www.enterprisedb.com
>>>> The Enterprise PostgreSQL Company
>>>>
>>>> On Wed, Feb 21, 2018 at 11:39 PM, Harshal Dhumal <
>>>> harshaldhuma...@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> On Wed, Feb 21, 2018 at 10:59 PM, Joao De Almeida Pereira <
>>>>> jdealmeidapere...@pivotal.io> wrote:
>>>>>
>>>>>> Yep I installed the V2 file
>>>>>>
>>>>>> On Wed, Feb 21, 2018 at 11:31 AM Dave Page  wrote:
>>>>>>
>>>>>>> On Wed, Feb 21, 2018 at 4:22 PM, Joao De Almeida Pereira <
>>>>>>> jdealmeidapere...@pivotal.io> wrote:
>>>>>>>
>>>>>>>> Hello Harshal,
>>>>>>>>
>>>>>>>> I passed the patch through our CI and all the tests passed. The
>>>>>>>> changes do not break previous behavior but because there are no tests 
>>>>>>>> on
>>>>>>>> the new feature  we could not be sure it was really working. So we did 
>>>>>>>> some
>>>>>>>> manual testing and sometimes it doesn't work, like it gets stuck in a 
>>>>>>>> place
>>>>>>>> and you need to press the shortcut again in order for it to move.
>>>>>>>>
>>>>>>>
>>>>>>> It stuck because I have to wait until next tab is completely visible
>>>>> (fade in effect is completed).
>>>>> The fade in or fade out transition duration is 150 ms (set by
>>>>> bootstrap). So I can not set focus back to tab pane
>>>>> until fade in or fade out transition is completed. May be one
>>>>> improvement I can do is to reduce wait time to
>>>>> something 200 ms (currently it's 500 ms).
>>>>>
>>>>> Note that the original issue reported by Dave is already fixed in
>>>>> updated patch.
>>>>>
>>>>>
>>>>>> Was that with the updated patch? It sounds like the issue I saw with
>>>>>>> the original one.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Codewise I have some suggestions:
>>>>>>>>  - dialogTabNavigator looks a nice candidate for a class with its
>>>>>>>> own file. This way we can test the behavior
>>>>>>>>
>>>>>>> Ok I'll move dialogTabNavigator to new file and will add test case

PEP-8 fixes

[Harshal Dhumal]

Hi,

Please find patch to fix pep-8 issues for given modules.

1. server group (__init__.py)
2. server (__init__.py, gpdb.py, types.py, utils.py)


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/browser/server_groups/__init__.py b/web/pgadmin/browser/server_groups/__init__.py
index 73cdabe..633d9ee 100644
--- a/web/pgadmin/browser/server_groups/__init__.py
+++ b/web/pgadmin/browser/server_groups/__init__.py
@@ -240,7 +240,8 @@ class ServerGroupView(NodeView):
 
 return jsonify(
 node=self.blueprint.generate_browser_node(
-"%d" % (sg.id),None,
+"%d" % sg.id,
+None,
 sg.name,
 "icon-%s" % self.node_type,
 True,
@@ -297,7 +298,8 @@ class ServerGroupView(NodeView):
 for group in groups:
 nodes.append(
 self.blueprint.generate_browser_node(
-"%d" % (group.id), None,
+"%d" % group.id,
+None,
 group.name,
 "icon-%s" % self.node_type,
 True,
@@ -306,7 +308,7 @@ class ServerGroupView(NodeView):
 )
 else:
 group = ServerGroup.query.filter_by(user_id=current_user.id,
- id=gid).first()
+id=gid).first()
 if not group:
 return gone(
 errormsg=gettext("Could not find the server group.")
diff --git a/web/pgadmin/browser/server_groups/servers/__init__.py b/web/pgadmin/browser/server_groups/servers/__init__.py
index 08d139e..dfa9d62 100644
--- a/web/pgadmin/browser/server_groups/servers/__init__.py
+++ b/web/pgadmin/browser/server_groups/servers/__init__.py
@@ -26,6 +26,7 @@ from config import PG_DEFAULT_DRIVER
 from pgadmin.model import db, Server, ServerGroup, User
 from pgadmin.utils.driver import get_driver
 
+
 def has_any(data, keys):
 """
 Checks any one of the keys present in the data given
@@ -42,8 +43,10 @@ def has_any(data, keys):
 
 return False
 
+
 def recovery_state(connection, postgres_version):
-recovery_check_sql = render_template("connect/sql/#{0}#/check_recovery.sql".format(postgres_version))
+recovery_check_sql = render_template(
+"connect/sql/#{0}#/check_recovery.sql".format(postgres_version))
 
 status, result = connection.execute_dict(recovery_check_sql)
 if status and 'rows' in result and len(result['rows']) > 0:
@@ -54,6 +57,7 @@ def recovery_state(connection, postgres_version):
 wal_paused = None
 return in_recovery, wal_paused
 
+
 def server_icon_and_background(is_connected, manager, server):
 """
 
@@ -160,16 +164,16 @@ class ServerModule(sg.ServerGroupPluginModule):
 
 scripts.extend([{
 'name': 'pgadmin.browser.server.privilege',
-'path': url_for('%s.static'% self.name, filename='js/privilege'),
+'path': url_for('%s.static' % self.name, filename='js/privilege'),
 'when': self.node_type,
 'is_template': False,
 'deps': ['pgadmin.browser.node.ui']
 }, {
 'name': 'pgadmin.browser.server.variable',
-'path': url_for('%s.static'% self.name, filename='js/variable'),
+'path': url_for('%s.static' % self.name, filename='js/variable'),
 'when': self.node_type,
 'is_template': False
-},{
+}, {
 'name': 'pgadmin.server.supported_servers',
 'path': url_for('browser.index') + 'server/supported_servers',
 'is_template': True,
@@ -246,26 +250,32 @@ class ServerNode(PGChildNodeView):
 }],
 'check_pgpass': [{'get': 'check_pgpass'}]
 })
-EXP_IP4 = "^\s*((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\."\
-"(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\."\
-"(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\."\
-"(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\s*$"
-EXP_IP6 = '^\s*((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|'\
-   '(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|'\
-   '2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|'

RM3079 fix for wrong sql datetime/time related datatypes

[Harshal Dhumal]

Hi,

Please find patch to fix wrong sql issue for time related type.

Steps to reproduce:

Alter any time/datetime array related data type, it generates sql with
addition array bracket

[image: Inline image 1]




-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/types/templates/type/macros/get_full_type_sql_format.macros b/web/pgadmin/browser/server_groups/servers/databases/schemas/types/templates/type/macros/get_full_type_sql_format.macros
index 1520232..a2ad322 100644
--- a/web/pgadmin/browser/server_groups/servers/databases/schemas/types/templates/type/macros/get_full_type_sql_format.macros
+++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/types/templates/type/macros/get_full_type_sql_format.macros
@@ -58,6 +58,6 @@ time({{ data.attlen }}) with time zone {% endif %}{% if o_data.hasSqrBracket %}[
 {% if data.attlen and data.attlen != 'None' %}
 ({{ data.attlen }}{% elif o_data.attlen and o_data.attlen != 'None' %}({{ o_data.attlen }}{% endif %}{% if data.attprecision and data.attprecision != 'None' %}
 , {{ data.attprecision }}){% elif o_data.attprecision and o_data.attprecision != 'None' %}, {{ o_data.attprecision }}){% else %}){% endif %}
-{% endif %}
 {% endif %}{% if o_data.hasSqrBracket %}[]{% endif %}
+{% endif %}
 {% endmacro %}

Re: RM3079 fix for wrong sql datetime/time related datatypes

[Harshal Dhumal]

Hi,

On Tue, Feb 27, 2018 at 8:54 PM, Dave Page  wrote:

> Hi
>
> On Tue, Feb 27, 2018 at 2:36 PM, Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi,
>>
>> Please find patch to fix wrong sql issue for time related type.
>>
>> Steps to reproduce:
>>
>> Alter any time/datetime array related data type, it generates sql with
>> addition array bracket
>>
>> [image: Inline image 1]
>>
>
> This seems to be missing the test case that you noted should be included
> in the original bug report!
>

Please find updated patch with test cases.


Thanks,

>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/tests/test_column_add.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/tests/test_column_add.py
index 54215d4..a2a8f17 100644
--- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/tests/test_column_add.py
+++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/tests/test_column_add.py
@@ -24,7 +24,7 @@ from regression.python_test_utils import test_utils as utils
 class ColumnAddTestCase(BaseTestGenerator):
 """This class will add new column under table node."""
 scenarios = [
-('Add table Node URL', dict(url='/browser/column/obj/'))
+('Add column Node URL', dict(url='/browser/column/obj/'))
 ]
 
 def setUp(self):
diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/tests/test_column_delete.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/tests/test_column_delete.py
index 28fdc7a..1496ea8 100644
--- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/tests/test_column_delete.py
+++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/tests/test_column_delete.py
@@ -24,7 +24,7 @@ from . import utils as columns_utils
 class ColumnDeleteTestCase(BaseTestGenerator):
 """This class will delete column under table node."""
 scenarios = [
-('Delete table Node URL', dict(url='/browser/column/obj/'))
+('Delete column Node URL', dict(url='/browser/column/obj/'))
 ]
 
 def setUp(self):
diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/tests/test_column_msql.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/tests/test_column_msql.py
new file mode 100644
index 000..3cdc292
--- /dev/null
+++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/tests/test_column_msql.py
@@ -0,0 +1,183 @@
+##
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##
+
+import json
+import uuid
+
+from pgadmin.browser.server_groups.servers.databases.schemas.tables.tests \
+import utils as tables_utils
+from pgadmin.browser.server_groups.servers.databases.schemas.tests import \
+utils as schema_utils
+from pgadmin.browser.server_groups.servers.databases.tests import utils as \
+database_utils
+from pgadmin.utils.route import BaseTestGenerator
+from regression import parent_node_dict
+from regression.python_test_utils import test_utils as utils
+from . import utils as columns_utils
+
+try:
+from urllib.parse import urlencode
+except ImportError as e:
+from urllib import urlencode
+
+
+class ColumnMsqlTestCase(BaseTestGenerator):
+"""This class will test msql route of column with various combinations."""
+scenarios = [
+('msql column change timestamp array length',
+ dict(
+ url='/browser/column/msql/',
+ data_type='timestamp(3) with time zone[]',
+ new_len=6,
+ expected_res='ALTER TABLE {schema}.{table}\nALTER COLUMN '
+  '{column} TYPE timestamp({len}) with time zone [];'
+ )),
+('msql column change timestamp length',
+ dict(
+ url='/browser/column/msql/',
+ data_type='timestamp(4) with time zone',
+ new_len=7,
+ expected_res='ALTER TABLE {schema}.{table}\nALTER COLUMN '
+  '{column} TYPE timestamp({len}) with time zone ;'
+ )),
+('msql

Re: RM3079 fix for wrong sql datetime/time related datatypes

[Harshal Dhumal]

Great, thanks!

On Feb 28, 2018 8:15 PM, "Joao De Almeida Pereira" <
jdealmeidapere...@pivotal.io> wrote:

> Hello Harshal,
>
> I tried the example you showed and it works. Also passed the patch through
> our CI Pipeline and everything is good.
>
> Thanks
> Joao
>
> On Wed, Feb 28, 2018 at 6:25 AM Harshal Dhumal <
> harshal.dhu...@enterprisedb.com> wrote:
>
>> Hi,
>>
>> On Tue, Feb 27, 2018 at 8:54 PM, Dave Page  wrote:
>>
>>> Hi
>>>
>>> On Tue, Feb 27, 2018 at 2:36 PM, Harshal Dhumal <
>>> harshal.dhu...@enterprisedb.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Please find patch to fix wrong sql issue for time related type.
>>>>
>>>> Steps to reproduce:
>>>>
>>>> Alter any time/datetime array related data type, it generates sql with
>>>> addition array bracket
>>>>
>>>> [image: Inline image 1]
>>>>
>>>
>>> This seems to be missing the test case that you noted should be included
>>> in the original bug report!
>>>
>>
>> Please find updated patch with test cases.
>>
>>
>> Thanks,
>>
>>>
>>> --
>>> Dave Page
>>> Blog: http://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EnterpriseDB UK: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>>

Re: Bug #3083 fix

[Harshal Dhumal]

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Apr 3, 2018 at 6:10 PM, Dave Page  wrote:

> Argh, managed to send before I finished typing...
>
> On Tue, Apr 3, 2018 at 1:38 PM, Dave Page  wrote:
>
>> Hi
>>
>> On Thu, Mar 29, 2018 at 4:29 PM, Joao De Almeida Pereira <
>> jdealmeidapere...@pivotal.io> wrote:
>>
>>> Hi Dave,
>>> That looks like in the surrounding area of the change. We run our
>>> pipeline and everything was green.
>>> Can you provide more details, which python version are you using? OS?
>>>
>>
>> That was on my travel laptop, which is macOS Sierra with the Apple
>> supplied Python 2.7.
>>
>> Interestingly, I'm on my dev laptop today (same OS and Python) and it's
>> working just fine. The difference is that the travel machine is a 12"
>> Macbook, whilst the dev machine is
>>
>
> a 15" MacBook Pro with 2 24" external monitors. That makes me wonder if
> the small screen size is causing a problem with this test, something we
> have seen before.
>
>

Yes, screen size does cause problem. Slick grid does not render all columns
if viewport is not wide enough (like it does for rows).
Remaining columns would render when user scrolls right.

To avoid similar problem in datatype feature test (commit:
88bcd3b5129db88975421e26c1bf188daf4892f9
<https://git.postgresql.org/gitweb/?p=pgadmin4.git;a=commitdiff;h=88bcd3b5129db88975421e26c1bf188daf4892f9>)
I have executed
queries in batch to limit number of columns in single query result.



>
>>
>>>
>>> Thanks
>>> Joao
>>>
>>> On Thu, Mar 29, 2018 at 9:03 AM Dave Page  wrote:
>>>
>>>> Hi
>>>>
>>>> On Wed, Mar 28, 2018 at 7:06 PM, Joao De Almeida Pereira <
>>>> jdealmeidapere...@pivotal.io> wrote:
>>>>
>>>>> Hey Akshay and Neethu
>>>>>
>>>>> We refactored the patch to add tests for the resize feature.  We were
>>>>> able to write test cases for the drag event by using spies and setting the
>>>>> rect dimensions.  In cases like this, we can just test some components in
>>>>> order to have enough confidence in the code.  So we isolated the function
>>>>> that implements the behavior of this feature and tested that it was
>>>>> performing as expected.
>>>>>
>>>>> We ran the patch through the pipelines and all of the tests passed.
>>>>>
>>>>
>>>> I'm consistently seeing the feature test failure below with this patch
>>>> applied:
>>>>
>>>> ==
>>>> FAIL: runTest (pgadmin.feature_tests.view_da
>>>> ta_dml_queries.CheckForViewDataTest)
>>>> Validate Insert, Update operations in View/Edit data with given test
>>>> data
>>>> --
>>>> Traceback (most recent call last):
>>>>   File 
>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
>>>> line 125, in runTest
>>>> self._verify_row_data(True)
>>>>   File 
>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
>>>> line 325, in _verify_row_data
>>>> self.assertEquals(cells[idx], config_data[str(idx)][1])
>>>> AssertionError: u'[null]' != u'1'
>>>> - [null]
>>>> + 1
>>>>
>>>>
>>>> --
>>>> Dave Page
>>>> Blog: http://pgsnake.blogspot.com
>>>> Twitter: @pgsnake
>>>>
>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>> The Enterprise PostgreSQL Company
>>>>
>>>
>>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

Re: RM#3294 - User need to reset the layout to see the changed preferences parameters

[Harshal Dhumal]

On Mon, Jun 11, 2018 at 6:02 PM, Aditya Toshniwal <
aditya.toshni...@enterprisedb.com> wrote:

> Hi Hackers,
>
> This is regarding RM3294 which I am working on, where user needs to reset
> the layout to see the changed preferences parameters. I am able to
> implement it, and have used the following approach:
> 1) when preference changed, fire a custom event - prefChangedEvent
>
We can also fire preference specific event instead of generic event.
And we can generate event name from preference itself like
*:::*
eg.:* browser:display:show_system_objects:update*


> 2) listent to prefChangedEvent wherever preferences should be reflected in
> realtime.
> 3) In the event handler, make the changes as per the module.
> 4) So whenever the preferences are changed, the event fires, the listener
> executes the event handler and changes are done.
>
> Benefit of this approach is no polling. But, there is a problem here.
> It is not possible to fire event in another tab/new browser window. For
> example, query tool can be open in another tab. And thus, changes are not
> reflected there. There are solutions available like updating the
> localStorage of the browser but those are not reliable and does not work
> properly on different browsers.
>
To communicate between browser tabs we can use cookie polling on client
side it self (at least it will avoid polling over http).
The main tab will update only preference specific cookie when preference is
updated and other tabs will poll required cookies (not all)
with specific interval (1 second can be configurable).


>
> Request you to kindly suggest if you any better idea.
>
> Thanks and Regards,
> Aditya Toshniwal
> Software Engineer | EnterpriseDB Software Solutions | Pune
> "Don't Complain about Heat, Plant a tree"
>

Fix for RM3547

[Harshal Dhumal]

Hi,

Please find attached patch to fix RM3547

Thanks,
-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/utils/session.py b/web/pgadmin/utils/session.py
index fa313e0..cdf39ce 100644
--- a/web/pgadmin/utils/session.py
+++ b/web/pgadmin/utils/session.py
@@ -24,6 +24,7 @@ import random
 import string
 import time
 from uuid import uuid4
+from threading import Lock
 from flask import current_app, request, flash, redirect
 from flask_login import login_url
 from pgadmin.utils.ajax import make_json_response
@@ -50,6 +51,9 @@ def _calc_hmac(body, secret):
 ).decode()
 
 
+sess_lock = Lock()
+
+
 class ManagedSession(CallbackDict, SessionMixin):
 def __init__(self, initial=None, sid=None, new=False, randval=None,
  hmac_digest=None):
@@ -111,8 +115,9 @@ class CachingSessionManager(SessionManager):
 def _normalize(self):
 if len(self._cache) > self.num_to_store:
 # Flush 20% of the cache
-while len(self._cache) > (self.num_to_store * 0.8):
-self._cache.popitem(False)
+with sess_lock:
+while len(self._cache) > (self.num_to_store * 0.8):
+self._cache.popitem(False)
 
 def new_session(self):
 session = self.parent.new_session()
@@ -122,59 +127,64 @@ class CachingSessionManager(SessionManager):
 if request.path.startswith(sp):
 return session
 
-self._cache[session.sid] = session
+with sess_lock:
+self._cache[session.sid] = session
 self._normalize()
 
 return session
 
 def remove(self, sid):
-self.parent.remove(sid)
-if sid in self._cache:
-del self._cache[sid]
+with sess_lock:
+self.parent.remove(sid)
+if sid in self._cache:
+del self._cache[sid]
 
 def exists(self, sid):
-if sid in self._cache:
-return True
-return self.parent.exists(sid)
+with sess_lock:
+if sid in self._cache:
+return True
+return self.parent.exists(sid)
 
 def get(self, sid, digest):
 session = None
-if sid in self._cache:
-session = self._cache[sid]
-if session.hmac_digest != digest:
-session = None
+with sess_lock:
+if sid in self._cache:
+session = self._cache[sid]
+if session.hmac_digest != digest:
+session = None
 
-# reset order in Dict
-del self._cache[sid]
+# reset order in Dict
+del self._cache[sid]
 
-if not session:
-session = self.parent.get(sid, digest)
+if not session:
+session = self.parent.get(sid, digest)
 
-# Do not store the session if skip paths
-for sp in self.skip_paths:
-if request.path.startswith(sp):
-return session
+# Do not store the session if skip paths
+for sp in self.skip_paths:
+if request.path.startswith(sp):
+return session
 
-self._cache[sid] = session
+self._cache[sid] = session
 self._normalize()
 
 return session
 
 def put(self, session):
-self.parent.put(session)
+with sess_lock:
+self.parent.put(session)
 
-# Do not store the session if skip paths
-for sp in self.skip_paths:
-if request.path.startswith(sp):
-return
+# Do not store the session if skip paths
+for sp in self.skip_paths:
+if request.path.startswith(sp):
+return
 
-if session.sid in self._cache:
-try:
-del self._cache[session.sid]
-except Exception:
-pass
+if session.sid in self._cache:
+try:
+del self._cache[session.sid]
+except Exception:
+pass
 
-self._cache[session.sid] = session
+self._cache[session.sid] = session
 self._normalize()
 
 

Fix for debugger issue with EPAS package procedures/functions INOUT params #RM3191

[Harshal Dhumal]

Hi,

Please find attached patch to fix below debugger issues.

1. Allow debugging of EPAS package procedures/functions with INOUT params.
2. Add Support for indirect debugging for EPAS package procedures/functions.
3. Allow debugging with NULL param values.


Thanks,

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/tools/debugger/static/js/debugger.js b/web/pgadmin/tools/debugger/static/js/debugger.js
index 39caca7..848737c 100644
--- a/web/pgadmin/tools/debugger/static/js/debugger.js
+++ b/web/pgadmin/tools/debugger/static/js/debugger.js
@@ -354,6 +354,24 @@ define([
 'func_id': debuggerUtils.getProcedureId(treeInfo),
   }
 );
+  } else if (d._type == 'edbfunc') {
+// Get the existing function parameters available from sqlite database
+baseUrl = url_for('debugger.initialize_target_for_function', {
+  'debug_type': 'indirect',
+  'sid': treeInfo.server._id,
+  'did': treeInfo.database._id,
+  'scid': treeInfo.schema._id,
+  'func_id': treeInfo.edbfunc._id,
+});
+  } else if (d._type == 'edbproc') {
+// Get the existing function parameters available from sqlite database
+baseUrl = url_for('debugger.initialize_target_for_function', {
+  'debug_type': 'indirect',
+  'sid': treeInfo.server._id,
+  'did': treeInfo.database._id,
+  'scid': treeInfo.schema._id,
+  'func_id': treeInfo.edbproc._id,
+});
   } else if (d._type == 'trigger_function') {
 baseUrl = url_for(
   'debugger.initialize_target_for_function', {
@@ -449,7 +467,8 @@ define([
 i = item || t.selected(),
 d = i && i.length == 1 ? t.itemData(i) : undefined,
 node = d && pgBrowser.Nodes[d._type],
-self = this;
+self = this,
+is_edb_proc = d._type == 'edbproc';
 
   if (!d)
 return;
@@ -465,7 +484,7 @@ define([
 
 // Open Alertify the dialog to take the input arguments from user if function having input arguments
 if (res.data[0]['require_input']) {
-  get_function_arguments(res.data[0], 0);
+  get_function_arguments(res.data[0], 0, is_edb_proc);
 } else {
   // Initialize the target and create asynchronous connection and unique transaction ID
   // If there is no arguments to the functions then we should not ask for for function arguments and
diff --git a/web/pgadmin/tools/debugger/static/js/debugger_ui.js b/web/pgadmin/tools/debugger/static/js/debugger_ui.js
index 6f6fad5..38f6869 100644
--- a/web/pgadmin/tools/debugger/static/js/debugger_ui.js
+++ b/web/pgadmin/tools/debugger/static/js/debugger_ui.js
@@ -120,14 +120,17 @@ define([
 }
   };
 
-  var res = function(args, restart_debug) {
+  var res = function(debug_info, restart_debug, is_edb_proc) {
 if (!Alertify.debuggerInputArgsDialog) {
   Alertify.dialog('debuggerInputArgsDialog', function factory() {
 return {
-  main: function(title, data, restart_debug) {
+  main: function(title, debug_info, restart_debug, is_edb_proc) {
 this.set('title', title);
-this.data = data;
-this.restart_debug = restart_debug;
+
+// setting value in alertify settings allows us to access it from
+// other functions other than main function.
+this.set('debug_info', debug_info);
+this.set('restart_debug', restart_debug);
 
 // Variables to store the data sent from sqlite database
 var func_args_data = this.func_args_data = [];
@@ -182,10 +185,10 @@ define([
 } else {
   // Get the existing function parameters available from sqlite database
   _Url = url_for('debugger.get_arguments', {
-'sid': this.data.server_id,
-'did': this.data.database_id,
-'scid': this.data.schema_id,
-'func_id': this.data.function_id,
+'sid': debug_info.server_id,
+'did': debug_info.database_id,
+'scid': debug_info.schema_id,
+'func_id': debug_info.function_id,
   });
 }
 $.ajax({
@@ -279,60 +282,63 @@ define([
 // Below will calculate the input argument id required to store in sqlite database
 var input_arg_id = this.input_arg_id = [],
   k;
-if (this.data['proargmodes'] != null) {
- 

Re: [pgAdmin4][Patch]: RM #3551 pgAdmin4 doesn't handle \'s in data fields correctly

[Harshal Dhumal]

Hi Akshay,

We can do one minor improvement for python 2. Instead of below check
*if type(val) in (str, unicode) :*
we can check val against *basestring
<https://docs.python.org/2/library/functions.html#basestring>*

Thanks,
-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


On Thu, Sep 20, 2018 at 8:16 PM Akshay Joshi 
wrote:

> Hi Hackers,
>
> Attached is the modified patch with regression tests.
>
> On Wed, Sep 19, 2018 at 11:32 AM, Akshay Joshi <
> akshay.jo...@enterprisedb.com> wrote:
>
>> Hi Dave
>>
>> On Fri, Sep 14, 2018 at 8:02 PM, Dave Page  wrote:
>>
>>> Hi
>>>
>>> On Fri, Sep 14, 2018 at 12:48 PM, Akshay Joshi <
>>> akshay.jo...@enterprisedb.com> wrote:
>>>
>>>> Hi Hackers,
>>>>
>>>> Attached is the patch to fix RM #3551 pgAdmin4 doesn't handle \'s in
>>>> data fields correctly.
>>>>
>>>
>>> This doesn't work for me. If I add a new row and enter:
>>>
>>> \\server\path\to\file
>>>
>>> as a value, I get the following on the Messages tab:
>>>
>>
>> This works fine for me, I have enter the same value and data is
>> stored. I have tested it with character varying column. Though I have found
>> some other issues as well. If I fix one issue then another will get
>> introduce, so not concluded yet how to fix this and currently working on
>> it.
>>
>>>
>>>  500
>>> Internal Server Error Internal Server Error The server
>>> encountered an internal error and was unable to complete your request.
>>> Either the server is overloaded or there is an error in the application.
>>>
>>> And this on the Python (3.6) console:
>>>
>>> 2018-09-14 15:30:28,516: INFO werkzeug: 127.0.0.1 - - [14/Sep/2018
>>> 15:30:28] "POST /sqleditor/save/4073406 HTTP/1.1" 500 -
>>> 2018-09-14 15:30:28,522: ERROR werkzeug: Error on request:
>>> Traceback (most recent call last):
>>>   File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/werkzeug/serving.py",
>>> line 270, in run_wsgi
>>> execute(self.server.app)
>>>   File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/werkzeug/serving.py",
>>> line 258, in execute
>>> application_iter = app(environ, start_response)
>>>   File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/flask/app.py",
>>> line 1997, in __call__
>>> return self.wsgi_app(environ, start_response)
>>>   File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/flask/app.py",
>>> line 1985, in wsgi_app
>>> response = self.handle_exception(e)
>>>   File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/flask/app.py",
>>> line 1540, in handle_exception
>>> reraise(exc_type, exc_value, tb)
>>>   File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/flask/_compat.py",
>>> line 33, in reraise
>>> raise value
>>>   File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/flask/app.py",
>>> line 1982, in wsgi_app
>>> response = self.full_dispatch_request()
>>>   File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/flask/app.py",
>>> line 1614, in full_dispatch_request
>>> rv = self.handle_user_exception(e)
>>>   File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/flask/app.py",
>>> line 1517, in handle_user_exception
>>> reraise(exc_type, exc_value, tb)
>>>   File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/flask/_compat.py",
>>> line 33, in reraise
>>> raise value
>>>   File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/flask/app.py",
>>> line 1612, in full_dispatch_request
>>> rv = self.dispatch_request()
>>>   File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/flask/app.py",
>>> line 1598, in dispatch_request
>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>   File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/flask_login.py",
>>> line 792, in deco

[pgAdmin4][Patch] RM #3700 fix connection garbage collector

[Harshal Dhumal]

Hi,

This issues caused as we were iterating over manager keys (which contains
server ids) and
not on server managers.

Attached patch fixes this issue.


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/utils/driver/psycopg2/__init__.py b/web/pgadmin/utils/driver/psycopg2/__init__.py
index 5fefdee..3fc8731 100644
--- a/web/pgadmin/utils/driver/psycopg2/__init__.py
+++ b/web/pgadmin/utils/driver/psycopg2/__init__.py
@@ -200,10 +200,9 @@ class Driver(BaseDriver):
 if sess == session.sid:
 sess_mgr['pinged'] = curr_time
 continue
-
 if curr_time - sess_mgr['pinged'] >= session_idle_timeout:
 for mgr in [
-m for m in sess_mgr if isinstance(m, ServerManager)
+m for m in sess_mgr.values() if isinstance(m, ServerManager)
 ]:
 mgr.release()
 

[pgAdmin4][RM3703] Clear user connections on logout

[Harshal Dhumal]

Hi,

Please find patch to clear user connections on logout.

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/__init__.py b/web/pgadmin/__init__.py
index 9b2cf35..36dfee3 100644
--- a/web/pgadmin/__init__.py
+++ b/web/pgadmin/__init__.py
@@ -550,6 +550,13 @@ def create_app(app_name=None):
 def force_session_write(app, user):
 session.force_write = True
 
+@user_logged_out.connect_via(app)
+def clear_current_user_connections(app, user):
+from config import PG_DEFAULT_DRIVER
+from pgadmin.utils.driver import get_driver
+_driver = get_driver(PG_DEFAULT_DRIVER)
+_driver.gc_own()
+
 ##
 # Load plugin modules
 ##
diff --git a/web/pgadmin/utils/driver/psycopg2/__init__.py b/web/pgadmin/utils/driver/psycopg2/__init__.py
index 5fefdee..f9dfd17 100644
--- a/web/pgadmin/utils/driver/psycopg2/__init__.py
+++ b/web/pgadmin/utils/driver/psycopg2/__init__.py
@@ -207,6 +207,22 @@ class Driver(BaseDriver):
 ]:
 mgr.release()
 
+def gc_own(self):
+"""
+Release the connections for current session
+This is useful when (eg. logout) we want to release all
+connections (except dedicated connections created by utilities
+like backup, restore etc) of all servers for current user.
+"""
+
+sess_mgr = self.managers.get(session.sid, None)
+
+if sess_mgr:
+for mgr in (
+m for m in sess_mgr.values() if isinstance(m, ServerManager)
+):
+mgr.release()
+
 @staticmethod
 def qtLiteral(value):
 adapted = adapt(value)

[pgAdmin4] [RM3862] Fix Dialog tabset keyboard navigation

[Harshal Dhumal]

Hi,
This patch fixes Dialog tabset keyboard navigation.
This regression was caused due to bootstrap 4 changes.
Also I have added jasmine test cases for the same


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/static/js/dialog_tab_navigator.js b/web/pgadmin/static/js/dialog_tab_navigator.js
index 19b2045..ea5fec3 100644
--- a/web/pgadmin/static/js/dialog_tab_navigator.js
+++ b/web/pgadmin/static/js/dialog_tab_navigator.js
@@ -86,7 +86,7 @@ class dialogTabNavigator {
 var self = this,
   nextTabPane,
   innerTabContainer,
-  prevtab = $(tabs).find('li.active').prev('li');
+  prevtab = $(tabs).find('li').has('a.active').prev('li');
 
 if (prevtab.length > 0) {
   prevtab.find('a').tab('show');
@@ -116,7 +116,7 @@ class dialogTabNavigator {
 var self = this,
   nextTabPane,
   innerTabContainer,
-  nexttab = $(tabs).find('li.active').next('li');
+  nexttab = $(tabs).find('li').has('a.active').next('li');
 
 if(nexttab.length > 0) {
   nexttab.find('a').tab('show');
@@ -149,4 +149,4 @@ class dialogTabNavigator {
 
 module.exports = {
   dialogTabNavigator: dialogTabNavigator,
-};
\ No newline at end of file
+};
diff --git a/web/regression/javascript/dialog_tab_navigator_spec.js b/web/regression/javascript/dialog_tab_navigator_spec.js
index f355e88..5c46996 100644
--- a/web/regression/javascript/dialog_tab_navigator_spec.js
+++ b/web/regression/javascript/dialog_tab_navigator_spec.js
@@ -14,10 +14,10 @@ describe('dialogTabNavigator', function () {
   let dialog, tabNavigator, backward_shortcut, forward_shortcut;
 
   beforeEach(() => {
-let dialogHtml =$(''+
+let dialogHtml = $(''+
 '   '+
-'  '+
-'  General'+
+'  '+
+'  General'+
 '  '+
 ' '+
 '  Default Privileges'+
@@ -112,4 +112,93 @@ describe('dialogTabNavigator', function () {
 
   });
 
-});
\ No newline at end of file
+
+  describe('navigateForward from fist tab to second tab', function () {
+var navigateForwardResult;
+beforeEach(() => {
+  spyOn(tabNavigator, 'navigateForward').and.callThrough();
+
+  navigateForwardResult = tabNavigator.navigateForward(
+dialog.$el.find('ul.nav-tabs:first'),
+dialog.$el.find('div#1')
+  );
+});
+
+it('should return true', function () {
+
+  expect(navigateForwardResult).toEqual(true);
+
+});
+
+  });
+
+
+  describe('navigateForward from last tab', function () {
+var navigateForwardResult;
+beforeEach(() => {
+
+  // set second tab active
+  dialog.$el.find('ul.nav-tabs li a.active').removeClass('active');
+
+  dialog.$el.find('ul.nav-tabs li a[href="#3"]').addClass('active');
+
+  spyOn(tabNavigator, 'navigateForward').and.callThrough();
+
+  navigateForwardResult = tabNavigator.navigateForward(
+dialog.$el.find('ul.nav-tabs:first'),
+dialog.$el.find('div#1')
+  );
+});
+
+it('should return false', function () {
+
+  expect(navigateForwardResult).toEqual(false);
+
+});
+
+  });
+
+  describe('navigateBackward from second tab to first tab', function () {
+var navigateBackwardResult;
+beforeEach(() => {
+  // set second tab active
+  dialog.$el.find('ul.nav-tabs li a.active').removeClass('active');
+
+  dialog.$el.find('ul.nav-tabs li a[href="#2"]').addClass('active');
+
+  spyOn(tabNavigator, 'navigateBackward').and.callThrough();
+
+  navigateBackwardResult = tabNavigator.navigateBackward(
+dialog.$el.find('ul.nav-tabs:first'),
+dialog.$el.find('div#1')
+  );
+});
+
+it('should return true', function () {
+
+  expect(navigateBackwardResult).toEqual(true);
+
+});
+
+  });
+
+  describe('navigateBackward from first tab', function () {
+var navigateBackwardResult;
+beforeEach(() => {
+  spyOn(tabNavigator, 'navigateBackward').and.callThrough();
+
+  navigateBackwardResult = tabNavigator.navigateBackward(
+dialog.$el.find('ul.nav-tabs:first'),
+dialog.$el.find('div#1')
+  );
+});
+
+it('should return false', function () {
+
+  expect(navigateBackwardResult).toEqual(false);
+
+});
+
+  });
+
+});

Re: [pgAdmin4] [RM3862] Fix Dialog tabset keyboard navigation

[Harshal Dhumal]

Hi,

Please find attached updated patch.
In this patch I have fixed two issues:
i. Dialog tab navigation should work even if focus is on footer buttons
(Save, Cancel, etc..)
ii. Focus should be set to first editable element of dialog when tab cycle
goes through all editable footer buttons.


-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


On Thu, Jan 10, 2019 at 1:16 PM Harshal Dhumal <
harshal.dhu...@enterprisedb.com> wrote:

> Hi,
> This patch fixes Dialog tabset keyboard navigation.
> This regression was caused due to bootstrap 4 changes.
> Also I have added jasmine test cases for the same
>
>
> --
> *Harshal Dhumal*
> *Sr. Software Engineer*
>
> EnterpriseDB India: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
diff --git a/web/pgadmin/browser/static/js/keyboard.js b/web/pgadmin/browser/static/js/keyboard.js
index 79266bf..4d9b6ae 100644
--- a/web/pgadmin/browser/static/js/keyboard.js
+++ b/web/pgadmin/browser/static/js/keyboard.js
@@ -349,11 +349,11 @@ _.extend(pgBrowser.keyboardNavigation, {
   d: selectedTreeNodeData,
 };
   },
-  getDialogTabNavigator: function(dialog) {
+  getDialogTabNavigator: function(dialogContainer) {
 const backward_shortcut = pgBrowser.get_preference('browser', 'dialog_tab_backward').value;
 const forward_shortcut = pgBrowser.get_preference('browser', 'dialog_tab_forward').value;
 
-return new dialogTabNavigator.dialogTabNavigator(dialog, backward_shortcut, forward_shortcut);
+return new dialogTabNavigator.dialogTabNavigator(dialogContainer, backward_shortcut, forward_shortcut);
   },
 });
 
diff --git a/web/pgadmin/browser/static/js/node.js b/web/pgadmin/browser/static/js/node.js
index f5cb8cd..30605dd 100644
--- a/web/pgadmin/browser/static/js/node.js
+++ b/web/pgadmin/browser/static/js/node.js
@@ -412,8 +412,6 @@ define('pgadmin.browser.node', [
 view.render();
 setFocusOnEl();
 newModel.startNewSession();
-// var dialogTabNavigator = pgBrowser.keyboardNavigation.getDialogTabNavigator(view);
-pgBrowser.keyboardNavigation.getDialogTabNavigator(view);
   },
   error: function(xhr, error, message) {
 var _label = that && item ?
@@ -450,8 +448,6 @@ define('pgadmin.browser.node', [
 view.render();
 setFocusOnEl();
 newModel.startNewSession();
-// var dialogTabNavigator = pgBrowser.keyboardNavigation.getDialogTabNavigator(view);
-pgBrowser.keyboardNavigation.getDialogTabNavigator(view);
   }
 }
 
@@ -1083,7 +1079,7 @@ define('pgadmin.browser.node', [
 // All buttons will be created within a single
 // div area.
 var btnGroup =
-  $('').addClass(
+  $('').addClass(
 'pg-prop-btn-group'
   ),
   // Template used for creating a button
@@ -1200,7 +1196,6 @@ define('pgadmin.browser.node', [
 });
   },
 });
-
 createButtons(buttons, 'header', 'pg-prop-btn-group-above');
   }
   j.append(content);
@@ -1392,7 +1387,7 @@ define('pgadmin.browser.node', [
 );
 
 // Create proper buttons
-createButtons([{
+let btn_grp = createButtons([{
   label: '',
   type: 'help',
   tooltip: gettext('SQL help for this object type.'),
@@ -1458,6 +1453,18 @@ define('pgadmin.browser.node', [
 });
   },
 }], 'footer', 'pg-prop-btn-group-below');
+
+btn_grp.on('keydown', 'button', function(event) {
+  if (event.keyCode == 9 && $(this).nextAll('button:not([disabled])').length == 0) {
+// set focus back to first editable input element of current active tab once we cycle through all enabled buttons.
+commonUtils.findAndSetFocus(view.$el.find('.tab-content div.active'));
+return false;
+  }
+});
+
+setTimeout(function() {
+  pgBrowser.keyboardNavigation.getDialogTabNavigator(panel.pgElContainer);
+}, 200);
   }
 
   // Create status bar.
diff --git a/web/pgadmin/static/js/dialog_tab_navigator.js b/web/pgadmin/static/js/dialog_tab_navigator.js
index 19b2045..4472172 100644
--- a/web/pgadmin/static/js/dialog_tab_navigator.js
+++ b/web/pgadmin/static/js/dialog_tab_navigator.js
@@ -13,13 +13,13 @@ import { findAndSetFocus } from './utils';
 import { parseShor

[pgAdmin4][RM3908] backgrid navigation fix for Select2Cell and PrivilegeCell

[Harshal Dhumal]

Hi,
Please find attached patch to fix backgrid navigation using keyboard
for Select2Cell and PrivilegeCell

-- 
*Harshal Dhumal*
*Sr. Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/static/js/column.js b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/static/js/column.js
index 9031d94..b00998b 100644
--- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/static/js/column.js
+++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/static/js/column.js
@@ -269,39 +269,7 @@ define('pgadmin.node.column', [
   type: 'text', disabled: 'notInSchema', mode: ['properties'],
 },{
   id: 'cltype', label: gettext('Data type'),
-  cell: Backgrid.Extension.NodeAjaxOptionsCell.extend({
-exitEditMode: function(e) {
-  var self = this;
-  this.$select.off('blur', this.exitEditMode);
-  this.$select.select2('close');
-  this.$el.removeClass('editor');
-// Once user have selected a value
-// we can shift to next cell if it is editable
-  var next_cell, length_cell = this.$el.next(),
-not_null_cell = this.$el.next().next().next();
-
-// Add delay so that Select2 cell tab event is captured
-// first before triggerring backgrid:edited event.
-  setTimeout(function() {
-  // First check Length column if it is disable then goto
-  // Not Null column
-if(length_cell && length_cell.hasClass('editable') && e) {
-  next_cell = length_cell;
-} else if(not_null_cell && not_null_cell.hasClass('editable') && e) {
-  next_cell = not_null_cell;
-}
-
-if(next_cell) {
-  e.preventDefault();
-  e.stopPropagation();
-  var command = new Backgrid.Command({key: 'Tab', keyCode: 9, which: 9});
-  self.model.trigger('backgrid:edited', self.model, self.column,
-  command);
-  next_cell.trigger('focus');
-}
-  }, 20);
-},
-  }),
+  cell: Backgrid.Extension.NodeAjaxOptionsCell,
   type: 'text', disabled: 'inSchemaWithColumnCheck',
   control: 'node-ajax-options', url: 'get_types', node: 'table',
   cellHeaderClasses:'width_percent_30', first_empty: true,
diff --git a/web/pgadmin/browser/server_groups/servers/static/js/privilege.js b/web/pgadmin/browser/server_groups/servers/static/js/privilege.js
index c5663f7..5476cf8 100644
--- a/web/pgadmin/browser/server_groups/servers/static/js/privilege.js
+++ b/web/pgadmin/browser/server_groups/servers/static/js/privilege.js
@@ -329,13 +329,13 @@ define(['sources/gettext', 'underscore', 'jquery', 'backbone', 'backform',
 '">',
 ' ',
 '  ',
-'   >',
+'   >',
 '   <%- privilege_label %>',
 '  ',
 ' ',
 ' ',
 '  ',
-'<%= enable_with_grant ? "" : \'disabled\'%>>',
+'<%= enable_with_grant ? "" : \'disabled\'%>>',
 '   WITH GRANT OPTION',
 '  ',
 ' ',
@@ -344,6 +344,7 @@ define(['sources/gettext', 'underscore', 'jquery', 'backbone', 'backform',
   events: {
 'change': 'privilegeChanged',
 'blur': 'lostFocus',
+'keydown': 'lostFocus',
   },
 
   render: function () {
@@ -608,7 +609,42 @@ define(['sources/gettext', 'underscore', 'jquery', 'backbone', 'backform',
   node = node.parentNode;
 }
 return false;
-  };
+  },
+  model = this.model,
+  column = this.column,
+  command = new Backgrid.Command(ev),
+  coll = this.model.get(this.column.get('name'));
+
+if (command.moveUp() || command.moveDown() || command.save()) {
+// backgrid vertical navigation (Up/Down arrow key)
+  ev.preventDefault();
+  ev.stopPropagation();
+  model.trigger