sql questions
Hi, a) I am running some select query select ... order by Now, I would like to preserver the ordering through further processing by adding a sequence number Of course I can do: create temp sequence mseq; select xx.*, nextval('mseq') as ord from (select ... order by ) xx; drop sequence mseq; Is there a simpler way (avoiding the create/drop parts) b) can a sql function return the count of affected rows of some query? create function merge_names(int, int) returns void as $_$ update namelinks set nid = $2 where nid = $1; -- want the affected rows of the above query delete from names where nid = $1 -- return result here $_$ language sql; Best regards Wolfgang Hamann
Re: sql questions
haman...@t-online.de wrote: > a) I am running some select query > select ... order by > Now, I would like to preserver the ordering through further processing by > adding a sequence number > Of course I can do: > create temp sequence mseq; > select xx.*, nextval('mseq') as ord from (select ... order by ) xx; > drop sequence mseq; > Is there a simpler way (avoiding the create/drop parts) A window function would be the best thing: SELECT ..., row_number() OVER (ORDER BY ...) FROM ... > b) can a sql function return the count of affected rows of some query? > create function merge_names(int, int) returns void as > $_$ > update namelinks set nid = $2 where nid = $1; > -- want the affected rows of the above query > delete from names where nid = $1 > -- return result here > $_$ > language sql; You cannot do it in an SQL function. In PL/pgSQL you can use GET DIAGNOSTICS avariable = ROW_COUNT; Yours, Laurenz Albe -- Cybertec | https://www.cybertec-postgresql.com
Re: sql questions
Zitat von haman...@t-online.de: a) I am running some select query select ... order by Now, I would like to preserver the ordering through further processing by adding a sequence number Of course I can do: create temp sequence mseq; select xx.*, nextval('mseq') as ord from (select ... order by ) xx; drop sequence mseq; Is there a simpler way (avoiding the create/drop parts) Can't you just do the ordering at the end of the processing? Maybe you need to drag along the order by columns and just dump them at the very end if applicable. This message was sent using IMP, the Internet Messaging Program. binLSH7ZizYEL.bin Description: PGP Public Key
Re: sql questions
On Fri, Jul 20, 2018 at 4:27 AM wrote: > > b) can a sql function return the count of affected rows of some query? > create function merge_names(int, int) returns void as > $_$ > update namelinks set nid = $2 where nid = $1; > -- want the affected rows of the above query > delete from names where nid = $1 > -- return result here > $_$ > language sql; > > Yes. You can do this in pure SQL by using CTEs like the following example. with myupdate as ( update test set a = 4 where a = 1 returning a ), mydelete as ( delete from testnames where nid = 1 ) select count(1) from myupdate; You can then just wrap a function around this. Full test case below. -- Create test tables create table test ( a integer ); insert into test values (1),(1),(3); create table testnames ( nid integer ); insert into testnames values (1); -- Update, delete, and return the number of updates in a single statement create function test_names(integer, integer) returns bigint as $_$ with myupdate as ( update test set a = $2 where a = $1 returning a ), mydelete as ( delete from testnames where nid = $1 ) select count(1) from myupdate $_$ language sql; -- Run it # select test_names(1,4); test_names 2 (1 row) -- Verify results =# select * from test; a --- 3 4 4 (3 rows) =# select * from testnames; nid - (0 rows)
Re: User documentation vs Official Docs
On Thu, 19 Jul 2018 21:02:16 -0400, Melvin Davidson wrote: >As universities DO NOT ALLOW software to be installed on shared computers, >and this is the case especially in a library, it implies the user has >their own computer. Many (most?) universities do allow students to install and run software locally under their own accounts. Of course, that doesn't help a visitor using a lab or library computer. >As libraries allow users/citizens to request books be purchased at no >cost to the user/citizen, the argument that someone cannot afford a book >is now a moot point. Libraries can't afford to purchase everything anyone might want. However, libraries lend to one another as well as to their patrons. You always can ask your library to borrow the book from some other library that does have it. Most libraries belong to one or more inter-library lending networks. [Of course, there is no telling how long it will take to get a book that way - you may wait months for something that's hard to find, or if you happen to be far from a decent sized city.] Another nice thing: often when the book needs to be ordered from another library, you can keep it checked out longer than if it came from the local collection. E.g., my local library allows (most) books to be checked out for up to 2 weeks. Specially ordered books, however, can be checked out for up to 5 weeks. These times may not be typical, but wherever I have been, the libraries have allowed for keeping specially ordered books longer. YMMV, George
Re: User documentation vs Official Docs
On 07/20/2018 10:38 AM, George Neuner wrote: As libraries allow users/citizens to request books be purchased at no cost to the user/citizen, the argument that someone cannot afford a book is now a moot point. This thread is getting off topic. The tl;dr; of this particular subthread is that we are not here just for the relatively rich Western World, students or not (although I certainly appreciate that pain). We are an International community with varying levels of financial capabilities. If we want to enable the *entire* community to succeed with PostgreSQL we have to have resources that are Free (as in Beer and Software). Back to the original idea, it would be great if those participating would be willing to help even a little in determining an actual direction to take this. Thanks, JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc *** A fault and talent of mine is to tell it exactly how it is. *** PostgreSQL centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://postgresconf.org * Unless otherwise stated, opinions are my own. *
Re: User documentation vs Official Docs
On 07/20/2018 11:45 AM, Joshua D. Drake wrote: On 07/20/2018 10:38 AM, George Neuner wrote: As libraries allow users/citizens to request books be purchased at no cost to the user/citizen, the argument that someone cannot afford a book is now a moot point. This thread is getting off topic. The tl;dr; of this particular subthread is that we are not here just for the relatively rich Western World, students or not (although I certainly appreciate that pain). We are an International community with varying levels of financial capabilities. If we want to enable the *entire* community to succeed with PostgreSQL we have to have resources that are Free (as in Beer and Software). Back to the original idea, it would be great if those participating would be willing to help even a little in determining an actual direction to take this. I would say that discussion should take place in --docs: https://www.postgresql.org/list/pgsql-docs/ Thanks, JD -- Adrian Klaver adrian.kla...@aklaver.com
Postgres function with output parameters and resultset
Hi, I need to create a function which should return resultset and output parameters. For example, I need to retrieve all the records from EMP table whose Grade is 'A' as resultset and total number of matched records, Success or Failure flag & Error message as output parameters. Is there a way in Postgres to achieve this? Kindly help!! CREATE OR REPLACE FUNCTION TESTFN (IN GRADE CHARACTER(01) ,OUT EMP_CNT INT ,OUT SUCCESS_FG CHARACTER(01) ,OUT SQLCD CHARACTER(05) ,OUT ERROR_MSG CHARACTER(10)) RETURNS RECORD AS $$ DECLARE REFCUR REFCURSOR; BEGIN IF (GRADE IS NULL OR GRADE = '') THEN SUCCESS_FG := 'E'; SQLCD := ''; ERROR_MSG := 'GRADE IS NULL OR BLANK'; RETURN; END IF; EMP_CNT := SELECT COUNT(*) FROM EMP WHERE GRADE_CD = GRADE; EXCEPTION WHEN OTHERS THEN SUCCESS_FG := 'E'; SQLCD := SQLSTATE; ERROR_MSG := 'SP EXECUTION ABORTED 1'; RETURN; OPEN REFCUR FOR SELECT NAME, AGE, BRANCH, TITLE FROM EMP WHERE GRADE_CD = GRADE; EXCEPTION WHEN OTHERS THEN SUCCESS_FG := 'E'; SQLCD := SQLSTATE; ERROR_MSG := 'SP EXECUTION ABORTED 2'; RETURN; RETURN REFCUR; END; $$ LANGUAGE plpgsql;
Re: Postgres function with output parameters and resultset
On Friday, July 20, 2018, Arulalan Narayanasamy < arulalan.narayanas...@gmail.com> wrote: > Hi, > I need to create a function which should return resultset and output > parameters. For example, I need to retrieve all the records from EMP table > whose Grade is 'A' as resultset and total number of matched records, > Success or Failure flag & Error message as output parameters. Is there a > way in Postgres to achieve this? Kindly help!! > A function can return a single two dimensional table - so, not directly. Though what you describe here seems like over-engineering. If you really want the count you'd need to add it to the table but the client can count the records in the table easily enough. Errors can be done separately via RAISE and likewise let the client deal with that as usual. Otherwise I've found JSON to be useful for non-tabular results. David J.
Re: Postgres function with output parameters and resultset
On 07/20/2018 02:19 PM, Arulalan Narayanasamy wrote: Hi, I need to create a function which should return resultset and output parameters. For example, I need to retrieve all the records from EMP table whose Grade is 'A' as resultset and total number of matched records, Success or Failure flag & Error message as output parameters. Is there a way in Postgres to achieve this? Kindly help!! Just thinking out loud. Wonder if you could use the multiple cursor example shown just above here: https://www.postgresql.org/docs/10/static/plpgsql-cursors.html#PLPGSQL-CURSOR-FOR-LOOP Use the one cursor for the result set(NOTE: I did not see where this was restricted to 'A' grade) This is the part I am not sure of. Still here it goes: 1) Create temp table for the message values. 2) Insert the message values in this table. 3) Open a cursor over the table and return it. CREATE OR REPLACE FUNCTION TESTFN (IN GRADE CHARACTER(01) ,OUT EMP_CNT INT ,OUT SUCCESS_FG CHARACTER(01) ,OUT SQLCD CHARACTER(05) ,OUT ERROR_MSG CHARACTER(10)) RETURNS RECORD AS $$ DECLARE REFCUR REFCURSOR; BEGIN IF (GRADE IS NULL OR GRADE = '') THEN SUCCESS_FG := 'E'; SQLCD:= ''; ERROR_MSG := 'GRADE IS NULL OR BLANK'; RETURN; END IF; EMP_CNT := SELECT COUNT(*) FROM EMP WHERE GRADE_CD = GRADE; EXCEPTION WHEN OTHERS THEN SUCCESS_FG := 'E'; SQLCD:= SQLSTATE; ERROR_MSG := 'SP EXECUTION ABORTED 1'; RETURN; OPEN REFCUR FOR SELECT NAME, AGE, BRANCH, TITLE FROM EMP WHERE GRADE_CD = GRADE; EXCEPTION WHEN OTHERS THEN SUCCESS_FG := 'E'; SQLCD:= SQLSTATE; ERROR_MSG := 'SP EXECUTION ABORTED 2'; RETURN; RETURN REFCUR; END; $$ LANGUAGE plpgsql; -- Adrian Klaver adrian.kla...@aklaver.com
Re: User documentation vs Official Docs
On 2018-Jul-20, Adrian Klaver wrote: > On 07/20/2018 11:45 AM, Joshua D. Drake wrote: > > Back to the original idea, it would be great if those participating > > would be willing to help even a little in determining an actual > > direction to take this. > > I would say that discussion should take place in --docs: > > https://www.postgresql.org/list/pgsql-docs/ I don't see why we need this thread to continue. This sounds like somebody looking for a solution when they don't yet know what the problem is. If people want to contribute, there are already some places where they can do so. Articles can be drafted in the wiki initially or, heck, even sites like StackOverflow[1], and if something gets to a level so great that they think it should be enshrined in DocBook, they can turn it into a documentation patch. [1] for extra points, write in SO and then add a link to the question to FAQ in the wiki. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: User documentation vs Official Docs
On Fri, Jul 20, 2018 at 6:59 PM, Alvaro Herrera wrote: > On 2018-Jul-20, Adrian Klaver wrote: > > > On 07/20/2018 11:45 AM, Joshua D. Drake wrote: > > > > Back to the original idea, it would be great if those participating > > > would be willing to help even a little in determining an actual > > > direction to take this. > > > > I would say that discussion should take place in --docs: > > > > https://www.postgresql.org/list/pgsql-docs/ > > I don't see why we need this thread to continue. This sounds like > somebody looking for a solution when they don't yet know what the > problem is. > > If people want to contribute, there are already some places where they > can do so. Articles can be drafted in the wiki initially or, heck, even > sites like StackOverflow[1], and if something gets to a level so great > that they think it should be enshrined in DocBook, they can turn it into > a documentation patch. > > [1] for extra points, write in SO and then add a link to the question to > FAQ in the wiki. > > -- > Álvaro Herrerahttps://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services > > > This sounds like somebody looking for a solution when they don't yet know what the problem is. +1 -- *Melvin Davidson* *Maj. Database & Exploration Specialist* *Universe Exploration Command – UXC* Employment by invitation only!
In certain cases, can UPDATE transactions fail rather than block waiting for “FOR UPDATE lock”?
I'm using Postgres 9.6.5. In the docs under-- [13.3. Explicit Locking][ https://www.postgresql.org/docs/9.6/static/explicit-locking.html] "13.3.2. Row-level Locks" -> "Row-level Lock Modes" -> "FOR UPDATE": ''' FOR UPDATE causes the rows retrieved by the SELECT statement to be locked as though for update. This prevents them from being locked, modified or deleted by other transactions until the current transaction ends. That is, other transactions that attempt UPDATE, DELETE, SELECT FOR UPDATE, SELECT FOR NO KEY UPDATE, SELECT FOR SHARE or SELECT FOR KEY SHARE of these rows will be blocked until the current transaction ends; conversely, SELECT FOR UPDATE will wait for a concurrent transaction that has run any of those commands on the same row, and will then lock and return the updated row (or no row, if the row was deleted). ... The mode is also acquired by any DELETE on a row, and also by an UPDATE that modifies the values on certain columns. Currently, the set of columns considered for the UPDATE case are those that have a unique index on them that can be used in a foreign key (so partial indexes and expressional indexes are not considered), but this may change in the future. ''' Regarding UPDATEs on rows that are locked via "SELECT FOR UPDATE" in another transaction, I read the above as follows: other transactions that attempt UPDATE of these rows will be blocked until the current transaction ( which did "SELECT FOR UPDATE" for those rows ) ends, unless the columns in these rows being UPDATE'ed are those that don't have a unique index on them that can be used in a foreign key. Is this correct ? If so, if I have a table "program" with a text column "stage" ( this column doesn't fit "have a unique index on them that can be used in a foreign key" ), and I have a transaction that does "SELECT FOR UPDATE" for some rows followed by UPDATE'ing "stage" in these rows, is it correct that other concurrent transactions doing "UPDATE" on these rows can fail, rather than block until the former transaction ends ? pk
Re: User documentation vs Official Docs
On 07/20/2018 03:59 PM, Alvaro Herrera wrote: I don't see why we need this thread to continue. This sounds like somebody looking for a solution when they don't yet know what the problem is. Unfortunately, you don't understand the problem which is why this thread is happening on -general and not -hackers. The problem is simple as illustrated, our user documentation is less than stellar for those trying to solve specific problems. This isn't a new problem nor is it one that is not communicated. I hear the issue from users ever single time I speak or attend a conference/meetup. I was hoping to get the -general community to step and build some recipes and howto articles without at the same time dictating the solution. That's a good thing because a non-dictated solution is likely to have more strength. The wiki is a terrible choice but if that is where the community thinks it should go, I welcome people starting to contribute in a structured fashion to the wiki. I hope it works out well. JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc *** A fault and talent of mine is to tell it exactly how it is. *** PostgreSQL centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://postgresconf.org * Unless otherwise stated, opinions are my own. *
Re: User documentation vs Official Docs
Greetings, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > I don't see why we need this thread to continue. This sounds like > somebody looking for a solution when they don't yet know what the > problem is. > > If people want to contribute, there are already some places where they > can do so. Articles can be drafted in the wiki initially or, heck, even > sites like StackOverflow[1], and if something gets to a level so great > that they think it should be enshrined in DocBook, they can turn it into > a documentation patch. +1. I'd personally like to see improvements to the tutorials, and patches could certainly be submitted or specific ideas discussed over on -docs. A few ideas around that would be: - Setting up async replication - Setting up sync replication, with quorum-based sync - Cascading replication - Parallel pg_dump-based backup/restore (with pg_dumpall for globals) - Using various important extensions (pg_stat_statements, pg_buffercache, pageinspect, pg_freespacemap, pg_visibility) - Using pg_basebackup to build replicas - Using pg_receivewal to have a WAL archive Of course, there's a lot of additional tutorials that would be nice to have which go beyond what's in core and leverage tools like pgbouncer, pgbackrest, patroni, etc, but they'd go on the wiki or elsewhere since they would be necessairly referring to bits that are outside of PG core. Thanks! Stephen signature.asc Description: PGP signature
Re: User documentation vs Official Docs
On 07/20/2018 05:48 PM, Joshua D. Drake wrote: On 07/20/2018 03:59 PM, Alvaro Herrera wrote: I don't see why we need this thread to continue. This sounds like somebody looking for a solution when they don't yet know what the problem is. Unfortunately, you don't understand the problem which is why this thread is happening on -general and not -hackers. The problem is simple as illustrated, our user documentation is less than stellar for those trying to solve specific problems. This isn't a new problem nor is it one that is not communicated. I hear the issue from users ever single time I speak or attend a conference/meetup. I was hoping to get the -general community to step and build some recipes and howto articles without at the same time dictating the solution. That's a good thing because a non-dictated solution is likely to have more strength. The wiki is a terrible choice but if that is where the community thinks it should go, I welcome people starting to contribute in a structured fashion to the wiki. I hope it works out well. JD I trust you took notes of specific things user were seeking. Lay those out, here or else where, and see if anyone steps up to answer something specific. The majority of the audience here (me seriously not included) is NOT looking for how-tos. Generally given the chance on this list, members of that majority step in and provide very specific answers to specific questions (and sometime general notions are addressed as well). They, that majority, simply don't know what isn't obvious - until it isn't to someone.
Re: User documentation vs Official Docs
On Friday, July 20, 2018, Joshua D. Drake wrote: > > I was hoping to get the -general community to step and build some recipes > and howto articles without at the same time dictating the solution. That's > a good thing because a non-dictated solution is likely to have more > strength. > People have chosen to solve this via books and writing applications that build onto the infrastructure PostgreSQL provides (in particular in the area of backups). There is room for making others' lives easier in a more structured way but that takes time - which if you limit any acceptable solution to "free as in beer" is going to likely result in status quo (publish stuff however each person wishes and let people find it via search engine or as a result of asking questions on -general or SO. We don't have to be everything to everyone and free to boot. Writing up documentation and guides to answer specific questions that are posed and lack answers elsewhere (or in a desireable format) is something that will have a good chance to garner a concrete positive result. This thread, at this point and IMO, has served its purpose - to remind others that we have a possible gap in our accessibility. That goal seems largely accomplished and anyone wanting to discuss specific thoughts for addressing this please post an appropriately subject line message to -general with those concrete thoughts. Otherwise we now have a good feel for current reality and can find resources and then point people to those that exist today next time questions come up (or have them asked on -general). David J.
Re: User documentation vs Official Docs
On 07/20/2018 04:48 PM, Joshua D. Drake wrote: On 07/20/2018 03:59 PM, Alvaro Herrera wrote: I don't see why we need this thread to continue. This sounds like somebody looking for a solution when they don't yet know what the problem is. Unfortunately, you don't understand the problem which is why this thread is happening on -general and not -hackers. The problem is simple as illustrated, our user documentation is less than stellar for those trying to solve specific problems. This isn't a new problem nor is it one that is not communicated. I hear the issue from users ever single time I speak or attend a conference/meetup. JD sit down, I am going to agree with you:) The documentation as it stands is very good, though it requires some fore knowledge to successfully navigate. On pages with a lot of content it often is not evident, to many, that there are actually examples at the bottom of the page. Also that the exceptions to the rules are called out there also. The general concept of presenting a task, writing a procedure to accomplish the task and pointing to the documentation that covers the procedure would be a helpful addition. It would be nice to point to something like that in a post rather then continually rebuilding the explanation every time a new user hits the list. Looking at the link posted upstream: https://www.postgresql.org/docs/10/static/tutorial-start.html shows a start in that direction. To me this would be the place for folks to contribute. I personally would not have a problem including non-core sections as well. We deal with that on --general all the time, so it would seem to be fair game. I was hoping to get the -general community to step and build some recipes and howto articles without at the same time dictating the solution. That's a good thing because a non-dictated solution is likely to have more strength. The wiki is a terrible choice but if that is where the community thinks it should go, I welcome people starting to contribute in a structured fashion to the wiki. I hope it works out well. JD -- Adrian Klaver adrian.kla...@aklaver.com
Re: User documentation vs Official Docs
On 07/20/2018 04:56 PM, Stephen Frost wrote: +1. I'd personally like to see improvements to the tutorials, and patches could certainly be submitted or specific ideas discussed over on -docs. A few ideas around that would be: - Setting up async replication - Setting up sync replication, with quorum-based sync - Cascading replication - Parallel pg_dump-based backup/restore (with pg_dumpall for globals) - Using various important extensions (pg_stat_statements, pg_buffercache, pageinspect, pg_freespacemap, pg_visibility) - Using pg_basebackup to build replicas - Using pg_receivewal to have a WAL archive I think this is a pretty good list. I would add: Practical Role management JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc *** A fault and talent of mine is to tell it exactly how it is. *** PostgreSQL centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://postgresconf.org * Unless otherwise stated, opinions are my own. *
Re: User documentation vs Official Docs
On 07/20/2018 05:31 PM, Adrian Klaver wrote: On 07/20/2018 04:48 PM, Joshua D. Drake wrote: On 07/20/2018 03:59 PM, Alvaro Herrera wrote: I don't see why we need this thread to continue. This sounds like somebody looking for a solution when they don't yet know what the problem is. Unfortunately, you don't understand the problem which is why this thread is happening on -general and not -hackers. The problem is simple as illustrated, our user documentation is less than stellar for those trying to solve specific problems. This isn't a new problem nor is it one that is not communicated. I hear the issue from users ever single time I speak or attend a conference/meetup. JD sit down, I am going to agree with you:) Hey it happens once every 18 months or so ;) JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc *** A fault and talent of mine is to tell it exactly how it is. *** PostgreSQL centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://postgresconf.org * Unless otherwise stated, opinions are my own. *
Re: sql questions
>> Zitat von haman...@t-online.de: >> >> > a) I am running some select query >> > select ... order by >> > Now, I would like to preserver the ordering through further >> > processing by adding a sequence number >> > Of course I can do: >> > create temp sequence mseq; >> > select xx.*, nextval('mseq') as ord from (select ... order by ) xx; >> > drop sequence mseq; >> > Is there a simpler way (avoiding the create/drop parts) >> >> Can't you just do the ordering at the end of the processing? Maybe you >> need to drag along the order by columns and just dump them at the very >> end if applicable. >> Hi, in this specific case every search result consists of a pair of related entries that are not close to each other in ordering. So I order by first entry and use the row number to keep the second entry next to the first one, BTW: the use case is scanning a database of people for duplicates. Whenever there are 3 or more components in a name, the split betwwen first and last name can be ambiguous, and so its is common to find both "Ludwig" "van Beethoven" and "Ludwig van" "Beethoven" Best regards WOlfgang
Re: In certain cases, can UPDATE transactions fail rather than block waiting for “FOR UPDATE lock”?
On 2018-Jul-20, Praveen Kumar wrote: > Regarding UPDATEs on rows that are locked via "SELECT FOR UPDATE" in > another transaction, I read the above as follows: other transactions that > attempt UPDATE of these rows will be blocked until the current transaction > ( which did "SELECT FOR UPDATE" for those rows ) ends, unless the columns > in these rows being UPDATE'ed are those that don't have a unique index on > them that can be used in a foreign key. > > Is this correct ? No. What it means that if you UPDATE the columns-indexed-by-unique-idx then the FOR UPDATE lock is acquired underneath. If your UPDATE modifies some other column, then a FOR NO KEY UPDATE lock is acquired instead. In both cases, concurrent transactions would be blocked rather than erroring out. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services