Re: JSON query

2021-10-18 Thread Scott Ribe
> On Oct 18, 2021, at 10:02 PM, David G. Johnston > wrote: > > (jsonb - text[]) = ‘{}’::jsonb …? Aha, thank you!

JSON query

2021-10-18 Thread Scott Ribe
What's a good way to query jsonb column for "no keys other than those in this list of keys" in other words "containing only keys from this list of keys" -- Scott Ribe scott_r...@elevated-dev.com https://www.linkedin.com/in/scottribe/

Re: export to parquet

2020-08-26 Thread Scott Ribe
> On Aug 26, 2020, at 1:11 PM, Chris Travers wrote: > > For simple exporting, the simplest thing is a single-node instance of Spark. Thanks. > You can read parquet files in Postgres using > https://github.com/adjust/parquet_fdw if you so desire but it does not > support writing as parquet fil

export to parquet

2020-08-26 Thread Scott Ribe
I have no Hadoop, no HDFS. Just looking for the easiest way to export some PG tables into Parquet format for testing--need to determine what kind of space reduction we can get before deciding whether to look into it more. Any suggestions on particular tools? (PG 12, Linux) -- Scott Ribe

Re: privileges oddity

2020-08-07 Thread Scott Ribe
So, one last follow-up, perhaps \du or \du+ should show when a role is mapped that way. If I'd seen a clue to this setting that had been made "before I got here" it would have been figured out sooner. I realize ALTER ROLE... SET... can be used to set many more defaults, and there could be some

Re: privileges oddity

2020-08-07 Thread Scott Ribe
> On Aug 7, 2020, at 1:32 PM, Tom Lane wrote: > > Yes, you are. It looks like what you actually issued is > > ALTER USER akanzler SET role confidential_read_only; > > but that would have the effect that subsequent session starts would > automatically do "SET ROLE confidential_read_only". AHA!

Re: privileges oddity

2020-08-07 Thread Scott Ribe
> On Aug 7, 2020, at 1:08 PM, Adrian Klaver wrote: > > "Using this command, it is possible to either add privileges or restrict > one's privileges. If the session user role has the INHERIT attribute, then it > automatically has all the privileges of every role that it could SET ROLE to; > in t

Re: privileges oddity

2020-08-07 Thread Scott Ribe
> On Aug 7, 2020, at 12:45 PM, Tom Lane wrote: > > If I'm reading this correctly, you have set things up so that any > session logging in as akanzler will immediately do "SET ROLE > confidential_read_only", after which it's the privileges of that > role not akanzler that determine what happens.

Re: privileges oddity

2020-08-07 Thread Scott Ribe
On Aug 7, 2020, at 12:27 PM, Adrian Klaver wrote: > > So what privileges does role 'confidential_read_only' have? read on everything I tried creating a new user without it, just doing the same grants otherwise as for akanzler, that worked. Then I added that user to confidential_read_only, sti

Re: privileges oddity

2020-08-07 Thread Scott Ribe
> On Aug 7, 2020, at 12:17 PM, Adrian Klaver wrote: > > Well if this for the same line as before it represents table privileges. The > problem is with schema access. Continuing grasping at straws: > > select * from pg_roles where rolname = 'aakanzler'; rolname | rolsuper | rolinherit | rolcre

Re: privileges oddity

2020-08-07 Thread Scott Ribe
> On Aug 7, 2020, at 11:31 AM, Scott Ribe wrote: > > Wondering if there's a code path somewhere that lets the default take > precedence??? So, I changed the defaults, now I see akanzler=arwdDxt/srv_risk, problem persists

Re: privileges oddity

2020-08-07 Thread Scott Ribe
> > What happens if you do?: > > select has_schema_privilege('akanzler', 'zoewang', 'usage'); risk_oltp_prod=# select has_schema_privilege('akanzler', 'zoewang', 'usage'); has_schema_privilege -- t (1 row) > In psql what does > > \ddp > > show? risk_oltp_prod=# \ddp

Re: privileges oddity

2020-08-07 Thread Scott Ribe
Further update: create a new user, grant all on schema & the table, works reboot of server did not change anything, so the problem is in persistent state

Re: privileges oddity

2020-08-06 Thread Scott Ribe
FYI, REVOKE ALL ON SCHEMA... followed by GRANT ALL ON SCHEMA... did not change anything

Re: privileges oddity

2020-08-06 Thread Scott Ribe
> On Aug 6, 2020, at 12:53 PM, Stephen Frost wrote: > > Are you 110% sure that you're actually connecting to the same instance > in both cases (I'd say database too, but hopefully psql isn't lying to > you about that on your prompt, but maybe double-check anyway...). yes--double checked > Have

Re: privileges oddity

2020-08-06 Thread Scott Ribe
> On Aug 6, 2020, at 12:38 PM, Tom Lane wrote: > > Hmph. Any chance of getting a stack trace from the point of the error? possibly > Also, which PG version is this? 12.3 It is probably relevant that we cleaned up roles & privs yesterday, lots of REVOKE & GRANT, and some DROP ROLE. I started

Re: privileges oddity

2020-08-06 Thread Scott Ribe
> On Aug 6, 2020, at 12:36 PM, Adrian Klaver wrote: > > No triggers or FOREIGN KEYS? No. No keys or indexes either--that was the entire table def.

Re: privileges oddity

2020-08-06 Thread Scott Ribe
On Aug 6, 2020, at 12:22 PM, Tom Lane wrote: > > Gonna need more context. The session-level user seems to have the > right privileges, but maybe something is happening inside a > security-definer function that doesn't have privileges? The only security definer function in the db is a simple pg_

Re: privileges oddity

2020-08-06 Thread Scott Ribe
> On Aug 6, 2020, at 12:22 PM, Adrian Klaver wrote: > > Schema for the table? Nothing relevant: Column| Type | Collation | Nullable | Default -+---+---+--+- curve_name | character varying(30) | |

privileges oddity

2020-08-06 Thread Scott Ribe
_risk_ro=U/srv_risk | (1 row) HUH? (And the user also has all privs on all the tables in the schema...) -- Scott Ribe scott_r...@elevated-dev.com https://www.linkedin.com/in/scottribe/

Re: is JIT available

2020-07-28 Thread Scott Ribe
> On Jul 27, 2020, at 9:33 PM, Tom Lane wrote: > > The general, non-hacker meaning of "jit is enabled" would seem to > be pretty much what this function is already doing; and for that > matter, the same can be said for "JIT compilation is available". > We need something that's less tautological-l

Re: is JIT available

2020-07-28 Thread Scott Ribe
> On Jul 27, 2020, at 6:04 PM, David Rowley wrote: > > "returns true if jit is enabled and JIT compilation is available in > this session (see Chapter 31)." That is clearer. I didn't submit a suggestion myself because I'm not clear on the actual circumstances. I know it won't be available if:

Re: bad JIT decision

2020-07-27 Thread Scott Ribe
> On Jul 27, 2020, at 4:00 PM, Alvaro Herrera wrote: > > I don't quite understand why is it that a table with 1000 partitions > means that JIT compiles the thing 1000 times. Sure, it is possible that > some partitions have a different column layout, but it seems an easy bet > that most cases are

Re: is JIT available

2020-07-27 Thread Scott Ribe
> On Jul 25, 2020, at 8:02 AM, Christoph Moench-Tegeder > wrote: > > ## Scott Ribe (scott_r...@elevated-dev.com): > >> So JIT is enabled in your conf, how can you tell from within a client >> session whether it's actually available (PG compiled w

Re: is JIT available

2020-07-25 Thread Scott Ribe
> On Jul 25, 2020, at 6:21 AM, Pavel Stehule wrote: > > It shows if Postgres was compiled with JIT support. > > When you run EXPLAIN ANALYZE SELECT ... then you can see info about JIT > overhead. If you don't see notices about JIT in EXPLAIN, then JIT was not > used. The presence of "jit = on

Re: is JIT available

2020-07-25 Thread Scott Ribe
> On Jul 24, 2020, at 9:55 PM, Pavel Stehule wrote: > > SELECT * FROM pg_config; That doesn't tell me whether or not it can actually be used.

is JIT available

2020-07-24 Thread Scott Ribe
y not working for some reason, lead me down a dead end for a bit.) -- Scott Ribe scott_r...@elevated-dev.com https://www.linkedin.com/in/scottribe/

Re: bad JIT decision

2020-07-24 Thread Scott Ribe
> On Jul 24, 2020, at 4:37 PM, Tom Lane wrote: > > Yeah. I'm fairly convinced that the v12 defaults are far too low, > because we are constantly seeing complaints of this sort. They are certainly too low for our case; not sure if for folks who are not partitioning if they're way too low. The

Re: bad JIT decision

2020-07-24 Thread Scott Ribe
> On Jul 24, 2020, at 4:26 PM, David Rowley wrote: > > It does not really take into account the cost of jitting. That is what I was missing. I read about JIT when 12 was pre-release; in re-reading after my post I see that it does not attempt to estimate JIT cost. And in thinking about it, I r

bad JIT decision

2020-07-24 Thread Scott Ribe
out the restrictiveness of conditions, and is therefore anticipating running the functions against a great many rows? -- Scott Ribe scott_r...@elevated-dev.com https://www.linkedin.com/in/scottribe/

Re: query, probably needs window functions

2020-05-23 Thread Scott Ribe
> On May 22, 2020, at 1:37 PM, Michael Lewis wrote: > > I believe something like this is what you want. You might be able to do it > without a sub-query by comparing the current name value to the lag value and > null it out if it's the same. > ... Thanks, that's what I needed! (And better than

query, probably needs window functions

2020-05-22 Thread Scott Ribe
first row for a person was filled in, with rest of that person's phones showing blanks for those columns? I'm guessing that window functions provide this capability, but I don't know how. -- Scott Ribe scott_r...@elevated-dev.com https://www.linkedin.com/in/scottribe/

Re: parameter limit

2020-04-23 Thread Scott Ribe
> On Apr 23, 2020, at 8:48 AM, Adrian Klaver wrote: > > No. Ah, thanks a lot for that info. It's not that I really normally want to have bazillions of params. But we have some moderately large inserts, that are showing a sudden non-linear dropoff when scaling up, and a suspicion that the perf

parameter limit

2020-04-23 Thread Scott Ribe
it of the driver/library, correct? -- Scott Ribe scott_r...@elevated-dev.com https://www.linkedin.com/in/scottribe/

Re: logical replication protocol

2019-12-24 Thread Scott Ribe
> On Dec 24, 2019, at 6:12 AM, Tatsuo Ishii wrote: > > The logical replication protocol builds on the primitives of the > physical streaming replication protocol as stated in the document. The > explanation of 'k' and 'w' messages can be found in the "Streaming > Replication Protocol" section. T

logical replication protocol

2019-12-24 Thread Scott Ribe
I haven't been able to find documentation on the actual messages used in the logical replication protocol ('k' & 'w', lower case). I've figured things out mostly by reading pg_recvlogical.c, but "Read The Fine Source" doesn't seem in line with the way PG usually does it ;-) Did I miss a doc som

Re: Ideas to deal with table corruption

2018-01-06 Thread scott ribe
should usually be indexed, lest a delete on the referenced table require a table scan on the referencing table. -- Scott Ribe https://www.linkedin.com/in/scottribe/ (303) 722-0567