Re: [GENERAL] Need help for import of text file

2012-12-16 Thread Peter Bex
On Sun, Dec 16, 2012 at 06:30:24PM -0500, Steve Clark wrote: > why not use the squeeze option of tr. > > tr -s " " I wasn't aware of that one, it's even simpler and more elegant. Thanks! For this particular case, tr(1) won't do for the same reason the simple sed(1) expression I gave won't do: it

Re: [GENERAL] Authenticate with hash instead of plaintext password?

2012-12-16 Thread Peter Bex
On Sun, Dec 16, 2012 at 11:17:25AM -0800, Adrian Klaver wrote: > On 12/16/2012 11:07 AM, Peter Bex wrote: > > I don't know how Postgres stores its passwords internally or how > > its authentication works exactly. Maybe one of the developers > > can shine

Re: [GENERAL] Authenticate with hash instead of plaintext password?

2012-12-16 Thread Peter Bex
On Sun, Dec 16, 2012 at 07:42:40PM +0100, Murray Cumming wrote: > On Sun, 2012-12-16 at 17:51 +0100, Peter Bex wrote: > > Hashes (if properly salted and > > stretched) are only useful if they are only ever checked against the > > password itself. Storing a hash of any ki

Re: [GENERAL] Authenticate with hash instead of plaintext password?

2012-12-16 Thread Peter Bex
On Sun, Dec 16, 2012 at 01:30:29PM -0500, Tom Lane wrote: > Peter Bex writes: > > On Sun, Dec 16, 2012 at 12:51:08PM -0500, Tom Lane wrote: > >> Right, they can break into *this account*. > > > Not *just* this one, but any account on any service that uses this >

Re: [GENERAL] Need help for import of text file

2012-12-16 Thread Peter Bex
On Sun, Dec 16, 2012 at 06:48:35PM +0100, Andreas wrote: > With sed as startingpoint I figured it out. > Those 3 steps make the input files consumable for COPY > > 1. dos2unix > 2. sed -i 's/[ \t]*$//' > 3. sed -i 's/ / /g' You can reduce this to one invocation by separating the commands by a se

Re: [GENERAL] Authenticate with hash instead of plaintext password?

2012-12-16 Thread Peter Bex
On Sun, Dec 16, 2012 at 12:51:08PM -0500, Tom Lane wrote: > Peter Bex writes: > > If they do break in and are able to retrieve the password hash, they > > can still break in with that hash. > > Right, they can break into *this account*. Not *just* this one, but any accoun

Re: [GENERAL] Authenticate with hash instead of plaintext password?

2012-12-16 Thread Peter Bex
On Sun, Dec 16, 2012 at 05:51:23PM +0100, Peter Bex wrote: > The best solution I can come up with is not provide a web UI at all > but let the user connect directly to the database using a secure > method (e.g. SSL client certs, GSSAPI etc). Speaking of which, a custom implementati

Re: [GENERAL] Authenticate with hash instead of plaintext password?

2012-12-16 Thread Peter Bex
On Sun, Dec 16, 2012 at 05:38:37PM +0100, Murray Cumming wrote: > On Sun, 2012-12-16 at 17:24 +0100, Peter Bex wrote: > > What's the use of that? > [snip] > > I would not be storing the plaintext password anywhere. That makes it > harder for someone get the plaintext p

Re: [GENERAL] Authenticate with hash instead of plaintext password?

2012-12-16 Thread Peter Bex
On Sun, Dec 16, 2012 at 04:54:30PM +0100, Murray Cumming wrote: > libpq lets me open a connection by specifying a password: > http://www.postgresql.org/docs/9.2/static/libpq-connect.html#LIBPQ-PARAMKEYWORDS > > Is there any way to specify a hash of the password when connecting, instead > of > pr

Re: [GENERAL] Need help for import of text file

2012-12-15 Thread Peter Bex
On Sat, Dec 15, 2012 at 10:16:55PM +0100, Peter Bex wrote: > A simple sed(1) expression should do the trick: > > sed -E 's/ +/ /g' old-file > new-file I just remembered where I could check, and the GNU sed equivalent is: sed -r 's/ +/ /g' old-file > new-fil

Re: [GENERAL] Need help for import of text file

2012-12-15 Thread Peter Bex
On Sat, Dec 15, 2012 at 08:06:44PM +0100, Andreas wrote: > Hi, > > Problem here is in the morning the first digit of the hour is shown as a > blank so there are 2 blanks before the time so COPY misstakes this as an > empty column and gets confused. > > Can someone point me in the direction of a

Re: [GENERAL] "Too far out of the mainstream"

2012-09-01 Thread Peter Bex
On Sat, Sep 01, 2012 at 12:43:15AM +0200, Geert Mak wrote: > There is this case studies section as well - > > http://www.postgresql.org/about/casestudies/ > > Which appear to me a little old and a little too little, one could try to add > more, perhaps. I noticed that the "Share Your Story" lin

Re: [GENERAL] BI tools and postgresql

2012-07-26 Thread Peter Bex
On Thu, Jul 26, 2012 at 08:17:19AM -0400, Chris Curvey wrote: > 1) a tool for power users to be able to get their own data. There are > a few folks here who are smart enough to be trusted with direct access > to the database, but I don't want to ask them to learn SQL. Something > like an open sou

Re: [GENERAL] Feature discussion: Should syntax errors abort a transaction?

2012-06-19 Thread Peter Bex
On Tue, Jun 19, 2012 at 02:20:57AM -0400, Tom Lane wrote: > Craig Ringer writes: > > I've been working in psql a lot recently, and have started to wonder why > > statements with syntax errors or other problems that render them > > unexecutable terminate the transaction. > > Well, the obvious re

Re: [GENERAL] Memory Management in pqlib, Garbage Collection support

2012-05-03 Thread Peter Bex
On Thu, May 03, 2012 at 09:39:29AM +0200, Alexander Reichstadt wrote: > Thanks, that's answering my question. In Objective-C as well as many other > languages there is the feature to turn on Garbage Collection. It's a separate > thread that scans memory for strong pointers, their source and origi

Re: [GENERAL] Memory Management in pqlib, Garbage Collection support

2012-05-03 Thread Peter Bex
On Thu, May 03, 2012 at 09:08:53AM +0200, Alexander Reichstadt wrote: > > Hi, > > since I got no answer so far I searched through the docu again. I searched > for GC as well as Garbage, and all garbage refers to is with regard to > vacuuming a database. But my question refers to wether or not m

Re: [GENERAL] Zero-length character breaking query?

2012-03-16 Thread Peter Bex
On Thu, Mar 15, 2012 at 05:09:32PM -0600, Doug Gorley wrote: > G'day, > > "select * from tdt_unsent where str_name_l = 'SMITH'" returns 0 rows. > "select * from tdt_unsent where str_name_l ~ '^SMITH'" returns 3 rows. > "select * from tdt_unsent where str_name_l ~ '^SMITH$'" returns 0 rows. > "sele

Re: [GENERAL] How to tame a gigantic (100+ lines) query in a web app?

2011-08-14 Thread Peter Bex
On Sun, Aug 14, 2011 at 10:39:48AM -0400, W. Matthew Wilson wrote: > I'm sure I'm not the first person to end up with a gigantic query that > does lots of left joins and subselects. > > It seems to work, but I would love to break it up into smaller chunks. > > I'm thinking about rewriting the que

Re: [GENERAL] Implementing "thick"/"fat" databases

2011-07-27 Thread Peter Bex
On Wed, Jul 27, 2011 at 04:35:45PM +0200, Karsten Hilbert wrote: > > I wonder which other languages have first class support for these areas of > > Pg? > > While already supporting most if not all standard PG datatypes the > psycopg2 interface lets you write in/out wrappers of arbitray > complexit

Re: [GENERAL] Postgres errors in Drupal install.

2011-06-26 Thread Peter Bex
On Sun, Jun 26, 2011 at 11:23:30PM +0200, Dave Coventry wrote: [snip] > 2011-06-26 17:13:24 EDT DETAIL: Failed system call was > shmget(key=5432001, size=29278208, 03600). > 2011-06-26 17:13:24 EDT HINT: This error usually means that > PostgreSQL's request for a shared memory segment exceeded ava

Re: [GENERAL] Postgres errors in Drupal install.

2011-06-26 Thread Peter Bex
On Sun, Jun 26, 2011 at 09:36:59PM +0200, Dave Coventry wrote: > I'm having huge problems with a Drupal install using Postgres 8.4.8. > > I'm getting the following error: Do you get the error in the logs or do you see it while going to the site manually and does it show this instead of rendering

Re: [GENERAL] Executing prepared statements via bind params

2011-06-16 Thread Peter Bex
On Thu, Jun 16, 2011 at 05:19:41PM -0400, Tom Lane wrote: > Peter Bex writes: > > But when I try to do the same but pas the 2 as a parameter, > > (I do "EXECUTE bar($1)" with $1 bound to "2"), I get an error: > > Why would you do that, rather than executi

[GENERAL] Executing prepared statements via bind params

2011-06-16 Thread Peter Bex
Hi all, I'm seeing a bit strange (some might say inconsistent) behaviour, possibly a bug. First, I prepare a statement by running the query: "PREPARE bar (int) AS (SELECT 1, $1, 3)" Then I try to use the generic libpq query function PQsendQueryParams with a query of "EXECUTE bar(2)" and I ge

Re: [GENERAL] alter table add column - specify where the column will go?

2010-11-24 Thread Peter Bex
On Wed, Nov 24, 2010 at 09:37:02AM +, Grzegorz Jaƛkiewicz wrote: > just never use SELECT *, but always call columns by names. You'll > avoid having to depend on the order of columns, which is never > guaranteed, even if the table on disk is one order, the return columns > could be in some other

[GENERAL] Postgres tips in the latest Chicken Gazette

2010-11-22 Thread Peter Bex
Hello all, I'm not sure if this would be considered off topic, but I'd like to point out this week's issue of the Chicken Gazette in which I describe integration of array and "ROW" composite type handling with the PostgreSQL library for the Chicken Scheme compiler: http://gazette.call-cc.org/issue

Re: [GENERAL] More then 1600 columns?

2010-11-12 Thread Peter Bex
On Fri, Nov 12, 2010 at 10:18:31AM +0100, Thomas Kellerer wrote: > The hstore module would also be a viable alternative - and it's indexable > as well. That's what the post I replied to mentioned :) Also, when I looked at hstore, it had a limitation on its length which made it a little dangerous

Re: [GENERAL] More then 1600 columns?

2010-11-11 Thread Peter Bex
On Fri, Nov 12, 2010 at 10:43:14AM +0300, Dmitriy Igrishin wrote: > Hey Peter, > > Unfortunately, there is no indexes on arrays (only on expressions). > With hstore we can easily create GiST index for effective access. True. In my project I only ever needed to search on a particular key, and I ma

Re: [GENERAL] More then 1600 columns?

2010-11-11 Thread Peter Bex
On Fri, Nov 12, 2010 at 10:17:50AM +0300, Dmitriy Igrishin wrote: > Hey Mark, > > Yeah, I can't imagine an entity in a real project even with more than 100 > columns. Its rare case. > But if you entities (rows/tuples) of some class (table) can contains > variable > set of columns (properties) you

Re: [GENERAL] Advice needed on application/database authentication/authorization/auditing model

2010-10-22 Thread Peter Bex
On Fri, Oct 22, 2010 at 08:20:11PM +0400, Dmitriy Igrishin wrote: > Hey Peter, Hello Dmitriy, > > As far as I can see, this would imply either creating views on the > > for every user (or company?), or manually crafting queries > > to do the same. The latter is of course what most webapps do, an

Re: [GENERAL] Advice needed on application/database authentication/authorization/auditing model

2010-10-22 Thread Peter Bex
On Fri, Oct 22, 2010 at 12:21:17AM +0400, Dmitriy Igrishin wrote: > Hey Tony, > > 2010/10/21 Tony Cebzanov > > > I have a web application with a Postgres backend. In my initial > > prototype, I decided not to have a Postgres database user created for > > each application user, opting instead to

Re: [GENERAL] Prepared statements and unknown types

2010-09-29 Thread Peter Bex
On Wed, Sep 29, 2010 at 07:33:53PM +0100, Thom Brown wrote: > Okay, I understand what's happening. But does the planner need to > understand the type of literals in the select list if it's not used > anywhere else? Fields sent back to the client also carry their type with them. There's no "unknow

Re: [GENERAL] Prepared statements and unknown types

2010-09-29 Thread Peter Bex
On Wed, Sep 29, 2010 at 07:08:22PM +0100, Thom Brown wrote: > Could someone explain why the following doesn't work? > > test=# PREPARE meow(unknown) AS > test-# SELECT $1 as meow; > ERROR: could not determine data type of parameter $1 > > The problem is that using PDO in PHP, prepared statements

Re: [GENERAL] MySQL versus Postgres

2010-08-06 Thread Peter Bex
On Fri, Aug 06, 2010 at 02:25:27PM -0600, Scott Marlowe wrote: > For an extra added bonus, show a click-path from the front page > mysql.com to the documentation. I gave up and just entered > mysql.com/documentation which kicked off the search engine and let me > click somewhere into the 5.1 docs.

Re: [GENERAL] MySQL versus Postgres

2010-08-06 Thread Peter Bex
On Fri, Aug 06, 2010 at 02:09:43PM -0600, Scott Marlowe wrote: > I'm all in favor of a nice tutorial section in the docs. But I > certainly don't want pgsql docs to mimick the mysql docs method of > using a tutorial method for most of the beginner information. It's so > dang hard to find anything

Re: [GENERAL] How Big is Too Big for Tables?

2010-07-28 Thread Peter Bex
On Wed, Jul 28, 2010 at 02:05:47PM -0500, P Kishor wrote: > each row is half a dozen single byte values, so, it is actually 6 > bytes per row (six columns). Even if I combine them somehow, still the > per row overhead (which, I believe, is about 23 bytes) is more than > the data. But, that is not t

Re: [GENERAL] Tuple storage overhead

2010-04-20 Thread Peter Bex
On Fri, Apr 16, 2010 at 11:28:36AM -0400, Merlin Moncure wrote: > If you are storing big data and want to keep the overhead low, the > first thing you need to examine is organizing your data into arrays. > This involves tradeoffs of course and may not work but it's worth a > shot! That does sound

Re: [GENERAL] Tuple storage overhead

2010-04-20 Thread Peter Bex
On Fri, Apr 16, 2010 at 12:40:21PM +0200, Szymon Guz wrote: > I thought that the default fillfactor was much smaller (and haven't checked > that now)... sorry for messing that up. > But let's think of it from the other side: what do you want to do with that > data? Maybe PostgreSQL with it's MVCC'

Re: [GENERAL] Tuple storage overhead

2010-04-16 Thread Peter Bex
On Fri, Apr 16, 2010 at 11:59:38AM +0200, Szymon Guz wrote: > File pages are not fully filled from the start as that could result in bad > performance of queries later. The manual page you linked to says something else: "The fillfactor for a table is a percentage between 10 and 100. 100 (complete

[GENERAL] Tuple storage overhead

2010-04-16 Thread Peter Bex
Hi all, I have a table with three columns: one integer and two doubles. There are two indexes defined (one on the integer and one on one of the doubles). This table stores 70 records, which take up 30 Mb according to pg_relation_size(), and the total relation size is 66 Mb. I expected the di

Re: [GENERAL] Array value syntax and escaping

2010-04-01 Thread Peter Bex
On Thu, Apr 01, 2010 at 04:37:23PM -0400, Tom Lane wrote: > There is not, but you could always look at the source for array_out and > array_in if you want definitive answers ;-). Personally what I'd do is > always double-quote each non-null array element; then the rules reduce > to "backslash any

[GENERAL] Array value syntax and escaping

2010-04-01 Thread Peter Bex
Hello all, I am currently adding array value handling to the PostgreSQL interface for the Chicken Scheme compiler[*] and I was wondering if there's a more detailed documentation for the exact syntax of arrays than the short natural language explanation in the manual. There doesn't appear to be a