Re: [GENERAL] Question about synchronous replication

2014-05-13 Thread Borodin Vladimir

12 мая 2014 г., в 22:26, Adrian Klaver  написал(а):

> On 05/12/2014 09:42 AM, Borodin Vladimir wrote:
>> Hi all.
>> 
>> Right now synchronous replication in postgresql chooses one replica as
>> synchronous and waits for replies from it (with synchronous_commit = on
>> | remote_write) until this replica host does not disconnect from master.
>> 
>> Are there any plans to implement something like semi synchronous
>> replication in MySQL 5.6 or replication with write_concern=2 in MongoDB
>> when the master waits for a reply from any of the replica hosts?
> 
> This does not work for what you want?:
> 
> http://www.postgresql.org/docs/9.3/interactive/runtime-config-replication.html#GUC-SYNCHRONOUS-STANDBY-NAMES

Actually no. I have such settings:

wal_sender_timeout = 3s
wal_receiver_status_interval = 1s
synchronous_standby_names = ‘*’

When the sync replica dies or the network between master and sync replica 
flaps, 3 seconds must pass before the potential replica becomes sync and 
transaction commits continue. Instead postgresql could wait for confirm from 
first or second replica hosts (doesn’t matter which of them answers first), 
couldn’t it? In this case transaction commits will not stuck for 
wal_sender_timeout.

> 
>> 
>> In this case network flaps between master and any one replica will not
>> affect writing load and in case of master fail it would be necessary to
>> find the most recent replica and promote it. Or there are pitfalls that
>> I do not see?
>> 
>> --
>> Vladimir
>> 
>> 
>> 
>> 
> 
> 
> -- 
> Adrian Klaver
> adrian.kla...@aklaver.com


--
Да пребудет с вами сила...
http://simply.name







Re: [GENERAL] Log Data Analytics : Confused about the choice of Database

2014-05-13 Thread Dorian Hoxha
Why not store session as integer?

And timestamp as timesamp(z?) ?

If you know the types of events, also store them as integer , and save a
map of them in the app or on another table ?

And save the parameters as a json column, so you have more data-types?
Hstore only has strings.

Be carefull with the mongodb hipster on the stackoverflow post.
Elasticsearch is often used for log collection.

So, what really is the problem ?



On Tue, May 13, 2014 at 4:11 AM, Peeyush Agarwal <
peeyushagarwal1...@gmail.com> wrote:

> Hi,
>
> I have log data of the following format:
>
>
> SessionTimestampEventParameters1  1   
>  Started Session  1  2Logged In
> Username:"user1"2  3Started Session1  3   
>  Started Challengetitle:"Challenge 1", level:"2"2  4
> Logged InUsername:"user2"
>
> Now, a person wants to carry out analytics on this log data (And would
> like to receive it as a JSON blob after appropriate transformations). For
> example, he may want to receive a JSON blob where the Log Data is grouped
> by Session and TimeFromSessionStart and CountOfEvents are added before
> the data is sent so that he can carry out meaningful analysis. Here I
> should return:
>
>
> [
>   {
> 
> "session":1,"CountOfEvents":3,"Actions":[{"TimeFromSessionStart":0,"Event":"Session
>  Started"}, {"TimeFromSessionStart":1, "Event":"Logged In", 
> "Username":"user1"}, {"TimeFromSessionStart":2, "Event":"Startd Challenge", 
> "title":"Challenge 1", "level":"2" }]
>   },
>   {
> "session":2, 
> "CountOfEvents":2,"Actions":[{"TimeFromSessionStart":0,"Event":"Session 
> Started"}, {"TimeFromSessionStart":2, "Event":"Logged In", 
> "Username":"user2"}]
>   }]
>
>
> Here, TimeFromSessionStart, CountOfEvents etc. [Let's call it synthetic
> additional data] will not be hard coded and I will make a web interface to
> allow the person to decide what kind of synthetic data he requires in the
> JSON blob. I would like to provide a good amount of flexibility to the
> person to decide what kind of synthetic data he wants in the JSON blob.
>
> If I use PostgreSQL, I can store the data in the following manner: Session
>  and Event can be string, Timestamp can be date and Parameters can be
> hstore(key value pairs available in PostgreSQL). After that, I can use
> SQL queries to compute the synthetic (or additional) data, store it
> temporarily in variables in a Rails Application (which will interact with
> PostgreSQL database and act as interface for the person who wants the JSON
> blob) and create JSON blob from it.
>
> However I am not sure if PostgreSQL is the best choice for this use case.
> I have put the detailed question on SO at
> http://stackoverflow.com/questions/23544604/log-data-analytics
>
>  Looking for some help from the community.
>
> Peeyush Agarwal
>


[GENERAL] Natural key woe

2014-05-13 Thread Oliver Kohll - Mailing Lists
I'm sure no one else on this list has done anything like this, but here's a 
cautionary tale.

I wanted to synchronise data in two tables (issue lists) - i.e. whenever a 
record is added into one, add a similar record into the other. The two tables 
are similar in format but not exactly the same so only a subset of fields are 
copied. Both tables have synthetic primary keys, these can't be used to match 
data as they are auto-incrementing sequences that might interfere. What I could 
have done perhaps is get both tables to use the same sequence, but what I 
actually did is:

* join both tables based on a natural key
* use that to copy any missing items from table1 to table2
* truncate table1 and copy all of table2's rows to table1
* run this routine once an hour

The natural key was based on the creation timestamp (stored on insert) and the 
one of the text fields, called 'subject'.

The problem came when someone entered a record with no subject, but left it 
null. When this was copied over and present in both tables, the *next* time the 
join was done, a duplicate was created because the join didn't see them as 
matching (null != null).

So after 1 hour there were two records. After two there were four, after 3, 8 
etc.

When I logged in after 25 hrs and noticed table access was a little slow, there 
were 2^25 = 33 million records.

That's a learning experience for me at least. It's lucky I did check it at the 
end of that day rather than leaving it overnight, otherwise I think our server 
would have ground to a halt.

One other wrinkle to note. After clearing out these rows, running 'VACUUM 
table2', 'ANALYZE table2' and 'REINDEX table table2', some queries with simple 
sequence scans were taking a few seconds to run even though there are only a 
thousand rows in the table. I finally found that running CLUSTER on the table 
sorted that out, even though we're on an SSD so I would have thought seeking 
all over the place for a seq. scan wouldn't have made that much difference. It 
obviously does still make some.

Oliver Kohll
www.agilebase.co.uk






-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Natural key woe

2014-05-13 Thread Robin

Oliver

I've read your email, with interest. I haven't had to deal with this 
sort of problem in PostgreSQL, but I have frequently dealt with it in a 
Sybase environment, first encountered about 25 years ago.


I am most curious to know why you didn't use the same sequence for both 
tables, I must be missing something. If there is a gotcha, I'd like to 
know about it as I can see this being an issue in a load sharing 
environment.
Many years ago, before auto sequencing was common, we set up explicit 
functions to generate sequwnce numbers. Whilst this had some perormance 
costs in multi-user systems, it did have benefits in terms of making it 
easier to restrict the need for row locking to the underlying data table.


Robin St.Clair
On 13/05/2014 10:44, Oliver Kohll - Mailing Lists wrote:

I'm sure no one else on this list has done anything like this, but here's a 
cautionary tale.

I wanted to synchronise data in two tables (issue lists) - i.e. whenever a 
record is added into one, add a similar record into the other. The two tables 
are similar in format but not exactly the same so only a subset of fields are 
copied. Both tables have synthetic primary keys, these can't be used to match 
data as they are auto-incrementing sequences that might interfere. What I could 
have done perhaps is get both tables to use the same sequence, but what I 
actually did is:

* join both tables based on a natural key
* use that to copy any missing items from table1 to table2
* truncate table1 and copy all of table2's rows to table1
* run this routine once an hour

The natural key was based on the creation timestamp (stored on insert) and the 
one of the text fields, called 'subject'.

The problem came when someone entered a record with no subject, but left it 
null. When this was copied over and present in both tables, the *next* time the 
join was done, a duplicate was created because the join didn't see them as 
matching (null != null).

So after 1 hour there were two records. After two there were four, after 3, 8 
etc.

When I logged in after 25 hrs and noticed table access was a little slow, there 
were 2^25 = 33 million records.

That's a learning experience for me at least. It's lucky I did check it at the 
end of that day rather than leaving it overnight, otherwise I think our server 
would have ground to a halt.

One other wrinkle to note. After clearing out these rows, running 'VACUUM 
table2', 'ANALYZE table2' and 'REINDEX table table2', some queries with simple 
sequence scans were taking a few seconds to run even though there are only a 
thousand rows in the table. I finally found that running CLUSTER on the table 
sorted that out, even though we're on an SSD so I would have thought seeking 
all over the place for a seq. scan wouldn't have made that much difference. It 
obviously does still make some.

Oliver Kohll
www.agilebase.co.uk










--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Natural key woe

2014-05-13 Thread Eelke Klein
>
>> One other wrinkle to note. After clearing out these rows, running 'VACUUM
>> table2', 'ANALYZE table2' and 'REINDEX table table2', some queries with
>> simple sequence scans were taking a few seconds to run even though there
>> are only a thousand rows in the table. I finally found that running CLUSTER
>> on the table sorted that out, even though we're on an SSD so I would have
>> thought seeking all over the place for a seq. scan wouldn't have made that
>> much difference. It obviously does still make some.
>>
>> A VACUUM FULL table2 would have made the CLUSTER unnecesary. A normal
VACUUM only marks dead rows as free but does not shrink the table. A VACUUM
FULL removes all the free space from the table and returns it to the OS.

Eelke Klein


Re: [GENERAL] Question about synchronous replication

2014-05-13 Thread Adrian Klaver

On 05/13/2014 12:08 AM, Borodin Vladimir wrote:


12 мая 2014 г., в 22:26, Adrian Klaver mailto:adrian.kla...@aklaver.com>> написал(а):


On 05/12/2014 09:42 AM, Borodin Vladimir wrote:

Hi all.

Right now synchronous replication in postgresql chooses one replica as
synchronous and waits for replies from it (with synchronous_commit = on
| remote_write) until this replica host does not disconnect from master.

Are there any plans to implement something like semi synchronous
replication in MySQL 5.6 or replication with write_concern=2 in MongoDB
when the master waits for a reply from any of the replica hosts?


This does not work for what you want?:

http://www.postgresql.org/docs/9.3/interactive/runtime-config-replication.html#GUC-SYNCHRONOUS-STANDBY-NAMES


Actually no. I have such settings:

wal_sender_timeout = 3s
wal_receiver_status_interval = 1s
synchronous_standby_names = ‘*’

When the sync replica dies or the network between master and sync
replica flaps, 3 seconds must pass before the potential replica becomes
sync and transaction commits continue. Instead postgresql could wait for
confirm from first or second replica hosts (doesn’t matter which of them
answers first), couldn’t it? In this case transaction commits will not
stuck for wal_sender_timeout.



Postgres is doing what you tell it. You have said you are willing to 
waits 3 secs for any inactivity between the master and the standby to 
resolve itself before declaring that particular connection dead. Once 
that happens then Postgres starts working through the list of names. The 
wal_sender_timeout value by default is in milliseconds so you can make 
that value very small. It would seem prudent to have some value there in 
order to deal with temporary network hiccups.


--
Adrian Klaver
adrian.kla...@aklaver.com


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] Log Data Analytics : Confused about the choice of Database

2014-05-13 Thread Peeyush Agarwal
Hi,

I have log data of the following format:

SessionTimestampEventParameters1  1
Started Session  1  2Logged In
   Username:"user1"2  3Started Session1  3
   Started Challengetitle:"Challenge 1", level:"2"2
  4Logged InUsername:"user2"

Now, a person wants to carry out analytics on this log data (And would like
to receive it as a JSON blob after appropriate transformations). For
example, he may want to receive a JSON blob where the Log Data is grouped
by Session and TimeFromSessionStart and CountOfEvents are added before the
data is sent so that he can carry out meaningful analysis. Here I should
return:

[
  {

"session":1,"CountOfEvents":3,"Actions":[{"TimeFromSessionStart":0,"Event":"Session
Started"}, {"TimeFromSessionStart":1, "Event":"Logged In",
"Username":"user1"}, {"TimeFromSessionStart":2, "Event":"Startd
Challenge", "title":"Challenge 1", "level":"2" }]
  },
  {
"session":2,
"CountOfEvents":2,"Actions":[{"TimeFromSessionStart":0,"Event":"Session
Started"}, {"TimeFromSessionStart":2, "Event":"Logged In",
"Username":"user2"}]
  }]


Here, TimeFromSessionStart, CountOfEvents etc. [Let's call it synthetic
additional data] will not be hard coded and I will make a web interface to
allow the person to decide what kind of synthetic data he requires in the
JSON blob. I would like to provide a good amount of flexibility to the
person to decide what kind of synthetic data he wants in the JSON blob.

If I use PostgreSQL, I can store the data in the following manner: Session
 and Event can be string, Timestamp can be date and Parameters can be
hstore(key
value pairs available in PostgreSQL). After that, I can use SQL queries to
compute the synthetic (or additional) data, store it temporarily in
variables in a Rails Application (which will interact with PostgreSQL
database and act as interface for the person who wants the JSON blob) and
create JSON blob from it.

However I am not sure if PostgreSQL is the best choice for this use case. I
have put the detailed question on SO at
http://stackoverflow.com/questions/23544604/log-data-analytics

Looking for some help from the community.

Peeyush Agarwal


[GENERAL] better performance on poorer machine?

2014-05-13 Thread Vegard Bønes
Hi,

I have a database on a test server with queries that perform terribly. Trying 
to fix this problem, I copied the database (using pg_dump) to my laptop, and 
reran tests there. The same queries perform perfectly on my laptop.

I have tried to use the same postgresql.conf, and run ANALYZE and even VACUUM 
ANALYZE on the databases, but the problem remains.

EXPLAIN shows a somewhat different query plan for each database.

Now I have no idea what to do next. How can I go about trying to find the cause 
of this? I can see no other explanation than hardware issues, but in theory, 
the test servers (there are more than one, with the same performance problems) 
should be better than my laptop in all ways. Have I missed something obvious?


regards,
Vegard


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Natural key woe

2014-05-13 Thread Yeb Havinga

On 13/05/14 11:44, Oliver Kohll - Mailing Lists wrote:

The problem came when someone entered a record with no subject, but left it 
null. When this was copied over and present in both tables, the *next* time the 
join was done, a duplicate was created because the join didn't see them as 
matching (null != null).

Maybe you can use x IS NOT DISTINCT FROM y ?

regards,
Yeb



--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] better performance on poorer machine?

2014-05-13 Thread Rob Sargent

On 05/13/2014 08:48 AM, Vegard Bønes wrote:

Hi,

I have a database on a test server with queries that perform terribly. Trying 
to fix this problem, I copied the database (using pg_dump) to my laptop, and 
reran tests there. The same queries perform perfectly on my laptop.

I have tried to use the same postgresql.conf, and run ANALYZE and even VACUUM 
ANALYZE on the databases, but the problem remains.

EXPLAIN shows a somewhat different query plan for each database.

Now I have no idea what to do next. How can I go about trying to find the cause 
of this? I can see no other explanation than hardware issues, but in theory, 
the test servers (there are more than one, with the same performance problems) 
should be better than my laptop in all ways. Have I missed something obvious?


regards,
Vegard


First suspicion is of course a lack of indexing on the original.  I 
would compare the full definitions of the tables involved.





[GENERAL] Full-Text Search question

2014-05-13 Thread Jesus Rafael Sanchez Medrano
can postgres do FTS (full text search) on a json column? if possible,
please be so kindd to give some snippet/example.


Att.
==
Jesus Rafael Sanchez Medrano
"Life is a dream, of which all must wake up"


Re: [GENERAL] better performance on poorer machine?

2014-05-13 Thread Alban Hertroys
On 13 May 2014 16:48, Vegard Bønes  wrote:
> I have a database on a test server with queries that perform terribly. Trying 
> to fix this problem, I copied the database (using pg_dump) to my laptop, and 
> reran tests there. The same queries perform perfectly on my laptop.

We can but guess, but... Quite possibly your laptop is not actually a
poorer machine for single uncomplicated queries. If it's cores are
faster than the test servers, than your laptop would out-perform the
server for such queries.

Once you get other users starting to run queries as well, turning the
load into a parallel load, the server will probably turn out faster
again.

-- 
If you can't see the forest for the trees,
Cut the trees and you'll see there is no forest.


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] better performance on poorer machine?

2014-05-13 Thread Tom Lane
Alban Hertroys  writes:
> On 13 May 2014 16:48, Vegard Bønes  wrote:
>> I have a database on a test server with queries that perform terribly. 
>> Trying to fix this problem, I copied the database (using pg_dump) to my 
>> laptop, and reran tests there. The same queries perform perfectly on my 
>> laptop.

> We can but guess, but...

Less guess-worthy answers might be possible if you provide the information
suggested here:

https://wiki.postgresql.org/wiki/Slow_Query_Questions

regards, tom lane


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] logging hook

2014-05-13 Thread Alan Nilsson
All

Is there a hook that someone could direct me to that I can use to redirect 
logging output?  I.e.  I would like to write a small module that pre loads and 
substitutes the loggers file descriptor with my own to redirect that output.  
Possible?

thanks
alan



-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Full-Text Search question

2014-05-13 Thread Oleg Bartunov
Easy, you need to extract text fields from json and construct tsvector
from them (use concatenation, for example).

On Tue, May 13, 2014 at 7:38 PM, Jesus Rafael Sanchez Medrano
 wrote:
> can postgres do FTS (full text search) on a json column? if possible, please
> be so kindd to give some snippet/example.
>
>
> Att.
> ==
> Jesus Rafael Sanchez Medrano
> "Life is a dream, of which all must wake up"


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] logging hook

2014-05-13 Thread Susan Cassidy
You can log to syslog, and use the syslog definitions file (syslog.conf) to
pipe the output to a program that can do whatever you want with it.

Susan


On Tue, May 13, 2014 at 2:29 PM, Alan Nilsson  wrote:

> All
>
> Is there a hook that someone could direct me to that I can use to redirect
> logging output?  I.e.  I would like to write a small module that pre loads
> and substitutes the loggers file descriptor with my own to redirect that
> output.  Possible?
>
> thanks
> alan
>
>
>
> --
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>


Re: [GENERAL] logging hook

2014-05-13 Thread Alan Nilsson
Ah yes once again if I’ld just RTFM….. Thanks for pointing that out, I think I 
can work with that.

alan

On May 13, 2014, at 2:59 PM, Susan Cassidy 
 wrote:

> You can log to syslog, and use the syslog definitions file (syslog.conf) to 
> pipe the output to a program that can do whatever you want with it.
> 
> Susan
> 
> 
> On Tue, May 13, 2014 at 2:29 PM, Alan Nilsson  wrote:
> All
> 
> Is there a hook that someone could direct me to that I can use to redirect 
> logging output?  I.e.  I would like to write a small module that pre loads 
> and substitutes the loggers file descriptor with my own to redirect that 
> output.  Possible?
> 
> thanks
> alan
> 
> 
> 
> --
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
> 



Re: [GENERAL] Full-Text Search question

2014-05-13 Thread Jesus Rafael Sanchez Medrano
thanks... could you please be so kind to post some snippet/code for this?

Att.
==
Jesus Rafael Sanchez Medrano
"Life is a dream, of which all must wake up"


On Tue, May 13, 2014 at 5:33 PM, Oleg Bartunov  wrote:

> Easy, you need to extract text fields from json and construct tsvector
> from them (use concatenation, for example).
>
> On Tue, May 13, 2014 at 7:38 PM, Jesus Rafael Sanchez Medrano
>  wrote:
> > can postgres do FTS (full text search) on a json column? if possible,
> please
> > be so kindd to give some snippet/example.
> >
> >
> > Att.
> > ==
> > Jesus Rafael Sanchez Medrano
> > "Life is a dream, of which all must wake up"
>


Re: [GENERAL] Log Data Analytics : Confused about the choice of Database

2014-05-13 Thread Peeyush Agarwal
Hi,

Thanks for the reply :)

Yes, Storing timestamp as timestamp [ (p) ] would be better. I simplified
the session in question. It may contain alphabets as well. So, I will
probably need to store it as a string only.

The problem with types of events is that it is not fixed and will keep
increasing over time (as more event types are added depending on the need).
Would you still recommend saving a map in another table? Will it have a
significant storage benefit and are there any other benefits? Storing it in
the app will require some addition in code each time a new event type is
added which is not difficult but time consuming and tedious.

I am not very familiar with json data type. Can I query on it as
effectively as hstore? Also, is it possible to index it partially so as to
increase query speeds for certain types of queries? For eg. I would require
parameter username in a lot of queries so I would like to partially index
over it.

Peeyush Agarwal


On Tue, May 13, 2014 at 3:13 PM, Dorian Hoxha wrote:

> Why not store session as integer?
>
> And timestamp as timesamp(z?) ?
>
> If you know the types of events, also store them as integer , and save a
> map of them in the app or on another table ?
>
> And save the parameters as a json column, so you have more data-types?
> Hstore only has strings.
>
> Be carefull with the mongodb hipster on the stackoverflow post.
> Elasticsearch is often used for log collection.
>
> So, what really is the problem ?
>
>
>
> On Tue, May 13, 2014 at 4:11 AM, Peeyush Agarwal <
> peeyushagarwal1...@gmail.com> wrote:
>
>> Hi,
>>
>> I have log data of the following format:
>>
>> SessionTimestampEventParameters1  1  
>>   Started Session  1  2Logged In
>> Username:"user1"2  3Started Session1  3  
>>   Started Challengetitle:"Challenge 1", level:"2"2  4
>> Logged InUsername:"user2"
>>
>> Now, a person wants to carry out analytics on this log data (And would
>> like to receive it as a JSON blob after appropriate transformations). For
>> example, he may want to receive a JSON blob where the Log Data is grouped
>> by Session and TimeFromSessionStart and CountOfEvents are added before
>> the data is sent so that he can carry out meaningful analysis. Here I
>> should return:
>>
>> [
>>   {
>> 
>> "session":1,"CountOfEvents":3,"Actions":[{"TimeFromSessionStart":0,"Event":"Session
>>  Started"}, {"TimeFromSessionStart":1, "Event":"Logged In", 
>> "Username":"user1"}, {"TimeFromSessionStart":2, "Event":"Startd Challenge", 
>> "title":"Challenge 1", "level":"2" }]
>>   },
>>   {
>> "session":2, 
>> "CountOfEvents":2,"Actions":[{"TimeFromSessionStart":0,"Event":"Session 
>> Started"}, {"TimeFromSessionStart":2, "Event":"Logged In", 
>> "Username":"user2"}]
>>   }]
>>
>>
>> Here, TimeFromSessionStart, CountOfEvents etc. [Let's call it synthetic
>> additional data] will not be hard coded and I will make a web interface to
>> allow the person to decide what kind of synthetic data he requires in the
>> JSON blob. I would like to provide a good amount of flexibility to the
>> person to decide what kind of synthetic data he wants in the JSON blob.
>>
>> If I use PostgreSQL, I can store the data in the following manner:
>> Session and Event can be string, Timestamp can be date and Parameters can
>> be hstore(key value pairs available in PostgreSQL). After that, I can
>> use SQL queries to compute the synthetic (or additional) data, store it
>> temporarily in variables in a Rails Application (which will interact with
>> PostgreSQL database and act as interface for the person who wants the JSON
>> blob) and create JSON blob from it.
>>
>> However I am not sure if PostgreSQL is the best choice for this use case.
>> I have put the detailed question on SO at
>> http://stackoverflow.com/questions/23544604/log-data-analytics
>>
>>  Looking for some help from the community.
>>
>> Peeyush Agarwal
>>
>
>


-- 
Peeyush Agarwal
IIT Kanpur
+91 8953453689


[GENERAL] LDAP authentication not working

2014-05-13 Thread Jürgen Fuchsberger
Hi,

I'm running postgresql 9.1 on Debian and am trying to set up LDAP
authentication using the following configuration in pg_hba.conf:

hostssl testdb  all 143.50.203.0/24 ldap ldapserver="wegc24.uni-graz.at"
ldapport=636 ldapbinddn="cn=nss,dc=uni-graz,dc=at"
ldapbindpasswd="" ldapbasedn="dc=uni-graz,dc=at"


Trying to access testdb via psql fails with the following error in the log:
'''could not perform initial LDAP bind for ldapbinddn
"cn=nss,dc=uni-graz,dc=at" on server "wegc24.uni-graz.at": error code -1'''

Unfortunately I did not find what error code -1 means.

Ldapsearch works fine:
> ldapsearch -W -H ldaps://wegc24.uni-graz.at:636/ -D
"CN=nss,DC=uni-graz,DC=at"

Interesting is also, that postgres seems to not even reach the ldap
server: If I change parameter ldapserver to a non-existing url it gives
the same error code -1.

Any help much appreciated!

Best,
Juergen



signature.asc
Description: OpenPGP digital signature


Re: [GENERAL] LDAP authentication not working

2014-05-13 Thread Stephan Fabel
I don't think SSL support for LDAP is supported. Have you tried TLS on port
389?
On May 13, 2014 8:20 PM, "Jürgen Fuchsberger" <
juergen.fuchsber...@uni-graz.at> wrote:

> Hi,
>
> I'm running postgresql 9.1 on Debian and am trying to set up LDAP
> authentication using the following configuration in pg_hba.conf:
>
> hostssl testdb  all 143.50.203.0/24 ldap ldapserver="
> wegc24.uni-graz.at"
> ldapport=636 ldapbinddn="cn=nss,dc=uni-graz,dc=at"
> ldapbindpasswd="" ldapbasedn="dc=uni-graz,dc=at"
>
>
> Trying to access testdb via psql fails with the following error in the log:
> '''could not perform initial LDAP bind for ldapbinddn
> "cn=nss,dc=uni-graz,dc=at" on server "wegc24.uni-graz.at": error code
> -1'''
>
> Unfortunately I did not find what error code -1 means.
>
> Ldapsearch works fine:
> > ldapsearch -W -H ldaps://wegc24.uni-graz.at:636/ -D
> "CN=nss,DC=uni-graz,DC=at"
>
> Interesting is also, that postgres seems to not even reach the ldap
> server: If I change parameter ldapserver to a non-existing url it gives
> the same error code -1.
>
> Any help much appreciated!
>
> Best,
> Juergen
>
>