Re: [GENERAL] Restore 1 Table from pg_dumpall?

2013-01-27 Thread Ondrej Ivanič
Hi, On 23 January 2013 04:57, Rich Shepard wrote: > Is there a way I can extract a single table's schema and data from the > full backup? If so, I can then drop the fubar'd table and do it correctly > this time. You should grep for: - CREATE TABLE - COPY statements and then note line numbers

Re: [GENERAL] PostgreSQL vs Mongo

2013-10-16 Thread Ondrej Ivanič
Hi, On 17 October 2013 02:30, CS DBA wrote: > Anyone have any thoughts on why we would / would not use Mongo for a > reporting environment. hm.. I wouldn't use anything which doesn't support rich SQL as a backed for reporting system. In mongo, simple selects are fine but anything complex require

[GENERAL] table / query as a prameter for PL/pgSQL function

2011-08-07 Thread Ondrej Ivanič
Hi, It is possible to pass query result (or cursor?) as function parameter? I need a function which emits zero or more rows per input row (map function from map&reduce paradigm). Function returns record (or array): (value1, value2, value3) I've tried the following: 1) create or replace function t

Re: [GENERAL] table / query as a prameter for PL/pgSQL function

2011-08-08 Thread Ondrej Ivanič
Hi, 2011/8/9 Merlin Moncure : > You have a few of different methods for passing sets between functions. I do not want to pass data between functions. The ideal solution should look like this: select * from my_map_func() > 1) refcursor as David noted.  reasonably fast. however, I find the > 'FETC

[GENERAL] Postgres on SSD

2011-08-09 Thread Ondrej Ivanič
Hi, I'm about to evaluate this SSD card: FusionIO ioDrive Duo [1]. The main reason for this experiment is to see if SSD can significantly improve query performance. So, I have the following questions: - Could you please share your experience with SSD? Any issues? - What needs to be changed at Pos

Re: [GENERAL] Postgres on SSD

2011-08-10 Thread Ondrej Ivanič
Hi, 2011/8/10 Tomas Vondra : > On 10 Srpen 2011, 1:17, Ondrej Ivanič wrote: >> - What needs to be changed at Postgres/Operating system level? The >> obvious one is to change random_page_cost (now: 2) and seq_page_cost >> (now: 4). What else should I look at? > > Are y

Re: [GENERAL] Postgres on SSD

2011-08-11 Thread Ondrej Ivanič
Hi, 2011/8/11 Amitabh Kant : > There have been several discussions for SSD in recent months although not > specific to Fusion IO drives. > > See http://archives.postgresql.org/pgsql-general/2011-04/msg00460.php . You > can search the archives for more such reference. I've read this one several da

Re: [GENERAL] Where to start, graphs and routing.

2011-08-14 Thread Ondrej Ivanič
Hi, On 14 August 2011 20:25, k_b wrote: > Hi. > For learning purpose i would like to make a small database with a small > graph of locations, roads and public transport information. > Then calculate the fastest or cheapest way between two points. > > If we think of a minimal network, as below. >

[GENERAL] Postgres on SSD

2011-08-14 Thread Ondrej Ivanič
Hi, On 12 August 2011 14:57, Greg Smith wrote: > ioDrive hardware is fast at executing all sorts of I/O, but it particularly > excels compared to normal drives with really random workloads. That's what I hope for :). It looks like that ioDrive is 3 to 5 times faster for seq IO comparing to our S

Re: [GENERAL] Postgres on SSD

2011-08-23 Thread Ondrej Ivanič
Hi, On 12 August 2011 14:57, Greg Smith wrote: >> I'm about to evaluate this SSD card: FusionIO ioDrive Duo [1]. The >> main reason for this experiment is to see if SSD can significantly >> improve query performance The result is that FusionIO will help to our queries which was expected. Most of

Re: [GENERAL] Indexes on inheriting tables

2011-08-24 Thread Ondrej Ivanič
Hi, On 25 August 2011 11:17, Toby Corkindale wrote: > Do I need to make sure I re-create every index on every child table I > create? > That would be.. annoying, at best. Yes, it is little bit annoying but I like it. You don't need any index on parent table but you have to create them "manually"

[GENERAL] Sort Method: external merge

2011-08-25 Thread Ondrej Ivanič
Hi, I have several queries in *single* transaction and I want to figure out reasonable work_mem value. Here is the excerpt from "explain plan" -- each query has two sorts: 1) Sort Method: quicksort Memory: 6 324kB Sort Method: quicksort Memory: 1 932 13

Re: [GENERAL] Sort Method: external merge

2011-08-25 Thread Ondrej Ivanič
Hi, On 26 August 2011 00:14, Tom Lane wrote: > =?UTF-8?Q?Ondrej_Ivani=C4=8D?= writes: >> work_mem is set to 4 000 000 kb and I do not understand why few >> queries (3 and 5) used disk and the rest fit were able to data into >> memory. > > The on-disk representation of sort data is quite a bit mo

Re: [GENERAL] How can I merge two tables?

2011-09-01 Thread Ondrej Ivanič
Hi, On 2 September 2011 03:09, Jerry LeVan wrote: > I keep registration numbers for software and login/passwords for > various organizations, etc… > > As time goes by the tables on the various computers get out of > sync. > > Is there an elegant way I can get all of the differences (uniquely) I

Re: [GENERAL] pg_dump with select command

2011-09-11 Thread Ondrej Ivanič
Hi, On 12 September 2011 15:03, Adarsh Sharma wrote: > Today I need some part ( subset ) of some tables to another database to a > remote server. > I need to take backup of  tables after satisfying a select query. > > Is there any option to specify query in pg_dump command.I researched in the >

Re: [GENERAL] Query performance help with 'shadow table' approach.

2011-09-13 Thread Ondrej Ivanič
Hi, On 14 September 2011 07:44, Brian Fehrle wrote: > 2. I have appropriate indexes where they need to be. The issue is in the > query planner not using them due to it (i assume) just being faster to scan > the whole table when the data set it needs is as large as it is. Try to reduce random_pag

Re: [GENERAL] share nothing but reads cluster

2011-09-15 Thread Ondrej Ivanič
Hi, On 15 September 2011 23:40, Marc Mamin wrote: > Are there other way we should evaluate ? > Should we better wait foir POstgres 9.2+ ? You can try pgpool-II (Parallel Query mode) or MPP database like Greenplum (Community Edition). Another option is high IOPS (500k+) SSD card but they are not

[GENERAL] different unnest function

2011-09-16 Thread Ondrej Ivanič
Hi, I need function which unnest array in a different way. Input table has ineger[][] column: col1 -- {{1,2,3,4}, {5,6,7,8}, {9, 10, 11, 12}} {{11,12,13,14}, {15,16,17,18}, {19, 110, 111, 112}} ... and output should be: select unnest2(col1) from T unnest2 - {1,2,3,4} {5,6

Re: [GENERAL] different unnest function [SOLVED]

2011-09-18 Thread Ondrej Ivanič
Hi, > to get the output OP wants, you need to expand and rewrap: > create or replace function unnest2(anyarray) returns setof anyarray AS > $BODY$ > select array(select unnest($1[i:i])) from > generate_series(array_lower($1,1), array_upper($1,1)) i; > $BODY$ > language 'sql'; Yup, this is what I

Re: [GENERAL] Dynamic constraint names in ALTER TABLE

2011-09-19 Thread Ondrej Ivanič
Hi, On 20 September 2011 13:09, patrick keshishian wrote: > e.g., ALTER TABLE sales DROP CONSTRAINT (SELECT conname FROM > pg_constraint JOIN pg_class ON (conrelid=pg_class.oid) WHERE > pg_class.relname='sales' AND conkey[1] = 1 AND contype='f') ; You have to build query in different way: psql

Re: [GENERAL] COLUMNAR postgreSQL ?

2011-09-20 Thread Ondrej Ivanič
Hi, On 20 September 2011 18:16, Simon Riggs wrote: > It would be useful to get some balanced viewpoints on this. I see you > have Alterian experience, so if you are using both it could be > valuable info. I've never heard anyone describe the downsides of > columnar datastores, presumably there ar

Re: [GENERAL] COLUMNAR postgreSQL ?

2011-09-20 Thread Ondrej Ivanič
Hi, 2011/9/21 Tomas Vondra : >> Columnar store is good if: >> - you are selecting less than 60% of the total row size (our table has >> 400 cols and usual query needs 5 - 10 cols) >> - aggregates: count(*), avg(), ... > > Where did those numbers come from? What columnar database are you using? > W

Re: [GENERAL] limitby without orderby

2011-09-22 Thread Ondrej Ivanič
Hi, On 22 September 2011 21:32, Rohan Malhotra wrote: > Hi Gurus, > What is difference between > select * from items order by random() limit 5; > and > select * items limit 5; > my basic requirement is to get random rows from a table, my where clause This one says: give me first five rows which

Re: [GENERAL] Rules going away

2011-09-28 Thread Ondrej Ivanič
Hi, > folks, don't use RULES! use triggers -- and as much as possible, keep > triggers simple, short, and to the point (simple validation, custom > RI, auditing/logging, etc). I like them :). 'DO INSTEAD' rules are great for partitioning so you can insert (or update) to parent table and 'DO INSTE

Re: [GENERAL] plpgsql syntax error

2011-10-10 Thread Ondrej Ivanič
Hi, On 10 October 2011 21:35, József Kurucz wrote: > ERROR:  syntax error at or near "$1" > LINE 1: create table  $1  ( ) >                      ^ > QUERY:  create table  $1  ( ) > CONTEXT:  SQL statement in PL/PgSQL function "check_table" near line 22 I think you have to use "execute": execute

Re: [GENERAL] how to save primary key constraints

2011-10-11 Thread Ondrej Ivanič
Hi, On 12 October 2011 08:16, J.V. wrote: > I need to be able to query for all primary keys and save the table name and > the name of the primary key field into some structure that I can iterate > through later. psql -E is your friend here. Then use \d and you get several internal queries like

Re: [GENERAL] Drill-downs and OLAP type data

2011-10-12 Thread Ondrej Ivanič
Hi, On 12 October 2011 14:50, Anthony Presley wrote: > After a few weeks of searching around, we're running into dead-ends on the > front-end, and the back-end.  PG doesn't support OLAP / MDX and the GUI > tools that do this, for the most part, require MDX / OLAP (SPSS and DB2, MS > SQL Anal

Re: [GENERAL] Drill-downs and OLAP type data

2011-10-12 Thread Ondrej Ivanič
Hi, > The *problem* with Greenplum is that it's ultra-expensive once you leave the > CE version - and you're not supposed to be using the CE version for > commercial usage last I read the license.  Has that changed? Not sure. I haven't seen something like that in the license. After POC we bought

Re: [GENERAL] postgres maintenance db

2012-07-26 Thread Ondrej Ivanič
Hi, On 27 July 2012 08:07, hartrc wrote: > What is the purpose of the postgres database? I try and drop it and get > "maintenance database can't be dropped" error. 'postgres' database is something like 'mysql' database in MySQL. You should be able to see additional database like 'template0' and

[GENERAL] new material for "Postgres for MySQL users" presentation

2012-07-30 Thread Ondrej Ivanič
Hi, Several years ago I gave this presentation to a bunch of PHP developers in order to show then that something else is out there :). Presentation was based on MySQL 4.1 and Postgres 8.2/8.3 (early 2008). I would like to do it again and I'm looking for sources which can I re-use (and credit back

Re: [GENERAL] success with postgresql on beaglebone

2012-08-16 Thread Ondrej Ivanič
Hi, On 17 August 2012 07:14, Tomas Hlavaty wrote: > thanks for your reply. I should have mentioned that I was using the > Ångström Distribution where postgresql is not provided via package > manager. I wonder how did the Ubuntu guys managed to overcome the > insufficient memory limitation? You

[GENERAL] count number of concurrent requests

2012-08-18 Thread Ondrej Ivanič
Hi, I have the following table: dwh=> \d events Table "public.events" Column |Type | Modifiers --+-+--- datetime | timestamp without time zone | request_duration | integer

Re: [GENERAL] How hard would a "path" operator be to implement in PostgreSQL

2012-08-19 Thread Ondrej Ivanič
Hi, On 20 August 2012 11:28, Chris Travers wrote: > I have been reading up on object-relational features of Oracle and DB2 and > found that one of the big things they have that we don't is a path operator. > The idea is that you can use the path operator to follow some subset of > foreign keys ca

Re: [GENERAL] redundant fields in table for "performance optimizations"

2012-08-21 Thread Ondrej Ivanič
Hi, On 22 August 2012 07:07, Menelaos PerdikeasSemantix wrote: > Let's say you have a father-child (or master-detail if you wish) hierarchy > of tables of not just 2 levels, but, say, 5 levels. > E.g. tables A, B, C, D and E organized in successive 1-to-N relationships: > > A 1-to-N-> B

Re: [GENERAL] Statistical aggregates with intervals

2012-08-23 Thread Ondrej Ivanič
Hi, On 24 August 2012 07:39, Christopher Swingley wrote: > I don't know why, but you could convert 'interval' into something else > where all the functions work: > > CREATE OR REPLACE FUNCTION interval_to_seconds(interval) > RETURNS double precision AS $$ > SELECT (extract(days from $1) * 864

Re: [GENERAL] Some thoughts on table inheritance (which is uniquely awesome on PostgreSQL)

2012-08-23 Thread Ondrej Ivanič
Hi, On 24 August 2012 11:44, Chris Travers wrote: > One thing I have found looking through Oracle and DB2 docs is that > their table inheritance seems to have all the same problems as ours > and their solutions to these problems seem rather broken from a > pure relational perspective. I can

Re: [GENERAL] At what point does a big table start becoming too big?

2012-08-23 Thread Ondrej Ivanič
Hi, On 23 August 2012 23:37, Bill Moran wrote: > > And the advice I have along those lines is to establish now what > constitutes unacceptable performance, and put some sort of monitoring > and tracking in place to know what your performance degradation looks > like and predict when you'll have t

Re: [GENERAL] "Too far out of the mainstream"

2012-09-04 Thread Ondrej Ivanič
Hi, On 5 September 2012 12:14, Chris Travers wrote: > So people are using PostgreSQL in roles that aren't very visible anyway, > DBA's are usually coming to PostgreSQL from other RDBMS's, and few > applications are really distributed for PostgreSQL. > > Not only > this but there was significan

Re: [GENERAL] CentOS initd Script

2012-09-12 Thread Ondrej Ivanič
Hi, On 12 September 2012 16:41, Kenaniah Cerny wrote: > In the service script that gets installed to /etc/rc.d/init.d/, there is a > hard-coded value for PGPORT. Would it be possible to have this variable and > the corresponding -p flag set when calling postgres removed? My init.d script has the

Re: [GENERAL] should I increase default_statistics_target

2012-09-20 Thread Ondrej Ivanič
Hi, On 20 September 2012 20:49, AI Rumman wrote: > Using explain analyze of a large query I found that in every step there are > a lot difference between the number of rows between actual and estimated. > I am using default_statistics_target 200. Should I increase it? I would keep it at default

Re: [GENERAL] PostgreSQL, OLAP, and Large Clusters

2012-09-26 Thread Ondrej Ivanič
Hi, On 26 September 2012 21:50, Ryan Kelly wrote: > The size of our database is growing rather rapidly. We're concerned > about how well Postgres will scale for OLAP-style queries over terabytes > of data. Googling around doesn't yield great results for vanilla > Postgres in this application, but

Re: [GENERAL] PostgreSQL, OLAP, and Large Clusters

2012-09-27 Thread Ondrej Ivanič
Hi, On 28 September 2012 04:34, Ryan Kelly wrote: > On Thu, Sep 27, 2012 at 08:51:31AM +1000, Ondrej Ivanič wrote: >> - aggregation job ran every 15 minutes and completed under 2 minutes: >> 5mil rows -> aggregation -> 56 tables > 5mil overall, or matching your aggregatio

Re: [GENERAL] transitive pruning optimization on the right side of a join for partition tables

2012-09-30 Thread Ondrej Ivanič
Hi, On 30 September 2012 16:36, Waldo, Ethan wrote: > Once again I reiterate that I don't have control over the query construction > and I am currently running postgresql 9.1.5. My question is, does > postgresql support transitive pruning optimization on the right side of a > join for partition

Re: [GENERAL] transitive pruning optimization on the right side of a join for partition tables

2012-09-30 Thread Ondrej Ivanič
Hi, On 30 September 2012 16:36, Waldo, Ethan wrote: > My question is, does > postgresql support transitive pruning optimization on the right side of a > join for partition tables? If so, how do I get that to work? If not, are > there plans for this and when should a release with this feature be

Re: [GENERAL] transitive pruning optimization on the right side of a join for partition tables

2012-09-30 Thread Ondrej Ivanič
Hi, On 30 September 2012 21:00, Waldo, Ethan wrote: > Yeah, I actually saw that paper but couldn't find a date on it. Currently > their techniques are well outside > of the scope of my current problem particularly in consideration that I could > switch to MySQL which does support > the right s

Re: [GENERAL] transitive pruning optimization on the right side of a join for partition tables

2012-09-30 Thread Ondrej Ivanič
Hi, On 1 October 2012 01:14, Tom Lane wrote: > "Waldo, Ethan" writes: >> This query does a sequence scan and append across all the partition tables: >> select "dates"."date_description" FROM "myfact" as "myfact", "dates" as >> "dates" where "myfact"."recorded_on_id" = "dates"."recorded_on_id" a

Re: [GENERAL] Can I force a query plan to materialise part?

2012-10-01 Thread Ondrej Ivanič
Hi, On 2 October 2012 12:33, Toby Corkindale wrote: > I have a query that joins two views, and takes 28 seconds to run. > However if I create temporary tables that contain the contents of each view, > and then join them, the total time is 1.3 seconds. try "offset 0" (or you can tweak statistics

Re: [GENERAL] Can I force a query plan to materialise part?

2012-10-01 Thread Ondrej Ivanič
Hi, On 2 October 2012 13:28, Toby Corkindale wrote: >>> I have a query that joins two views, and takes 28 seconds to run. >>> However if I create temporary tables that contain the contents of each >>> view, >>> and then join them, the total time is 1.3 seconds. >> >> >> try "offset 0" (or you can

Re: [GENERAL] Index only scan

2012-10-10 Thread Ondrej Ivanič
Hi, On 10 October 2012 23:03, Lars Helge Øverland wrote: > We are now in the process of designing a new component for analytics > and this feature got me thinking we could utilize postgres over other > alternatives like column-oriented databases. Basically we will have a > wide, denormalized tabl

Re: [GENERAL] moving from MySQL to pgsql

2012-10-10 Thread Ondrej Ivanič
Hi, On 10 October 2012 19:47, Vineet Deodhar wrote: > 3) Can I simulate MySQL's TINYINT data-type (using maybe the custom data > type or something else) What do you exactly mean? Do you care about storage requirements or constraints? The smallest numeric type in postgres is smallint: range is +/

Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-10-12 Thread Ondrej Ivanič
Hi, On 13 October 2012 01:44, Chitra Creta wrote: > I currently have a table that is growing very quickly - i.e 7 million > records in 5 days. This table acts as a placeholder for statistics, and > hence the records are merely inserted and never updated or deleted. > > Many queries are run on thi

Re: [GENERAL] alter view foo set () -- fixed in 9.2 stable, but when will it be released?

2012-11-04 Thread Ondrej Ivanič
Hi, On 5 November 2012 08:39, Chris Angelico wrote: > On Sat, Nov 3, 2012 at 9:15 AM, Joe Van Dyk wrote: > Point of random curiosity: The commit mentioned adds the following line: > > if (rinfo->reloptions && strlen(rinfo->reloptions) > 0) > > Is there a reason this isn't done as: > > if (rinfo-

Re: [GENERAL] How to list all schema names inside a PostgreSQL database through SQL

2012-11-15 Thread Ondrej Ivanič
Hi, On 15 November 2012 23:31, Xiaobo Gu wrote: > How can I list all schema names inside a PostgreSQL database through > SQL, especially thoese without any objects created inside it. Use -E psql's option: -E, --echo-hiddendisplay queries that internal commands generate then you get S

Re: [GENERAL] About aggregates...

2012-11-29 Thread Ondrej Ivanič
Hi, On 30 November 2012 08:06, Michael Giannakopoulos wrote: > However an aggregate function > feeds me one a tuple for each call, but I would like to have access to a > batch of tuples per function call. Is there any possible way to perform > something like this? Yes, this might be good for you

Re: [GENERAL] Ubutu 12.04 and PostgreSQL9.2.1

2012-12-06 Thread Ondrej Ivanič
Hi, On 7 December 2012 14:17, a...@hsk.hk wrote: > I have questions about Linux Write cache sizing: > > 1) /proc/sys/vm/dirty_ratio : current value (default) 20 > 2) /proc/sys/vm/dirty_background_ratio: current value (default) 10 > > I am using Ubuntu 12.04 (8GB RAM) and PostgreSQL 9.2.1, what

Re: [GENERAL] Problem with aborting entire transactions on error

2012-12-10 Thread Ondrej Ivanič
Hi, On 11 December 2012 06:25, Виктор Егоров wrote: > On the other hand, it is possible to write "whenever sqlerror > continue;" and this will make ORACLE to process all the statements > inide the script, ignoring all errors. This is a general feature, > available not only for sqlplus scripts — a

Re: [GENERAL] large database

2012-12-10 Thread Ondrej Ivanič
Hi, On 11 December 2012 07:26, Mihai Popa wrote: > First, the project has been started using MySQL. Is it worth switching > to Postgres and if so, which version should I use? You should to consider several things: - do you have in-depth MySQL knowledge in you team? - do you need any sql_mode "fe

Re: [GENERAL] PostgreSQL contrib 9.2.x

2012-12-14 Thread Ondrej Ivanič
Hi, On 14 December 2012 17:56, a...@hsk.hk wrote: > I could see that it would install older PostgreSQL 9.1 and > postgresql-contrib-9.1. As I already have 9.2.1 and do not want to have > older version 9.1 in parallel, I aborted the apt install. > > How can I get pure postgresql-contrib for Postg

Re: [GENERAL] Regarding the shared disk fail over cluster configuration

2011-10-31 Thread Ondrej Ivanič
Hi, On 31 October 2011 23:33, Debasis Mishra wrote: > RHEL HA clustering is configured to have zero downtime. So if primary server > is down then HeartBeat will bring secondary server online. By "RHEL HA clustering" do you mean RedHat cluster suite? RHCS uses SIGTERM and then kill -9 after 30 se

Re: [GENERAL] Regarding the shared disk fail over cluster configuration

2011-11-01 Thread Ondrej Ivanič
Hi, On 2 November 2011 02:00, Debasis Mishra wrote: > Thanks a lot for your replay. I just wanna know whether it is required for > me to run initdb or setting the PGDATA environment variable is enough? Master needs to be properly initialised & configured - install postgres - run initdb - install

[GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-07 Thread Ondrej Ivanič
Hi, I have simple question (I think which is not easy to answer): why Postgres is so slow comparing to other Postgres based MPP products (even on the same box in single node configuration)? I'm mot talking about multi node setup; all benchmarks were done on single box (CentOS 5.5, 16 cores, 80GB

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-07 Thread Ondrej Ivanič
Hi, On 8 November 2011 16:58, Craig Ringer wrote: > Which one(s) are you referring to? In what kind of workloads? > > Are you talking about Greenplum or similar? Yes, mainly Geenplum and nCluster (AsterData). I haven't played with gridSQL and pgpool-II's parallel query mode too much. Queries are

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-08 Thread Ondrej Ivanič
Hi, >>  mostly heavy read >> workloads but OLTP performance is required (like run query over 100m+ >> dataset in 15 sec) > > that isn't OLTP, its OLAP.  Online Analytic Processing rather than Online > Transaction Processing   large complex reporting queries that have to > aggregate many rows is

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-08 Thread Ondrej Ivanič
Hi, 2011/11/8 Craig Ringer : > "Spreads reads too much" ? > > Are you saying there's too much random I/O? Is it possible it'd benefit from > a column store? > When you're using Greenplum are you using "Polymorphic Data Storage" column > storage "WITH (orientation=column)" ? yes, exactly. Column s

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-08 Thread Ondrej Ivanič
Hi, 2011/11/8 Tomas Vondra : > Sure you did - you've stated that "mostly heavy read > workloads but OLTP performance is required (like run query over 100m+ > dataset in 15 sec)." That clearly mentions OLTP  ... Whatever :) Let's make it clear: I need to run aggregates/roll ups/drill downs on larg

Re: [GENERAL] IP range in pg_hba.conf?

2011-11-08 Thread Ondrej Ivanič
Hi, On 8 November 2011 22:33, Chrishelring wrote: > I want to exclude access to our postgresql db using a configuration in the > pg_hba.conf file. I have a range of IP adress that should have access, but > how do I do that? > > The range is 10.17.64.1 - 10.17.79.254 (eg. 255.255.240.0 as subnet).

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-08 Thread Ondrej Ivanič
Hi, On 9 November 2011 04:53, John R Pierce wrote: > On 11/08/11 1:49 AM, Ondrej Ivanič wrote: >> >> Greenplum or Postgres + Fusion IO can deliver this performance for us. > > then, thats your answer!   it ain't free, oh well. FusionIO is little bit problematic: smalle

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-08 Thread Ondrej Ivanič
Hi, > it's a lot of work and right now the only people > who've done that work aren't giving it away for free - or not in any form > that can be integrated into PostgreSQL without removing other capabilities > other users need. One MPP vendor implemented columnar store in roughly six months -- lo

Re: [GENERAL] Learning to rephrase equivalent queries?

2011-11-10 Thread Ondrej Ivanič
Hi, On 11 November 2011 00:04, Jay Levitt wrote: > Sometimes the planner can't find the most efficient way to execute your > query. Thanks to relational algebra, there may be other, logically > equivalent queries that it DOES know how to optimize. > > But I don't know relational algebra.  yet.  (

Re: [GENERAL] Restore db

2011-11-13 Thread Ondrej Ivanič
Hi, On 14 November 2011 11:09, Alexander Burbello wrote: > What can I do to tune this database to speed up this restore?? > My current db parameters are: > shared_buffers = 256MB > maintenance_work_mem = 32MB You should increase maintenance_work_mem as much as you can. full_page_writes, archive_

Re: [GENERAL] Table Design question for gurus (without going to "NoSQL")...

2011-11-20 Thread Ondrej Ivanič
Hi, On 21 November 2011 00:33, Phoenix Kiula wrote: > Hi. > > Want to start another thread, loosely related to the performance > problems thread I have going. > > Need some DB design guidance from the gurus here. > > My big table now has about 70 million rows, with the following columns: You can

Re: [GENERAL] Exporting 1 text column from all rows from a table to a file per row?

2011-11-21 Thread Ondrej Ivanič
Hi, On 22 November 2011 06:10, Joost Kraaijeveld wrote: > Is it possible, and if so how, to export a single column of a table into > a separate file per row? I have a table with ~21000 rows that have a > column "body1" containing ASCII text and I want to have 21000 separate > ASCII files, each co

Re: [GENERAL] autovacuum stuck on a table for 18+ hours, consuming lots of CPU time

2011-11-22 Thread Ondrej Ivanič
Hi, On 23 November 2011 13:20, Lonni J Friedman wrote: >  I investigated, and found that for the past ~18 hours, > there's one autovacuum process that has been running, and not making > any obvious progress: snip... > I'm using the defaults for all the *vacuum* options in > postgresql.conf, exc

Re: [GENERAL] Hope for a new PostgreSQL era?

2011-12-11 Thread Ondrej Ivanič
Hi, On 12 December 2011 15:39, Jayadevan M wrote: > At the db level, Oracle provides "Database replay" feature. that lets you > replay the production server events in the development/test environment. > http://docs.oracle.com/cd/B28359_01/server.111/e12253/dbr_intro.htm > Won't something like thi

Re: [GENERAL] checking for table bloat

2011-12-28 Thread Ondrej Ivanič
Hi > As I'm sure many people know, check_postgres.pl has a wonderful (if rather > arcane) query to check table bloat, which has been copied all over the > intarwebz. When I try to use this query one one of my databases I'm told my > table (which has had no deletes) is wasting a whole lot of bytes,

Re: [GENERAL] Verifying a timestamp is null or in the past

2012-01-02 Thread Ondrej Ivanič
Hi, On 2 January 2012 03:26, Raymond O'Donnell wrote: >> And also - does PERFORM works with FOUND? > > Not sure what you mean - can you elaborate? No, perform (and execute) doesn't populate 'found' variable: http://www.postgresql.org/docs/9.0/static/plpgsql-statements.html#PLPGSQL-STATEMENTS-DIA

Re: [GENERAL] Verifying a timestamp is null or in the past

2012-01-02 Thread Ondrej Ivanič
Hi 2012/1/3 David Johnston : > On Jan 2, 2012, at 16:46, Ondrej Ivanič wrote: > Yes, PERFORM does populate FOUND. > > From the documentation you just linked to > > A PERFORM statement sets FOUND true if it produces (and discards) one or > more rows, false if no row i

Re: [GENERAL] Single Table Select With Aggregate Function

2012-01-03 Thread Ondrej Ivanič
Hi, On 4 January 2012 10:26, Rich Shepard wrote: > select max(quant), site, sampdate from chemistry where stream = 'SheepCrk' > and param = 'TDS' group by site, sampdate; > > but this gives me the value of each site and date, not the maximum for all > dates at a specific site. Postgres tells me t

Re: [GENERAL] Time to move table to new tablespace

2012-01-08 Thread Ondrej Ivanič
Hi, On 8 January 2012 01:52, Jason Buberel wrote: > psql> create tablespace 'newstorage' location '/some/new/path'; > psql> alter table city_summary set tablespace = 'newstorage'; Be aware that you are not going to move indexes (see ALTER INDEX name SET TABLESPACE tablespace_name). Maybe you don

Re: [GENERAL] Side effects of moving an index to a new tablespace

2012-01-09 Thread Ondrej Ivanič
Hi, On 10 January 2012 09:16, Jason Buberel wrote: > We have lots of them, they are much smaller than the tables, and that will > allow us to do the migrations more incrementally. In your case I would keep data and indexes on different table spaces (and lower random_page_cost). > One area where

Re: [GENERAL] Time to move table to new tablespace

2012-01-09 Thread Ondrej Ivanič
Hi, On 10 January 2012 06:10, Jason Buberel wrote: > "Select median price for every zip code as of 2012-01-06" (customer exports) > "Select median price for 94086 from 2005-01-01 through 2012-01-06" (charting > apps) > > So by partitioning in one dimension we impact queries in the other. I do no

Re: [GENERAL] self referencing table.

2012-01-17 Thread Ondrej Ivanič
Hi, On 18 January 2012 11:31, David Salisbury wrote: > > I've got a table: > > Taxa > Column |Type > +- > id | integer | > parent_id | integer | > taxonomic_ran

Re: [GENERAL] Interval ordering

2012-01-29 Thread Ondrej Ivanič
Hi, On 30 January 2012 09:19, Adam Rich wrote: > desired time, I want to show them the 5 times from the table that are > closest to their > > input.  I expected to do this using abs() like such: > > select mytime from mytable order by abs(usertime-mytime) asc limit 5; > > However, the difference

Re: [GENERAL] Puzzling full database lock

2012-02-01 Thread Ondrej Ivanič
Hi, On 2 February 2012 11:38, Christopher Opena wrote: > We've been running into some very strange issues of late with our PostgreSQL > database(s).  We have an issue where a couple of queries push high CPU on a > few of our processors and the entire database locks (reads, writes, console > canno

Re: [GENERAL] Rules of Thumb for Autovaccum

2012-02-15 Thread Ondrej Ivanič
Hi, On 16 February 2012 01:14, Robert James wrote: > What rules of thumb exist for: > * How often a table needs to be vacuumed? > * How often a table needs to be analyzed? > * How to tune Autovacuum? I prefer to use autovacuum daemon and sets thresholds on per table basis i.e. sets reasonable de

Re: [GENERAL] "canceling autovacuum time"

2012-02-27 Thread Ondrej Ivanič
Hi, On 28 February 2012 11:53, Jameison Martin wrote: > I'm seeing "GMTERROR: canceling autovacuum task" lines in my logs. That's *should* be fine. autovacuum daemon is smart enough to cancel it self when other query needs access to the table. The affected table will be vacuumed/analysed later.

[GENERAL] replication between US <-> EU

2012-03-06 Thread Ondrej Ivanič
Hi, I would like to get some ideas about subject. I do not have any preferred solution (hot-standby, Slony or pgpoll) so anything which can deliver/satisfy the following will good: - one side completely down: Client should use switch to other side transparently (Failover / High Availability) - d

Re: [GENERAL] replication between US <-> EU

2012-03-06 Thread Ondrej Ivanič
Hi, On 7 March 2012 10:36, John R Pierce wrote: > On 03/06/12 3:31 PM, Ondrej Ivanič wrote: >> >> - one side completely down: Client should use switch to other side >> transparently (Failover / High Availability) > > > what happens if the link between the sites

Re: [GENERAL] Synchronous replication + Fusion-io = waste of money OR significant performance boost? (compared to normal SATA-based SSD-disks)?

2012-03-08 Thread Ondrej Ivanič
Hi, On 9 March 2012 02:23, dennis jenkins wrote: > I've also looked at the Fusion-IO products.  They are not standard > flash drives.  They don't appear as SATA devices.  They contains an > FPGA that maps the flash directly to the PCI bus.  The kernel-mode > drivers blits data to/from them via DM

Re: [GENERAL] how to return the last inserted identity column value

2012-03-08 Thread Ondrej Ivanič
Hi, On 9 March 2012 05:20, Scott Marlowe wrote: > On Thu, Mar 8, 2012 at 11:16 AM,   wrote: >> In some languges you can use set l_localid = @@identity which returns >> the value of the identity column defined in the table.  How can I do >> this in Postgres 9.1 > > Assuming you created a table lik

Re: [GENERAL] COPY and indices?

2012-03-12 Thread Ondrej Ivanič
Hi, On 13 March 2012 15:11, François Beausoleil wrote: > When using COPY FROM STDIN to stream thousands of rows (20k and more hourly), > what happens with indices? Are they updated only once after the operation, or > are they updated once per row? Note that I'm not replacing the table's data:

Re: [GENERAL] PostgreSQL 64 Bit XIDs - Transaction IDs

2012-03-23 Thread Ondrej Ivanič
Hi, On 23 March 2012 19:14, Frank Lanitz wrote: > Am 23.03.2012 06:45, schrieb Gerhard Wiesinger: >> With a database admin of a commercial database system I've discussed >> that they have to provide and they also achieve 2^31 transactions per >> SECOND! > > Just corious: What is causing this many

Re: [GENERAL] Versioned, chunked documents

2012-04-01 Thread Ondrej Ivanič
Hi, On 2 April 2012 08:38, Ivan Voras wrote: > db=> set enable_seqscan to off; > > This huge cost of 100 which appeared out of nowhere in the > EXPLAIN output and the seq scan worry me - where did that come from? It is not possible to disable seq scan completely. The "enable_seqscan

Re: [GENERAL] Searchable chess positions in a Postgress DB

2012-04-11 Thread Ondrej Ivanič
Hi, On 11 April 2012 17:15, Sidney Cadot wrote: > I have written code to extract these positions, and now I want to put > them into a Postgres database. Specifically, I want to do this in a > way that allows *fast* lookups of positions, e.g. "give me all > positions that have a White King on c4 a

[GENERAL] remove some rows from resultset

2012-04-18 Thread Ondrej Ivanič
Hi, I have the following table: org_id | contract_name | org_specific_rule | count --+--+---+--- smpj28p2 | Group 123| f | 3 smpj28p2 | Group 2 | f | 3 smpj28p2 | Group 2 | t