On Thursday, May 13, 2010 11:51:08 pm Brian Modra wrote:
> Maybe the best way to solve this is not to do automatic distribution
> of the data, but rather to provide tools for implementing distributed
> references and joins.
Here's my vote! I'd *LOVE* it if I could do a simple cross-database join
On 14/05/2010, Bruce Momjian wrote:
> Brian Modra wrote:
>> Hi,
>> I've been told that PostgreSQL and other similar databases don't work
>> well on a parallelised operating system because they make good use of
>> shared memory which does not cross the boundary between nodes in a
>> cluster.
>>
>>
Brian Modra wrote:
> Hi,
> I've been told that PostgreSQL and other similar databases don't work
> well on a parallelised operating system because they make good use of
> shared memory which does not cross the boundary between nodes in a
> cluster.
>
> So I am wondering if any work is being done t
> There was an
> attempt to fix it (for example so that it could try to do a
> seqscan+sort
> instead of indexscan), but it stalled.
Actually I read that, but it's complicated... it involves planning and a lot of
other stuff I don't even know about...
My "solution" I guess would be easier (but,
Scara Maccai wrote:
> I mean: there's access exclusive lock on the table while clustering,
> so I don't see any problem in doing it... this way you could
>
> - avoid sorting (which is what is used in the method "create newtable
> as select * from oldtable order by mycol", and can be slow with 15
> I've found it easier to select everything into another
> table, truncate
> the original table, then insert the rows as:
that takes 50 seconds of pure sorting and 8GB of ram to sort; my method doesn't
require more memory than the size of the heap table, and no sorting, since the
index is alread
On Thu, Aug 20, 2009 at 8:28 AM, Scara Maccai wrote:
> Hi,
>
> I have a table with 15M rows. Table is around 5GB on disk.
>
> Clustering the table takes 5 minutes.
>
> A seq scan takes 20 seconds.
>
> I guess clustering is done using a seq scan on the index and then fetching
> the proper rows in t
On Wed, Jun 18, 2008 at 9:26 AM, Decibel! <[EMAIL PROTECTED]> wrote:
> On Jun 17, 2008, at 11:37 AM, Scott Ribe wrote:
>
>> BOOM! Deadlock.
>>>
>>
>> No more likely than with the current cluster command. Acquiring the lock
>> is
>> the same risk; but it is held for much less time.
>>
>
>
> Actuall
On Jun 17, 2008, at 11:37 AM, Scott Ribe wrote:
BOOM! Deadlock.
No more likely than with the current cluster command. Acquiring the
lock is
the same risk; but it is held for much less time.
Actually, no (at least in 8.2). CLUSTER grabs an exclusive lock
before it does any work meaning t
> BOOM! Deadlock.
No more likely than with the current cluster command. Acquiring the lock is
the same risk; but it is held for much less time.
> ...I think what makes a lot
> more sense is to have a form of clustering that puts effort into
> placing tuples in the correct location.
Agreed that w
On May 28, 2008, at 11:21 AM, Scott Ribe wrote:
If I'm not totally off-base, here's one way to enable clustering on
systems
that run 24/7:
1 cluster current rows
1.1 note current last committed transaction
1.2 copy all visible rows to new table in cluster order
1.3 build indexes on
Tom Lane wrote:
Anyway I think the main practical problem would be with deadlocks
against other transactions trying to update/delete tuples at the same
times you need to move them. Dealing with uncommitted insertions would
be tricky too --- I think you'd need to wait out the inserting
transacti
Craig Ringer <[EMAIL PROTECTED]> writes:
> Begin a transaction and free the first chunk (2 tuples in this case, but
> obviously many more in a real case):
> ---
> ..473612058
> ---
> Use that freed space to store the first ordered tuples:
> ---
> 014736.2.58
> --
Tom Lane wrote:
Craig Ringer <[EMAIL PROTECTED]> writes:
Later on, though, less new space would have to be allocated because more
and more of the space allocated earlier to hold moved tuples would be
being freed up in useful chunks that could be reused.
I don't see how that works. If the min
Craig Ringer <[EMAIL PROTECTED]> writes:
> Later on, though, less new space would have to be allocated because more
> and more of the space allocated earlier to hold moved tuples would be
> being freed up in useful chunks that could be reused.
I don't see how that works. If the minimum size of
Tom Lane wrote:
Craig Ringer <[EMAIL PROTECTED]> writes:
So ... is this crazy? Concurrently clustering the table by moving each
record *twice*, in batches, with pauses to allow old versions to cease
being visible by any live transaction? Or can it actually work?
It seems to me you'd have a pr
Craig Ringer <[EMAIL PROTECTED]> writes:
> So ... is this crazy? Concurrently clustering the table by moving each
> record *twice*, in batches, with pauses to allow old versions to cease
> being visible by any live transaction? Or can it actually work?
It seems to me you'd have a pretty horrid b
Scott Ribe wrote:
Wouldn't new / updated tuples just get put in the hole, fairly rapidly
un-clustering the table again?
How is that different than putting them in newly-allocated space at the end
of the table?
It isn't, I just wasn't thinking straight.
This is probably a stupid idea
> Huh? A plain vacuum wouldn't fix that; a vacuum full would close up the
> hole, but (a) it'd not preserve the row ordering, and (b) it'd take an
> exclusive lock.
OK.
--
Scott Ribe
[EMAIL PROTECTED]
http://www.killerbytes.com/
(303) 722-0567 voice
--
Sent via pgsql-general mailing list (p
Scott Ribe <[EMAIL PROTECTED]> writes:
>> Huh? If I'm understanding you correctly you'll end up with rows in
>> order, but with a really big hole in the middle of the table. I'm not
>> sure if that qualifies as "clusters".
> That's why he said vacuum when done.
Huh? A plain vacuum wouldn't fix t
> Wouldn't new / updated tuples just get put in the hole, fairly rapidly
> un-clustering the table again?
How is that different than putting them in newly-allocated space at the end
of the table?
--
Scott Ribe
[EMAIL PROTECTED]
http://www.killerbytes.com/
(303) 722-0567 voice
--
Sent via pgs
Scott Ribe wrote:
Huh? If I'm understanding you correctly you'll end up with rows in
order, but with a really big hole in the middle of the table. I'm not
sure if that qualifies as "clusters".
That's why he said vacuum when done. Anyway, I'm not sure that a big
*contiguous* hole in the mid
> Huh? If I'm understanding you correctly you'll end up with rows in
> order, but with a really big hole in the middle of the table. I'm not
> sure if that qualifies as "clusters".
That's why he said vacuum when done. Anyway, I'm not sure that a big
*contiguous* hole in the middle of the table wou
On Fri, May 2, 2008 at 9:12 AM, fschmidt <[EMAIL PROTECTED]> wrote:
>
> An implementation of clustering without locking would start by comparing the
> index to the table from the beginning to find the first mismatch. Rows
> before the mismatch are fine, and can be left alone. From here on, go
On Thu, May 01, 2008 at 05:12:52PM -0700, fschmidt wrote:
>
> An implementation of clustering without locking would start by comparing the
> index to the table from the beginning to find the first mismatch. Rows
> before the mismatch are fine, and can be left alone. From here on, go
> through th
On 8/4/07, hanasaki <[EMAIL PROTECTED]> wrote:
> clustering failover... ala Oracle Parallel server
Note that OPS is now called RAC (see http://orafaq.com/faq/what_is_rac_ops).
> How can the server be setup in a cluster for load-balancing and failover
> like perhaps OPS?
As I understand it, RAC i
On Tue, 26 Dec 2006, Ben wrote:
On Dec 26, 2006, at 7:30 AM, Marc Evans wrote:
What I have not been able to come up with a good semi-general purpose
solution to is cross-data-center HA.
Why does log shipping not work for you?
Well, it may, but is short, I believe that this comes down to
On Dec 26, 2006, at 7:30 AM, Marc Evans wrote:
What I have not been able to come up with a good semi-general
purpose solution to is cross-data-center HA.
Why does log shipping not work for you?
---(end of broadcast)---
TIP 1: if posting/readi
On Tue, 26 Dec 2006, Shoaib Mir wrote:
We are trying to achieve High Availability over load balancing, so
basically we always try and have 2 databases in the same state while >>both
are active,
What problems do you see with Slony + Linux HA combo there? I think a Slony
failover can do the sam
hoaib Mir
> > > > > > > EnterpriseDB (www.enterprisedb.com)
> > > > > > >
> > > > > > > On 12/25/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]>
> > > > > > > wrote:
> > > > > > > >
> > >
On Tue, 2006-12-26 at 15:11 +, Andy Dale wrote:
> ...
You guys please avoid top-posting.
--
The PostgreSQL Company - Command Prompt, Inc. 1.503.667.4564
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Managed Services, Shared and Dedicated Hosting
Co-Authors: plPHP, plP
12/25/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> > > > > > >
> > > > > > > Thanks Chris,
> > > > > > > I see you a core member of Slony team and a replication guru
> > > > > > > so I'll look
> >
hanks Chris,
> > > > > > I see you a core member of Slony team and a replication guru
> > > > > > so I'll look
> > > > > > into it.
> > > > > > I'm not slamming Slony I think its p
ably the right tool for
> > > > > type of work
> > > > > your company Afilias does. Just wish you would make an official
> > > > > Windows
> > > > > version of Slony as well.
> > > > > Anyway thanks for the education, and I
t; > > > version of Slony as well.
> > > > Anyway thanks for the education, and I think it would be a good
> > > > thing if
> > > > your site on replication, was also listed on Postgresql... good
> > > > research.
> > > > Merry Xmas
gt; Windows
> > > version of Slony as well.
> > > Anyway thanks for the education, and I think it would be a good
> > > thing if
> > > your site on replication, was also listed on Postgresql... good
> > > research.
> > > Merry Xmas
> &
research.
> > Merry Xmas
> >
> > - Original Message -
> > From: "Christopher Browne" <[EMAIL PROTECTED]>
> > To: < pgsql-general@postgresql.org>
> > Sent: Sunday, December 24, 2006 4:23 AM
> > Subject: Re: [GENERAL] Clustering & Lo
esearch.
> Merry Xmas
>
> - Original Message -
> From: "Christopher Browne" <[EMAIL PROTECTED]>
> To:
> Sent: Sunday, December 24, 2006 4:23 AM
> Subject: Re: [GENERAL] Clustering & Load Balancing & Replication
>
>
> > Centuries ago,
"Christopher Browne" <[EMAIL PROTECTED]>
To:
Sent: Sunday, December 24, 2006 4:23 AM
Subject: Re: [GENERAL] Clustering & Load Balancing & Replication
> Centuries ago, Nostradamus foresaw when [EMAIL PROTECTED] would write:
>> Suggest you download my little applicatio
ks for the education, and I think it would be a good thing if
your site on replication, was also listed on Postgresql... good research.
Merry Xmas
- Original Message -
From: "Christopher Browne" <[EMAIL PROTECTED]>
To:
Sent: Sunday, December 24, 2006 4:23 AM
Subject
Centuries ago, Nostradamus foresaw when [EMAIL PROTECTED] would write:
> Suggest you download my little application and read the documentation,
> you'll see its very different, maybe even interesting.
> Maybe they should change that to Postgres DOES HAVE a free multi-master
> replication system
tastic.
I will use nothing else
Thanks, Johnny
- Original Message -
From: "Bruce Momjian" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Cc:
Sent: Saturday, December 23, 2006 6:34 PM
Subject: Re: [GENERAL] Clustering & Load Balancing & Replication
I assume you have read this new documentation for 8.2:
http://www.postgresql.org/docs/8.2/static/high-availability.html
---
[EMAIL PROTECTED] wrote:
> >Hi,
> >
> >I am currently investigating the preferred method of
I believe there's a TODO item for index-organized tables/clustered
tables. If not, there's certainly been discussion about it on the
-hackers list.
On Sun, Sep 17, 2006 at 10:21:27PM -0700, CG wrote:
> As I'm waiting for a CLUSTER operation to finish, it occurs to me that in a
> lot of cases, the
There are a few options depending on your needs.
PGCluster, Slony and pgpool would be the main ones to look at.
More info on them can be found at http://pgfoundry.org
Searching for replication will find more results than cluster
On 20/9/2006 14:17, "Samad, Alex" <[EMAIL PROTECTED]> wrote:
>
On Tue, Nov 15, 2005 at 08:41:06PM -0500, Bruce Momjian wrote:
> Jim C. Nasby wrote:
> > Well, currently you can only cluster on a single index per table, and I
> > really doubt that will change. In any case, if someone's going to work
> > on clustered indexes I think it would be much more worthwhi
Jim C. Nasby wrote:
> On Tue, Nov 15, 2005 at 10:18:26AM -0800, Steve Crawford wrote:
> > Not trivial? Seems to me more like impossible to implement for the
> > general case which would require you to resolve the situation where
> > someone requests multiple, overlapping, clustered partial indexe
On Tue, Nov 15, 2005 at 10:18:26AM -0800, Steve Crawford wrote:
> Not trivial? Seems to me more like impossible to implement for the
> general case which would require you to resolve the situation where
> someone requests multiple, overlapping, clustered partial indexes
> where the ordering requ
On Tuesday 15 November 2005 10:12, Tom Lane wrote:
> Bruce Momjian writes:
> > Tom Lane wrote:
> >> CLUSTER says "order the table according to the order of the
> >> entries in this index". A partial index doesn't define an
> >> ordering for the whole table, only the rows that have entries in
> >>
Tom Lane wrote:
> Bruce Momjian writes:
> > Tom Lane wrote:
> >> CLUSTER says "order the table according to the order of the entries in
> >> this index". A partial index doesn't define an ordering for the whole
> >> table, only the rows that have entries in that index. So it doesn't
> >> seem to
Bruce Momjian writes:
> Tom Lane wrote:
>> CLUSTER says "order the table according to the order of the entries in
>> this index". A partial index doesn't define an ordering for the whole
>> table, only the rows that have entries in that index. So it doesn't
>> seem to me that you are asking for
Tom Lane wrote:
> "Keith C. Perry" <[EMAIL PROTECTED]> writes:
> > Quoting Tom Lane <[EMAIL PROTECTED]>:
> >> "Keith C. Perry" <[EMAIL PROTECTED]> writes:
> >>> This might have been discussed before but I wanted to know if clustering
> >>> tables by partial indexes will be availble in a later relea
"Keith C. Perry" <[EMAIL PROTECTED]> writes:
> Quoting Tom Lane <[EMAIL PROTECTED]>:
>> "Keith C. Perry" <[EMAIL PROTECTED]> writes:
>>> This might have been discussed before but I wanted to know if clustering
>>> tables by partial indexes will be availble in a later release of pgSQL?
>>
>> What i
Quoting Tom Lane <[EMAIL PROTECTED]>:
> "Keith C. Perry" <[EMAIL PROTECTED]> writes:
> > This might have been discussed before but I wanted to know if clustering
> tables
> > by partial indexes will be availble in a later release of pgSQL?
>
> What in the world would it mean to do that?
I'm not
"Keith C. Perry" <[EMAIL PROTECTED]> writes:
> This might have been discussed before but I wanted to know if clustering
> tables
> by partial indexes will be availble in a later release of pgSQL?
What in the world would it mean to do that?
regards, tom lane
-
On Wednesday June 22 2005 2:16 am, Gregory Youngblood wrote:
> I am looking for some information about clustering and
> replication options for postgresql.
Gregory,
FWIW, I've used Slony 1.0.5 for 10-15 db cluster migrations,
usually from 7.3.4 clusters on one box to 7.4.6 clusters on
another (
On Mon, 2005-05-09 at 13:41, Craig Bryden wrote:
> Hi
>
> In MS SQL there is a concept of Clustering database servers. This allows for
> load balancing.
> Does PostgreSQL have a similar concept?
This really kinda depends on what you are tring to load balance. I/O,
parallel users, CPU intensive
Slony is the master-slave (not multi-master IIRC) replication environment.
I haven't used it, but apparently there are cognoscenti among us who have.
I would think, a bit, about what kind of "load" (e.g. CPU, network, disk
queues, disk allocation, et cetera) you want to balance before picking u
From the title alone, "pgCluster" sounded like the perfect choice for
PostgreSQL clustering.
However on their homepage they provide very little information is a
rough english traslation from what it sounds like. Also:
http://pgcluster.projects.postgresql.org/feature.html
What happens when the lo
> Slony-I isn't clustering. They would have to look at pgCluster or
> something like that.
Sure but it seemed to fit their description of what they
wanted to do.
Karsten
--
GPG key ID E4071346 @ wwwkeys.pgp.net
E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346
---(end o
On Thu, 2005-04-28 at 14:54, Joshua D. Drake wrote:
> Karsten Hilbert wrote:
> > On Thu, Apr 28, 2005 at 11:59:59AM -0700, Patrick Haugen wrote:
> >
> >
> >>Pseudo code:
> >>When databse xyz table companyname is updated update the same table
> >>and rows in database abc;
> >
> > ...
> >
> >>Wha
Karsten Hilbert wrote:
On Thu, Apr 28, 2005 at 11:59:59AM -0700, Patrick Haugen wrote:
Pseudo code:
When databse xyz table companyname is updated update the same table
and rows in database abc;
...
What is a good clustering technique for PostgreSQL?
Slony-I.
Slony-I isn't clustering. They would h
On Thu, Apr 28, 2005 at 11:59:59AM -0700, Patrick Haugen wrote:
> Pseudo code:
> When databse xyz table companyname is updated update the same table
> and rows in database abc;
...
> What is a good clustering technique for PostgreSQL?
Slony-I.
Karsten
--
GPG key ID E4071346 @ wwwkeys.pgp.net
E1
On Thu, 2005-04-28 at 13:59, Patrick Haugen wrote:
> I haven't found any information on clustering with PostgreSQL.
>
> One idea we've been tossing around is through PostgreSQL you can
> create a function that does something when something else happens.
>
> Pseudo code:
> When databse xyz table c
On Sun, Mar 27, 2005 at 06:59:06PM +0100, Julian Scarfe wrote:
> I've got a database (7.4) whose system tables have been long neglected.
> Instead of the 100 or so pages I'd expect for 4000 rows after VACUUM, I've
> got 24,000 pages and a mere 1.4 million unused item pointers.
>
> If it were an
On Thu, Oct 14, 2004 at 10:29:11AM +0530, Nageshwar Rao wrote:
> Hi,
> We would like use Postgresql as our database. For high availability is it
> possible to cluster DB in Postgresql. Appreciate if you can let me know how
> this can be achieved.
> Else is there any other way to achieve High Avail
Google "slony"
Regards,
aaron.glenn
On Thu, 14 Oct 2004 10:29:11 +0530, Nageshwar Rao
<[EMAIL PROTECTED]> wrote:
>
>
>
> Hi,
>
> We would like use Postgresql as our database. For high availability is it
> possible to cluster DB in Postgresql. Appreciate if you can let me know how
> this c
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi,
On Mon, 6 Sep 2004, [ISO-8859-1] Björn Voigt wrote:
has postgresql the functionality for clustering, load balancing
and failover. I have to setup two redundant web-servers with and
want run a dbms cluster on this machines.
My webapps should only see
>
> Does anyone know what's the most reliable platform postgresql can run on?
> With or without scheduled downtime?
This reminded me... Not long ago I was looking at used sun servers. You
can pick up refurbished sun enterprise systems for between $4,000 and
$10,000 fairly easily. For instance
Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] Behalf Of Lincoln Yeoh
> Sent: Friday, August 13, 2004 10:25 AM
> To: Jon Brisbin; Richard Huxton
> Cc: pgSQL General
> Subject: Re: [GENERAL] Clustering, mirroriing, or replication?
>
>
> Most replication syst
Most replication systems add a fair amount of complexity. How reliable are
current replication systems? Are they replication systems for performance
or for reliability+availability?
How much does it cost to make sure that the probability of both master and
failover machines failing is lower or
To: Jon Brisbin
Cc: pgSQL General
Subject: Re: [GENERAL] Clustering, mirroriing, or replication?
Jon Brisbin wrote:
> We're trying to figure out how we can implement a reasonably simple
cluster
> of postgres servers on a private network at our store locations. The idea
> is to have a
Jon Brisbin wrote:
We're trying to figure out how we can implement a reasonably simple cluster
of postgres servers on a private network at our store locations. The idea
is to have a group of 2-6 PCs each be able to share data in a replicated
manner. Having a separate database server is not an opt
Hi, Stan,
We're in the early stages of testing a new Postgres (7.3) cluster. For
background, our database is about 14gb on disk, and we see about a
transaction a second (out of about 120 queries/sec.) Our application
is a large dynamic Apache-based web system, written in Perl. Our main
data
74 matches
Mail list logo
