p-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
Ph: (510) 642-8072
- Original Message -
From: Bruce Momjian <[EMAIL PROTECTED]>
Date: Wednesday, October 8, 2003 12:33 pm
Subject: Re: [HACKERS] Separate shared_buffer management process
>
> Added to TODO:
>
> * Us
+++
We'd love this feature as it would really help us write better test cases !
Regards
Sailesh
--
Sailesh Krishnamurthy
Amalgamated Insight
[W] (650) 242-3503
[C] (650) 804-6585
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Gregory Stark
+++
We'd love this feature as it would really help us write better test cases !
Regards
Sailesh
--
Sailesh Krishnamurthy
Amalgamated Insight
[W] (650) 242-3503
[C] (650) 804-6585
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Gregory Stark
, it's not really close to a release yet ... :-)
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
ecause we're actually implementing
the Eddy operator. But if your goal is just to try out your nice new
join algorithm, this would probably work and be a quick fix to get you
started.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
I have a small nit
Why is it that bit.h is in src/include/utils and bit.c is in
src/backend/lib ?
I can never for the life of me remember which is in which :-)
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP
>>>>> "Tom" == Tom Lane <[EMAIL PROTECTED]> writes:
Tom> Sailesh Krishnamurthy <[EMAIL PROTECTED]> writes:
>> Why is it that bit.h is in src/include/utils and bit.c is in
>> src/backend/lib ?
Tom> Possibly a more interestin
T though, postgres code, such as perhaps
the memory contexts is not MT-safe (of course the bufferpool/shmem
accesses are safe).
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
the newbie question, but if somebody could point me to
what exactly I need to install, given that configure thinks that I do
have docbook-3.1, I'd be obliged. Thanks !
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
.
Is there some magic incantation to be chanted to get this to work ?
Thanks !
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send
Quick question for the group
I'm assuming that the PG_TEMP_FILES_DIR for BufFile temps is
automatically under the PGDATA directory. Is that correct ?
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
>>>>> "Bruce" == Bruce Momjian <[EMAIL PROTECTED]> writes:
Bruce> Tom Lane wrote:
>> Sailesh Krishnamurthy <[EMAIL PROTECTED]> writes:
>> > Sadly, set_ps_display does not seem to have any effect in solaris
>> >
bits are set and some others are reset then
you know that there was a corruption.
This is of course based on the assumption that most disk arms manage
to atomically write 256 bytes at a time.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
book-dsssl-1.77/html/dblink.dsl:203:1:E: XRef
LinkEnd to missing ID 'STON89'
openjade:/usr/local/share/sgml/docbook-dsssl-1.77/html/dblink.dsl:203:1:E: XRef
LinkEnd to missing ID 'STON90b'
gmake: *** [admin.html] Error 1
Does anybody have any suggestions ? Am I doing something horribly
wrong ?
Many thanks !
--
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
e whole process of SGML documentation as separating
content from presentation. So I imagine I should surround our new
stuff with some kind of tag, that when used with an HTML (or other)
stylesheet generates appropriately changed documentation.
I hope I am on the right track ! Could somebody help me ple
Why don't you use elog(LOG, instead of printf ?
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTE
postmaster
Srikanth> for example,pls tell what should i do after i add code,
Srikanth> what make files should i run and where should i place
Srikanth> the new executables.
Well you sould start out by RTFM .. read how to build postgres. I run
"make install" in my pos
web is
based off of 7.2 code, but I hope to refresh with a beta based on 7.3
code in the next few weeks.
http://telegraph.cs.berkeley.edu/telegraphcq
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings
ssion test suite is great. We
have clearing the pgsql regression test a checkin requirement for
TelegraphCQ developers as our goal is to not break pgsql
functionality.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)--
s.
(Although I guess in practice, many people use Presumed Abort and not
vanilla 2PC as PA results in fewer log flushes for read-only
transactions.)
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match
>>>>> "Tom" == Tom Lane <[EMAIL PROTECTED]> writes:
Tom> Sailesh Krishnamurthy <[EMAIL PROTECTED]> writes:
>> I'm not sure if I understand Tom's beef - I think he is
>> concerned about what happens if a subordinate does no
odebase with unnecessary complexity.
Let's just not confuse "no need for XYZ functionality" with "XYZ
functionality is lame and can never work in practice".
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faqs/FAQ.html
be
descending ? It doesn't affect the Group and the Agg in any way ..
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match
hat you lose the modularity of the sort.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faqs/FAQ.html
I need to pass to configure to ensure that all
binaries are built appropriately ? That is, something _other_ than
--enable-debug as I always have that anyways.
Thanks !
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)
yet. We welcome
patches that help in porting TelegraphCQ.
Please send all comments (and patches) to [EMAIL PROTECTED]
--
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(e
Once more unto the breach -
Could you please abstain from sending HTML email to the list ?
Many thanks !
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ
in his
Ph.D. thesis in the mid-eighties for main-memory databases (where you
don't take the write penalty).
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
ves you more concurrency
at some very minor overhead of not being strictly serializable).
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
f the answer comes back
Why do you want to query the lock manager as a separate process ?
Why not have the traditional approach of a lock table in shared
memory, growing and shrinking as appropriate, and have each individual
process update it (need to protect it with a latch of course).
--
Pip-pip
S
>>>>> "Tom" == Tom Lane <[EMAIL PROTECTED]> writes:
Tom> Sailesh Krishnamurthy <[EMAIL PROTECTED]> writes:
>> Why not have the traditional approach of a lock table in shared
>> memory, growing and shrinking as appropriate,
Tom&
>>>>> "Tom" == Tom Lane <[EMAIL PROTECTED]> writes:
Tom> Sailesh Krishnamurthy <[EMAIL PROTECTED]> writes:
>> We implemented a Shared Memory MemoryContext using OSSP libmm
>> (used in Apache) for TelegraphCQ.
Tom> How po
>>>>> "Bruce" == Bruce Momjian <[EMAIL PROTECTED]> writes:
>> -- Pip-pip Sailesh http://www.cs.berkeley.edu/~sailesh
Bruce> ^^^ Watch out, that code from
Bruce> Berkeley usually is a mess. :-)
LOL !
That's
some experiments to measure the costs of our shared
memory MemoryContext.
Is there something specific you'd be interested in ? My first goal is
mainly to measure the amount of time it takes to allocate and
deallocate shared memory in the contested and uncontested cases.
--
P
edicate locking to get true serializability without
losing too much concurrency. While this falls short in the general
case, it turns out to be pretty acceptable normally (when indexes are
present).
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broad
If you read the autovacuum_naptime into an Interval object once, why can't
you just use timestamptz_pl_interval ? You won't be using the interval
input/output repeatedly surely.
Regards
Sailesh
--
Sailesh Krishnamurthy
Amalgamated Insight
[W] (650) 242-3503
[C] (650) 804-6585
---
gt;> for v.7.3.Is there any alternatives for the latest version of PostgreSQL
?
>
> The TelegraphCQ team has stopped public development. So it's
> pretty much waiting for someone to take on their code.
Regards
Sailesh
--
Sailesh Krishnamurthy
Amalgamate
se post on the tcq
mailing list.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so
able
>> space is small, as that the available space is fixed --- we
>> can't easily change it after postmaster start. The more finely
Again, I can suggest the shared memory MemoryContext we use in
TelegraphCQ that is based on the OSSP libmm memory manager. We use it
t
fine (and keep re-merging).
Which brings me to another question .. has anybody considered using
subversion instead of CVS ?
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 8: explain analyze is your friend
tion Tools
I'm at a loss as to how much we should focus on these sections. Do we
use what's in GBorg ? Do the hackers have any suggestions ?
There are 33 DBA tools and 19 Design tools in GBorg .. are there any
specific tools that are "recommended" ?
Suggestions
>>>>> "Marc" == Marc G Fournier <[EMAIL PROTECTED]> writes:
Marc> On Tue, 23 Mar 2004, Sailesh Krishnamurthy wrote:
>> Which brings me to another question .. has anybody considered
>> using subversion instead of CVS ?
Marc> Wh
; also exposure to
scripting languages such as PERL, PHP and their regular expression
libraries. Requires an ability to install and maintain web servers
and services. Exposure to the Database and/or Information Retrieval
research literature is desirable.
-------
is a lock against vaccuuming in the presence of a
tid-list-fetch.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faqs/FAQ.html
after, are how to deal with locking
Tom> considerations and what will be the implications of giving up
Tom> the property that indexscans deliver sorted output.
I don't know about the former, but as to the latter, we should
certainly have the ability for both output sorted b
y, does PG do a
subquery to join transformation ?
Thanks !
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
appen along with the
transformations ?
Alvaro> The query's path is SQL -> parse -> rewrite -> optimize ->
Alvaro> execute
Can you please point me to the code that indeed does such
transformations ?
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
pose after
applying a rule the resultant query tree is eligible for another rule,
does pgsql's rule system keep iterating over and over until it reaches
a fixed point or is there some heuristic in operation (just apply the
rules twice ..) ? From my cursory inspection of the code it looks like
the l
subquery because the upper query uses the
subquery with existential quantification
In general, I'm trying to understand all the transformations that
pgsql will try to do .. I'm not trying to figure out plan enumeration
for basic boxes (simple query tree).
--
Pip-pip
Sailesh
http:/
cular you
Tom> probably want to look at prepjointree.c and prepqual.c.
Tom> (Note prepqual also looks considerably different in CVS tip
Tom> than in prior releases.)
Thanks .. I've extracted cvstip .. sigh .. one of these days I'll have
to do another merge with the Tel
LECT time takes a while. Then they have to
pgsql> vacuum constanty. It just seems like an ugly and wastefull
pgsql> process.
Sounds like something that TelegraphCQ can do well ..
http://telegraph.cs.berkeley.edu
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
hich can do binary search to find an item within a page).
This is probably a crazy idea, but is it possible to organize the data
in a page of a hash bucket as a binary tree ? Then you wouldn't lose
wrt CPU time at least.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
--
crude perf studies and found that it didn't
really help if there was no sharing .. as is the case with pgsql).
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an
Several operating system services are examined with a
view toward their applicability to support of database
management functions. These services include buffer pool
management; the file system; scheduling, process
management, and interprocess communication; an
e it is true that some of XML db technology is evocative
of network databases, XML databases are certainly more than network
databases. For one, they are semi-structure .. in addition they
present query language access to their data (although I'm not a big
fan of XQuery).
--
Pip-pip
Sailesh
http:/
ot admit (at least
in public :-) to being a big fan of XQuery but that is because of
certain details, not anything fundamental.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
v 24 perhaps. I
have a conference deadline .. the papers that we write "just to secure
funding" - your tax dollars at work. Long live the NSF !
BTW, I'm not necessarily that far from your views. There is, however,
more to an XML database than just storing data - relational databases
DB2 supports cooked and raw file systems - SMS (System Manged Space)
and DMS (Database Managed Space) tablespaces.
The DB2 experience is that DMS tends to outperform SMS but requires
considerable tuning and administrative overhead to see these wins.
--
Pip-pip
Sailesh
http
vantage
provided you care enough to understand what's going on.
Anyway, I merely responded to provide a data point. Will PostgreSQL
users/administrators care for additional knobs or is there a
preference for "keep it simple, stupid" ?
--
Pip-pip
Sailesh
http://www.cs.b
based off of pgsql-7.3.2)
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
upedAgg work
for large numbers of unique values (by using a form of hybrid hashing).
Another thing I toyed with was having an implementation of a
Tid-List-Fetch .. sorting a TID-list from an index and fetching the
records of the relation off the sorted list for better IO
perform
division "Introduction to
>> Database Systems" class ..
>> http://www-inst.eecs.berkeley.edu/~cs186/
Chris> Hi Sailesh,
Chris> You know what would be kind of cool? If you could write a
Chris> "Guide to PostgreSQL to Teach Databases".
27;em buys you buffer cache locality. When there are large
numbers of hits, it also buys you sequential scans where the file
system prefetcher can help. The additional overhead you pay is the
sorting cost.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
tial
Mike> device rather than a random access device."
Mike> Isn't a TID-List-Fetch implementation a crucial first step in the
Mike> right direction?
I believe so .. I think it's a clear win. I believe there are some
concurrency issues although I'm not sure
chime
in.
Marcel Kornacker did implement concurrency for GiST - I confirmed as
much with Joe Hellerstein (his advisor). I know there's a paper he
wrote with C.Mohan on it. I don't know which version his
implementation was for.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sai
at IBM Almaden.
http://www.almaden.ibm.com/software/dm/SMART/leo.shtml
There is a vldb paper that describes it ..
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings
eliminating choices in your access plan.
As people have said there are plenty of papers on this in the
literature. While I am no theorist I can certainly help with reading
the papers .. not every bit of a paper is very useful.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
ourse, from a costing perspective these shouldn't be very different but ...)
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org
Building the Telegraph Dataflow System. SIGMOD Record 30(4): 103-114
(2001)
Apart from our group, the database research group in Wisconsin also
rewrote their Niagara system from Java to C++.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast
ignment. I was
planning to clean it up and submit it properly but I never got the
time)
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 8: explain analyze is your friend
pproach, is not a general solution. How would you handle the "update
current of cursor" scenarios ? In this case, there is some application
logic that determines the precise records that change and how they
change.
Ergo, it is my claim that while logical redo logg
pp has a lot of updates .. it's just that I'm not
convinced that logical logging is a clean solution.
I also don't have a solution for your problem :-)
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
should reduce the number of times ExecEvalVar (and
through it heapgetattr) are called.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match
o the IHT. All this was only
possible because you guys kept around the varnoold and the attnoold !!
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropria
this can be done without threads .. you just need inter-process
communication.
(BTW, there is at least one commercial system that follows exactly
this model).
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
27;m inclined
I think I've said it before, but we actually have such a beast - a
shared memory MemoryContext that uses libmm (which apparently Apache
also uses). It should be fairly easy to patch in if it's worth it for
you folks.
--
Pip-pip
Sailesh
http://www.cs.berkeley.
' listed yet? As I
Marc> recall it, until that gets done, parallelization of anything
Marc> was considered to be a relatively onerous task, no?
You don't really need to parallelize in separate threads .. you can
have more than one process working on one query
ano query processing system"
By Goetz Graefe in SIGMOD 1990.
Link: http://portal.acm.org/citation.cfm?id=98720
The above link also has references to Gamma but I really like the
exposition in the Volcano/Exchange work much better.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~saile
>>>>> "Yann" == Yann Michel <[EMAIL PROTECTED]> writes:
Yann> O.K. I downloaded it :-) We will see if and how I can
Yann> help
FYI .. in case you aren't aware already:
http://portal.acm.org/citation.cfm?id=98720
--
Pip-pip
S
even re-fetching from disk) the same heap page twice for a
given index scan.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 8: explain analyze is your friend
eal sure
that you will never kick out a page from the buffer cache ..
I believe that for large enough tables this can certainly help .. it
sure is something that many other systems have implemented.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broa
t maintaining visibility in
indexes. Index-only plans are thus viable as they require no change in
the physical structure of the index and no overhead on
update/delete/insert ops.
I don't know about Oracle, which I gather is the only commercial
system to have som
approach to defining transaction isolation is
"Generalized isolation level definitions" by Adya, Liskov and O'Neill
that appears in ICDE 2000.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP
where we use a hash table
until we run out of memory at which time we start spilling to disk. In
other words, no longer use SortAgg at all ..
Under what circumstances will a SortAgg consumer more IOs than a
hybrid hash strategy ?
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
-
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match
unnecessarily block
other operations.
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings
Sounds a bit like multi-dimensional clustering ...
http://www.research.ibm.com/mdc/
After the ARC experience though ...
--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
87 matches
Mail list logo