[BUGS] Bug #694: can't store {digits} using JDBC
Dan Brown ([EMAIL PROTECTED]) reports a bug with a severity of 2
The lower the number the more severe it is.
Short Description
can't store {digits} using JDBC
Long Description
We are using the JDBC driver, and are losing partial data. Whenever we try to store
{digits} (note: exactly as shown with the curly brackets) we are finding that postgres
is storing it as " igits ". We can insert it via psql correctly, but not via the JDBC
driver. Also note, if we store something like {vdigits}, we have no problem.
Also, it works fine in Oracle 8.
The version of postgreSQL we are using is:
PostgreSQL 7.2.1 on i686-pc-cygwin, compiled by GCC 2.95.3-5
( we are also seeing the same problem in Linux ).
Sample Code
No file was uploaded with this report
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
Re: [BUGS] Patch for memory leaks in index scan
With no solution on the horizon, and the author saying it fixes some of his trigger queries, I am inclined to apply this. I don't see any downside except for some extra pfrees if we ever fix this. --- Dmitry Tkach wrote: > Tom Lane wrote: > > >Dmitry Tkach <[EMAIL PROTECTED]> writes: > > > >>*** nodeIndexscan.c.origFri Apr 19 10:29:57 2002 > >>--- nodeIndexscan.c Fri Apr 19 10:30:00 2002 > >>*** > >>*** 505,510 > >>--- 505,514 > >> */ > >> ExecClearTuple(scanstate->cstate.cs_ResultTupleSlot); > >> ExecClearTuple(scanstate->css_ScanTupleSlot); > >>+ pfree(scanstate); > >>+ pfree(indexstate->iss_RelationDescs); > >>+ pfree(indexstate->iss_ScanDescs); > >>+ pfree(indexstate); > >> } > >> > > > >I do not believe this patch will fix anything. > > > Well... It does fix the query I mentioned in my first message (and a few > others I was doing when I ran into this problem) > > > > >In general, the EndNode routines do not bother with releasing memory, > > > Well... Not quite true. For example, ExecEndIndexScan () does release > lots of stuff, that was allocated in ExecInitIndexScan (), it just > forgets about these four things... > > > > >because it's the end of the query and we're about to drop or reset > >the per-query context anyway. > > > ... if the query manages to complete without running out of memory like > in my case :-) > > > If the above pfrees are actually needed > >then there are a heck of a lot of other places that need explicit > >pfrees. > > > Perhaps... I haven't run into any other places just yet... > > >And that is not a path to a solution, because there will > >*always* be places that forgot a pfree. > > > Sure :-) > You can say this about any bug pretty much :-) There will *always* be > bugs... That doesn't mean that the ones, that are found should not be > fixed though > > > What's needed is a structural > >solution. > > > I understand what you are saying... I was looking around for one for a > while, but it seems, but there doesn't seem > to be anything 'more structural' for this particular case, as far as I > can see - per query context does get released properly in the end, it's > just that the query requires too much temporary memory to complete, so > the end is actually never reached... After all, I repeat, > ExecEndIndexScan () DOES free up lots of memory, so I don't see how > adding these four things that got forgotten would hurt. > > > > >I think your real complaint is that SQL functions leak memory. They > >should be creating a sub-context for their queries that can be freed > >when the function finishes. > > > > I guess, you are right here - I tried using a subquery instead of a > function, and that is not leaking, because it does > ExecRescan () for each outer tuple, instead of reinitializing the > executor completely as it is the case with sql func . > > fmgr_sql () DOES use it's own context, but it looks like it doesn't want > to reset it every time, because of caching... > > Perhaps, it could just execute every command in a function as a SubPlan, > instead of reinitializing every time, but that wouldn't be a simple 4 > line fix, that certainly doesn't break anything that was working before :-) > > I was thinking in terms of a quick PATCH, that can fix a particular > problem, and would be safe to be put in. > > It does not prevent any "structural solution" from being implemented if > somebody comes up with one in the future, and it would STILL be useful > even when then solution is implemented, because it makes memory usage > more conservative, thus reducing the load on system resources... > > Frankly, I don't see what is your problem with it at all :-) > > Dima > > > > > > ---(end of broadcast)--- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to [EMAIL PROTECTED] so that your > message can get through to the mailing list cleanly > -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [BUGS] Patch for memory leaks in index scan
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > With no solution on the horizon, and the author saying it fixes some of > > his trigger queries, I am inclined to apply this. I don't see any > > downside except for some extra pfrees if we ever fix this. > > Sure, apply away. I was mainly expressing my angst over the lack of > a system-wide solution. Sure, many of the patches I do exist for only a few releases, but they meet users needs until a better solution comes along. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [BUGS] Patch for memory leaks in index scan
Bruce Momjian <[EMAIL PROTECTED]> writes: > With no solution on the horizon, and the author saying it fixes some of > his trigger queries, I am inclined to apply this. I don't see any > downside except for some extra pfrees if we ever fix this. Sure, apply away. I was mainly expressing my angst over the lack of a system-wide solution. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [BUGS] pg_dumpall should permit quiet operation
Without showing the database being dumped, the entire output of
pg_dumpall seems pretty useless so you may as well pipe the whole output
to /dev/null. I don't think a quiet feature for pg_dumpall has enough
use for ordinary users. Sorry.
---
Neil T. Spring wrote:
> pg_dumpall now prints status messages; I find them to be a
> nuisance, and the patch below implements a --quiet option,
> though not necessarily well.
>
> thanks,
> -neil
>
>
> POSTGRESQL BUG REPORT TEMPLATE
>
>
>
> Your name :Neil Spring
> Your email address: [EMAIL PROTECTED]
>
>
> System Configuration
> -
> Architecture (example: Intel Pentium) : PIII dual 1GHz
>
> Operating System (example: Linux 2.0.26 ELF): Debian Woody / 2.4.18
>
> PostgreSQL version (example: PostgreSQL-7.2.1): PostgreSQL-7.2.1
>
> Compiler used (example: gcc 2.95.2): 2.95.4
>
>
> Please enter a FULL description of your problem:
>
>
> pg_dumpall now prints status messages to stderr, which makes
> it less suitable for a nightly cron job to backup the
> database. I either get nightly useless mail, or redirect
> stderr so that I don't know about errors. pg_dumpall
> should not print those messages, or at least support
> a --quiet option.
>
>
> Please describe a way to repeat the problem. Please try to provide a
> concise reproducible example, if at all possible:
> --
>
> type "pg_dumpall > x". watch informational messages appear.
>
>
>
> If you know how this problem might be fixed, list the solution below:
> -
>
> patch to pg_dumpall.sh follows. A patch to pg_dumpall.1 is
> straightforward, assuming this is the way to deal with the problem.
>
> diff -cr postgresql-7.2.1/src/bin/pg_dump/pg_dumpall.sh
>postgresql/src/bin/pg_dump/pg_dumpall.sh
> *** postgresql-7.2.1/src/bin/pg_dump/pg_dumpall.shSun Feb 10 16:18:20 2002
> --- postgresql/src/bin/pg_dump/pg_dumpall.sh Tue Apr 23 14:22:56 2002
> ***
> *** 87,92
> --- 87,96
> echo "pg_dumpall (PostgreSQL) $VERSION"
> exit 0
> ;;
> + --quiet|-q)
> + quiet=t
> + break
> + ;;
> --host|-h)
> connectopts="$connectopts -h $2"
> shift;;
> ***
> *** 144,149
> --- 148,154
> echo " -g, --globals-only Only dump global objects, no databases"
> echo " -h, --host=HOSTNAMEServer host name"
> echo " -p, --port=PORTServer port number"
> + echo " -q, --quietDon't print status messages to stderr"
> echo " -U, --username=NAMEConnect as specified database user"
> echo " -W, --password Force password prompts (should happen
>automatically)"
> echo "Any extra options will be passed to pg_dump. The dump will be written"
> ***
> *** 169,175
> echo "DELETE FROM pg_shadow WHERE usesysid <> (SELECT datdba FROM pg_database
>WHERE datname = 'template0');"
> echo
>
> ! echo "connected to template1..." 1>&2
> $PSQL -d template1 -At -c "\
> SELECT
> 'CREATE USER \"' || usename || '\" WITH SYSID ' || usesysid
> --- 174,180
> echo "DELETE FROM pg_shadow WHERE usesysid <> (SELECT datdba FROM pg_database
>WHERE datname = 'template0');"
> echo
>
> ! [ "$quiet" ] || echo "connected to template1..." 1>&2
> $PSQL -d template1 -At -c "\
> SELECT
> 'CREATE USER \"' || usename || '\" WITH SYSID ' || usesysid
> ***
> *** 235,241
> fi
>
> echo "${BS}connect \"$DATABASE\" \"$DBOWNER\""
> ! echo "dumping database \"$DATABASE\"..." 1>&2
> $PGDUMP "$DATABASE" <&4
> if [ "$?" -ne 0 ] ; then
> echo "pg_dump failed on $DATABASE, exiting" 1>&2
> --- 240,246
> fi
>
> echo "${BS}connect \"$DATABASE\" \"$DBOWNER\""
> ! [ "$quiet" ] || echo "dumping database \"$DATABASE\"..." 1>&2
> $PGDUMP "$DATABASE" <&4
> if [ "$?" -ne 0 ] ; then
> echo "pg_dump failed on $DATABASE, exiting" 1>&2
>
> ---(end of broadcast)---
> TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
>
--
Bruce Momjian| http://candle.pha.pa.us
[EMAIL PROTECTED] | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup.| Drexel Hill, Pennsylvania 19026
---(end of broadca
Re: [BUGS] Patch for memory leaks in index scan
Your patch has been added to the PostgreSQL unapplied patches list at: http://candle.pha.pa.us/cgi-bin/pgpatches I will try to apply it within the next 48 hours. --- Dmitry Tkach wrote: > Ok, this sit around for a while, and, because there was no responses, I assume, that >nothing jumps out > at you as being terribly with my logic... > Here is the patch (see the original problem description in the bottom)... It seems >to be working (at least that query, > that used to be running out of memory is now able to finish). > > *** nodeIndexscan.c.origFri Apr 19 10:29:57 2002 > --- nodeIndexscan.c Fri Apr 19 10:30:00 2002 > *** > *** 505,510 > --- 505,514 > */ > ExecClearTuple(scanstate->cstate.cs_ResultTupleSlot); > ExecClearTuple(scanstate->css_ScanTupleSlot); > + pfree(scanstate); > + pfree(indexstate->iss_RelationDescs); > + pfree(indexstate->iss_ScanDescs); > + pfree(indexstate); >} > >/* > > I hope, it helps > > Dima > > > Original Message > > It seems that ExecInit/EndIndexScan is leaking some memory... > > For example, if I run a query, that uses an index scan, and call MemoryContextStats >(CurrentMemoryContext) before > ExecutorStart() and after ExecutorEnd() in ProcessQuery(), I am consistently seeing >that the 'after' call shows > 256 bytes more used, then 'before'... > > In many common cases, this is not a problem, because pg_exec_query_string () will >call > MemoryContextResetAndDeleteChildren(), that will clean everything up eventually... > > But it still seems to cause problems, when the index scan is not directly invoked >from pg_exec_query_string (). > For example: > > create table a > ( > id int primary key, > data char(100) > ); > > create table b > ( > id int references a, > more_data char(100) > ); > > create function merge_data (int,data) returns char(200) as 'select data || $2 from a >where id = $1;' language 'sql'; > > Now, if b is large enough, then something like: > > select merge_data (id,more_data) from b; > > Will eventually run out of memory - it will loose 256 bytes after each row, or about >a GIG after 4 million rows... > > The problem seems to be in ExecEndIndexScan - it does not release scanstate, >indexstate, indexstate->iss_RelationDescs > and indexstate -> iss_ScanDescs... > > I am not familiar enough with the core code, to just jump in and start fixing it, >but I can make the patch, > test it and send it to you, if you guys let me know if what I am saying makes sense >to you... > Or am I missing something here? > > Thanks! > > Dima > > > > ---(end of broadcast)--- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to [EMAIL PROTECTED] so that your > message can get through to the mailing list cleanly > -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
