[GENERAL] Returning a char from a C-language function
I have a small problem when trying to make a C-language function to return one character. The code is very simple and that's what's making it so strange. Datum val_ibool(PG_FUNCTION_ARGS) { iBool *input = (iBool *) PG_GETARG_POINTER(0); PG_RETURN_CHAR(input->value); } input is a structure containing a character as value. I also tried this, but it didn't work: Datum val_ibool(PG_FUNCTION_ARGS) { char test = 'a'; PG_RETURN_CHAR(test); } On the other side I have this: CREATE FUNCTION val(ibool) RETURNS char AS '/home/ivan/Desktop/diplomski/trunk/C/modbms', 'val_ibool' LANGUAGE C STRICT; Any advice? _ Hotmail: Trusted email with powerful SPAM protection. https://signup.live.com/signup.aspx?id=60969
[GENERAL] Persistence problem
Hello. I have a problem. I've created several types and functions in C language and implemented them successfully. Or at least I thought so... When I insert, select, update data, everything works fine, but in that session only. As soon as I close psql and start it agan, the data is corrupted. Is there some common mistake I might have made? Code is pretty big, so I don't think it would do any good if I put it here... Thanks. Ivan _ Hotmail: Powerful Free email with security by Microsoft. https://signup.live.com/signup.aspx?id=60969
Re: [GENERAL] Persistence problem
That was my first guess. I used palloc everywhere.. But to be sure, after I made the type, I tried to do the something like: mytype * result; mytype * realResult; result = createType(...); realResult = (mytype *)palloc(mytype->length); mempcy (realResult, result, result->length); It didn't help. Is that enough? > Subject: Re: [GENERAL] Persistence problem > From: dal...@solfertje.student.utwente.nl > Date: Wed, 12 May 2010 19:05:36 +0200 > CC: pgsql-general@postgresql.org > To: i@live.com > > On 12 May 2010, at 18:08, I. B. wrote: > > > Hello. > > > > I have a problem. I've created several types and functions in C language > > and implemented them successfully. Or at least I thought so... When I > > insert, select, update data, everything works fine, but in that session > > only. As soon as I close psql and start it agan, the data is corrupted. Is > > there some common mistake I might have made? Code is pretty big, so I don't > > think it would do any good if I put it here... > > That sounds like what the documentation warns you about if you modify a > structure directly from postgres without using palloc. > > Alban Hertroys > > -- > If you can't see the forest for the trees, > cut the trees and you'll see there is no forest. > > > !DSPAM:1077,4beadfe710418634015713! > > _ Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. https://signup.live.com/signup.aspx?id=60969
Re: [GENERAL] Persistence problem
Thanks for the reply. However, I don't really understand how it works... Can you help? How should this function look like? Can I still create the type the same way I did for now and add something like this: Datum mpoint_in(PG_FUNCTION_ARGS) { char *str = PG_GETARG_CSTRING(0); mPoint *result; mPoint *finalResult; result = (mPoint *) create_mPoint(str); if (result == NULL) { Log ("mpoint_in: reporting error for invalid input syntax for mPoint"); ereport(ERROR, (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), errmsg("invalid input syntax for mPoint: \"%s\"", str))); } finalResult = (mPoint *)palloc(VARSIZE(result)); memset (finalResult, 0, VARSIZE(result)); SET_VARSIZE(finalResult, VARSIZE(result)); memcpy((void *) VARDATA(finalResult), (void *) VARDATA(result), VARSIZE(result) - VARHDRSZ); PG_RETURN_POINTER(finalResult); } I should still return Datum, right? At least version 1 calling convetion says that. And should I change something on the SQL side? I would appreciate any help. Thanks. Ivan > Date: Thu, 13 May 2010 12:42:09 +0200 > From: klep...@svana.org > To: i@live.com > CC: dal...@solfertje.student.utwente.nl; pgsql-general@postgresql.org > Subject: Re: [GENERAL] Persistence problem > > On Thu, May 13, 2010 at 12:04:56PM +0200, I. B. wrote: > > > > > > I'll try to explain with as less code as possible. > > One of the types I wanted to create is called mpoint. This is a part of > > code: > > > > > typedef struct { > > int4 length; > > int noOfUnits; > > void *units; // this is later casted to uPoint * > > } mapping_t; > > > > This is not the correct way to handle varlena types. You can create the > datum that way, but if PostgreSQL decides to compress it (as it may > when writing to disk) you won't be able to read it back. Notably, the > "length" part of a varlena type is not always 4 bytes. > > Make sure you have fully understood this page: > http://www.postgresql.org/docs/8.4/static/xfunc-c.html > it has a number of examples dealing with variable length types. You > MUST use the VARDATA/VARATT/etc macros to construct and read your data. > > Hope this helps, > -- > Martijn van Oosterhout http://svana.org/kleptog/ > > Patriotism is when love of your own people comes first; nationalism, > > when hate for people other than your own comes first. > > - Charles de Gaulle _ Hotmail: Free, trusted and rich email service. https://signup.live.com/signup.aspx?id=60969
Re: [GENERAL] Persistence problem
UPDATE: When I do this: realResult = (mPoint *)palloc(result->length); memcpy(realResult, result, result->length); I get a right result in the same session, but corrupted in the next one. I've also found one place in create_mPoint where I used realloc. But why would it matter if I copied the memory into a new variable? Ivan From: i@live.com To: klep...@svana.org CC: dal...@solfertje.student.utwente.nl; pgsql-general@postgresql.org Subject: RE: [GENERAL] Persistence problem Date: Thu, 13 May 2010 12:04:56 +0200 I'll try to explain with as less code as possible. One of the types I wanted to create is called mpoint. This is a part of code: Datum mpoint_in(PG_FUNCTION_ARGS) { char *str = PG_GETARG_CSTRING(0); mPoint *result; result = (mPoint *) create_mPoint(str); if (result == NULL) { Log ("mpoint_in: reporting error for invalid input syntax for mPoint"); ereport(ERROR, (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), errmsg("invalid input syntax for mPoint: \"%s\"", str))); } PG_RETURN_POINTER(result); } In SQL the internallength for the type is set to VARIABLE. The type mpoint is complex. It is a structure containing other structures. typedef struct { int4 length; int noOfUnits; void *units; // this is later casted to uPoint * } mapping_t; typedef mapping_tmPoint; typedef struct { timeint_t interval; double x1, x0, y1, y0; } uPoint; typedef upoint_tuPoint; typedef struct { time_T start, end; short int LC, RC; } timeint_t; typedef struct { double time; short int infinity; } time_T; The function create_mPoint (char *str) creates the type. It is pretty complex and I don't think it's smart to post it here. Anyway, it creates dinamycally array of uPoints for units. During that period, I've used palloc. At the end I set the variable length: result->length = sizeof(int4) + sizeof(int) + result->noOfUnits * sizeof(uPoint); My first guess was that I don't have all the data directly next to each other in the memory. Then I've tried to add this in the function mpoint_in, after I get the result from create_mPoint: mPoint *finalResult; finalResult = (mPoint *)palloc(sizeof(mPoint)); finalResult->units = (uPoint *)palloc(result->noOfUnits * sizeof(uPoint)); memcpy(finalResult->units, result->units, result->noOfUnits * sizeof(uPoint)); finalResult->noOfUnits = result->noOfUnits; finalResult->length = result->length; PG_RETURN_POINTER(finalResult); When I call ToString (I've made it myself), I get the same content for both result and finalResult. However, unlike the first case when I return result and get normal data in that session which dissapears in next one, this time I get corrupted data even in the first session. More code? More explanation? Or someone has an idea? Thank you very much. Ivan > Date: Wed, 12 May 2010 19:45:26 +0200 > From: klep...@svana.org > To: i@live.com > CC: dal...@solfertje.student.utwente.nl; pgsql-general@postgresql.org > Subject: Re: [GENERAL] Persistence problem > > On Wed, May 12, 2010 at 07:12:10PM +0200, I. B. wrote: > > > > That was my first guess. I used palloc everywhere.. But to be sure, after I > > made the type, I tried to do the something like: > > > > mytype * result; > > mytype * realResult; > > result = createType(...); > > realResult = (mytype *)palloc(mytype->length); > > mempcy (realResult, result, result->length); > > Did you define the type properly at SQL level? Is it a varlena type or > fixed length? Did you return it properly (as Datum)? > > You're going to need to post more information before we can help you > usefully. > > Have a nice day, > -- > Martijn van Oosterhout http://svana.org/kleptog/ > > Patriotism is when love of your own people comes first; nationalism, > > when hate for people other than your own comes first. > > - Charles de Gaulle Hotmail: Trusted email with powerful SPAM protection. Sign up now. _ Hotmail: Trusted email with powerful SPAM protection. https://signup.live.com/signup.aspx?id=60969
Re: [GENERAL] Persistence problem
I'll try to explain with as less code as possible. One of the types I wanted to create is called mpoint. This is a part of code: Datum mpoint_in(PG_FUNCTION_ARGS) { char *str = PG_GETARG_CSTRING(0); mPoint *result; result = (mPoint *) create_mPoint(str); if (result == NULL) { Log ("mpoint_in: reporting error for invalid input syntax for mPoint"); ereport(ERROR, (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), errmsg("invalid input syntax for mPoint: \"%s\"", str))); } PG_RETURN_POINTER(result); } In SQL the internallength for the type is set to VARIABLE. The type mpoint is complex. It is a structure containing other structures. typedef struct { int4 length; int noOfUnits; void *units; // this is later casted to uPoint * } mapping_t; typedef mapping_tmPoint; typedef struct { timeint_t interval; double x1, x0, y1, y0; } uPoint; typedef upoint_tuPoint; typedef struct { time_T start, end; short int LC, RC; } timeint_t; typedef struct { double time; short int infinity; } time_T; The function create_mPoint (char *str) creates the type. It is pretty complex and I don't think it's smart to post it here. Anyway, it creates dinamycally array of uPoints for units. During that period, I've used palloc. At the end I set the variable length: result->length = sizeof(int4) + sizeof(int) + result->noOfUnits * sizeof(uPoint); My first guess was that I don't have all the data directly next to each other in the memory. Then I've tried to add this in the function mpoint_in, after I get the result from create_mPoint: mPoint *finalResult; finalResult = (mPoint *)palloc(sizeof(mPoint)); finalResult->units = (uPoint *)palloc(result->noOfUnits * sizeof(uPoint)); memcpy(finalResult->units, result->units, result->noOfUnits * sizeof(uPoint)); finalResult->noOfUnits = result->noOfUnits; finalResult->length = result->length; PG_RETURN_POINTER(finalResult); When I call ToString (I've made it myself), I get the same content for both result and finalResult. However, unlike the first case when I return result and get normal data in that session which dissapears in next one, this time I get corrupted data even in the first session. More code? More explanation? Or someone has an idea? Thank you very much. Ivan > Date: Wed, 12 May 2010 19:45:26 +0200 > From: klep...@svana.org > To: i@live.com > CC: dal...@solfertje.student.utwente.nl; pgsql-general@postgresql.org > Subject: Re: [GENERAL] Persistence problem > > On Wed, May 12, 2010 at 07:12:10PM +0200, I. B. wrote: > > > > That was my first guess. I used palloc everywhere.. But to be sure, after I > > made the type, I tried to do the something like: > > > > mytype * result; > > mytype * realResult; > > result = createType(...); > > realResult = (mytype *)palloc(mytype->length); > > mempcy (realResult, result, result->length); > > Did you define the type properly at SQL level? Is it a varlena type or > fixed length? Did you return it properly (as Datum)? > > You're going to need to post more information before we can help you > usefully. > > Have a nice day, > -- > Martijn van Oosterhout http://svana.org/kleptog/ > > Patriotism is when love of your own people comes first; nationalism, > > when hate for people other than your own comes first. > > - Charles de Gaulle _ Hotmail: Trusted email with powerful SPAM protection. https://signup.live.com/signup.aspx?id=60969
Re: [GENERAL] Persistence problem
Thanks for the reply. Why is that somewhere else in the memory if I reserve enough memory with palloc and copy the complete memory from the previously created type into that new object? realResult = (mPoint *)palloc(result->length); memcpy(realResult, result, result->length); OK, I suppose I should use the VARDATA and VARSIZE. But I really can't make it to work. I don't think I understood well enough how it works. This is what I'd like to do. I would like to create the type as I already did, and when it's created I would like to copy its memory into a new object. So, if I have: Datum mpoint_in(PG_FUNCTION_ARGS) { char *str = PG_GETARG_CSTRING(0); mPoint *result = (mPoint *) create_mPoint(str); mPoint *final result = ... //copy the memory from result PG_RETURN_POINTER(result); } Datum mpoint_out(PG_FUNCTION_ARGS) { mPoint * input = (mPoint *) PG_GETARG_POINTER(0); char * result = ToString (input, MPOINT); PG_RETURN_CSTRING(result); } Can you please help me to make this? How to copy the memory in a good way? Should I somehow change the type mPoint? Should I also change the SQL side? CREATE TYPE mpoint ( internallength = VARIABLE, input = mpoint_in, output = mpoint_out ); Please help, it would mean a lot. Thanks, Ivan > To: i@live.com > CC: klep...@svana.org; dal...@solfertje.student.utwente.nl; > pgsql-general@postgresql.org > Subject: Re: [GENERAL] Persistence problem > Date: Thu, 13 May 2010 15:08:58 -0400 > From: t...@sss.pgh.pa.us > > "I. B." writes: > > When I do this: > > realResult = (mPoint *)palloc(result->length); > > memcpy(realResult, result, result->length); > > I get a right result in the same session, but corrupted in the next > > one. > > I'm guessing a bit here, but I think what is happening is this: > > > typedef struct { > > int4 length; > > int noOfUnits; > > void *units; // this is later casted to uPoint * > > } mapping_t; > > You're storing the above-named struct on disk, right? And the "units" > pointer is pointing to an array that's somewhere else in memory? As > long as the somewhere-else array survives, it will seem like everything > is okay. But in a new session, that data in memory will certainly not > be there anymore. > > You can't use pointers in data structures that are to be stored on disk. > The array data needs to be "in line" in the data structure, and > accounted for in the length word. > > Martin's advice about using VARSIZE/VARDATA is good too. Depending on > which PG version you're using, you might be able to get along without > that so long as you haven't marked the data type toastable (by using > a non-PLAIN storage option in CREATE TYPE). But unless that array is > always pretty darn small, you're going to want to allow this type to > be toasted. > > regards, tom lane _ Hotmail: Trusted email with Microsoft’s powerful SPAM protection. https://signup.live.com/signup.aspx?id=60969
Re: [GENERAL] Persistence problem
I still have the same problem. Whatever I've tried didn't work out. It seems like VARSIZE is wrong. It's less than it should be. It seems like it's not counting on the size of units array, although it changes depending on the number of units. This is one of the things I've tried: Datum mbool_in(PG_FUNCTION_ARGS) { char *str = PG_GETARG_CSTRING(0); mBool *result; mBool *finalResult; int size; result = (mBool *) create_mConst(str, MBOOL); size = sizeof(int4) + sizeof(int) + result->noOfUnits*sizeof(uBool); finalResult = (mBool *) palloc(size); memcpy(finalResult, result, size); SET_VARSIZE(finalResult, size); PG_RETURN_POINTER(finalResult); } Datum mbool_out(PG_FUNCTION_ARGS) { mBool * input = (mBool *) PG_GETARG_POINTER(0); mBool * safeCopy; char * result; safeCopy = (mBool *) palloc(VARSIZE(input)); SET_VARSIZE(safeCopy, VARSIZE(input)); memcpy((void *) VARDATA(safeCopy), (void *) VARDATA(input), VARSIZE(input)); result = ToString (safeCopy, MBOOL); PG_RETURN_CSTRING(result); } How to fix this? The one who helps can count on the best Croatian dark beer shipped directly to his place. ;) Ivan From: i@live.com To: t...@sss.pgh.pa.us CC: klep...@svana.org; dal...@solfertje.student.utwente.nl; pgsql-general@postgresql.org Subject: RE: [GENERAL] Persistence problem Date: Fri, 14 May 2010 14:35:34 +0200 Thanks for the reply. Why is that somewhere else in the memory if I reserve enough memory with palloc and copy the complete memory from the previously created type into that new object? realResult = (mPoint *)palloc(result->length); memcpy(realResult, result, result->length); OK, I suppose I should use the VARDATA and VARSIZE. But I really can't make it to work. I don't think I understood well enough how it works. This is what I'd like to do. I would like to create the type as I already did, and when it's created I would like to copy its memory into a new object. So, if I have: Datum mpoint_in(PG_FUNCTION_ARGS) { char *str = PG_GETARG_CSTRING(0); mPoint *result = (mPoint *) create_mPoint(str); mPoint *final result = ... //copy the memory from result PG_RETURN_POINTER(result); } Datum mpoint_out(PG_FUNCTION_ARGS) { mPoint * input = (mPoint *) PG_GETARG_POINTER(0); char * result = ToString (input, MPOINT); PG_RETURN_CSTRING(result); } Can you please help me to make this? How to copy the memory in a good way? Should I somehow change the type mPoint? Should I also change the SQL side? CREATE TYPE mpoint ( internallength = VARIABLE, input = mpoint_in, output = mpoint_out ); Please help, it would mean a lot. Thanks, Ivan > To: i@live.com > CC: klep...@svana.org; dal...@solfertje.student.utwente.nl; > pgsql-general@postgresql.org > Subject: Re: [GENERAL] Persistence problem > Date: Thu, 13 May 2010 15:08:58 -0400 > From: t...@sss.pgh.pa.us > > "I. B." writes: > > When I do this: > > realResult = (mPoint *)palloc(result->length); > > memcpy(realResult, result, result->length); > > I get a right result in the same session, but corrupted in the next > > one. > > I'm guessing a bit here, but I think what is happening is this: > > > typedef struct { > > int4 length; > > int noOfUnits; > > void *units; // this is later casted to uPoint * > > } mapping_t; > > You're storing the above-named struct on disk, right? And the "units" > pointer is pointing to an array that's somewhere else in memory? As > long as the somewhere-else array survives, it will seem like everything > is okay. But in a new session, that data in memory will certainly not > be there anymore. > > You can't use pointers in data structures that are to be stored on disk. > The array data needs to be "in line" in the data structure, and > accounted for in the length word. > > Martin's advice about using VARSIZE/VARDATA is good too. Depending on > which PG version you're using, you might be able to get along without > that so long as you haven't marked the data type toastable (by using > a non-PLAIN storage option in CREATE TYPE). But unless that array is > always pretty darn small, you're going to want to allow this type to > be toasted. > > regards, tom lane Hotmail: Trusted email with Microsoft’s powerful SPAM protection. Sign up now. _ Hotmail: Trusted email with powerful SPAM protection. https://signup.live.com/signup.aspx?id=60969
Re: [GENERAL] Persistence problem
OK, here is the part of the code. typedef struct { int4 length; int noOfUnits; void *units; } mapping_t; typedef struct { timeint_t interval; double x1, x0, y1, y0; // fx(t) = x1*t+x0, fy(t) = y1*t+y0 } upoint_t; typedef struct { time_T start, end; short int LC, RC; // left or right closed } timeint_t; typedef struct { double time; short int infinity; } time_T; typedef upoint_tuPoint; typedef mapping_tmPoint; mPoint * create_mPoint(char *str) { /* * * str: ((start end x1 x0 y1 y0),(start end x1 x0 y1 y0),...) * */ char *unit = NULL; mPoint * result = NULL; uPoint * units = NULL; uPoint * singleUnit; int stringLength; char *strTemp; int i; if (str == NULL) { return NULL; } stringLength = strlen(str); if (stringLength < 2 || str[0] != '(' || str[stringLength-1] != ')') { return NULL; } else { str = &str[1]; strTemp = (char *) palloc ((stringLength-1)*sizeof(char)); strncpy(strTemp, str, stringLength-2); strTemp[stringLength-2] = '\0'; str = strTemp; }; // allocating memory result = (mPoint *) palloc(sizeof(mPoint)); result->noOfUnits = 0; unit = strtok(str, ","); while (unit != NULL) { Log ("create_mPoint: str units:"); Log2 ("\tunit:", unit); result->noOfUnits++; singleUnit = create_uPoint(unit); if (singleUnit == NULL) { return NULL; } units = (uPoint *) realloc(units, result->noOfUnits * sizeof(uPoint)); // EXPLAINED AT THE END OF THE POST units[result->noOfUnits - 1] = * singleUnit; unit = strtok(NULL, ","); }; result->units = units; // EXPLAINED AT THE END OF THE POST if (SortUnits(result, MPOINT) == NULL) { return NULL; }; result->length = sizeof(int4) + sizeof(int) + result->noOfUnits * sizeof(uPoint); //pfree(singleUnit); //pfree(strTemp); //pfree(unit); Log ("create_mPoint: moving type created"); return result; }; uPoint * create_uPoint(char *str) { double startTime, endTime; double x0, x1, y0, y1; timeint_t * interval; uPoint *result; if (sscanf(str, " ( %lf %lf %lf %lf %lf %lf )", &startTime, &endTime, &x1, &x0, &y1, &y0) != 6) { return NULL; } // allocate memory result = (uPoint *) palloc(sizeof(uPoint)); result->x0 = x0; result->x1 = x1; result->y0 = y0; result->y1 = y1; // insert interval interval = (timeint_t *) createTimeInterval(startTime, endTime); if (interval == NULL) { return NULL; }; result->interval = *interval; Log ("create_uPoint: uPoint (unit type) created"); return result; }; timeint_t * createTimeInterval(double startTime, double endTime) { timeint_t * interval = (timeint_t *) palloc(sizeof(timeint_t)); Log ("createTimeInterval: entering function"); if (startTime > endTime) { return NULL; }; interval->start.time = startTime; interval->end.time = endTime; if (startTime < 0) { interval->LC = 0; interval->start.infinity = 1; } else { interval->LC = 1; interval->start.infinity = 0; } if (endTime < 0) { interval->RC = 0; interval->end.infinity = 1; } else { interval->RC = 1; interval->end.infinity = 1; } return interval; }; The part where I've used realloc and set result->units=units, I've tried to replace it with palloc in the way to create the array units and then do this: result->units = (uPoint *) palloc(result->noOfUnits * sizeof(uPoint)); memcpy(result->units, units, result->noOfUnits * sizeof(uPoint)); I've also tried to reserve the whole memory with palloc: result = (mPoint *)palloc(sizeof(int4) + sizeof(int) + noOfUnits*sizeof(uPoint)); result->noOfUnits = noOfUnits; result->length = sizeof(int4) + sizeof(int) + noOfUnits*sizeof(uPoint); memcpy(result->units, units, result->noOfUnits * sizeof(uPoint)); I've then tried to palloc both result and result->units... I just don't know how I should do it. :-/ I hope you understand the problem better now. Thanks, Ivan > To: i@live.com > CC: klep...@svana.org; dal...@solfertje.student.utwente.nl; > pgsql-general@postgresql.org > Subject: Re: [GENERAL] Persistence problem > Date: Fri, 14 May 2010 17:32:49 -0400 > From: t...@sss.pgh.pa.us > > "I. B." writes: > > How to fix this? > > As long as you keep on showing us wrap
Re: [GENERAL] Persistence problem
Thanks a lot for the patience and help. It worked. Ivan > To: i@live.com > CC: klep...@svana.org; dal...@solfertje.student.utwente.nl; > pgsql-general@postgresql.org > Subject: Re: [GENERAL] Persistence problem > Date: Fri, 14 May 2010 18:34:14 -0400 > From: t...@sss.pgh.pa.us > > "I. B." writes: > > OK, here is the part of the code. > > Well, as suspected, you're doing this > > > typedef struct { > > void *units; > > } mapping_t; > > and this > > > units = (uPoint *) realloc(units, result->noOfUnits * > > sizeof(uPoint)); // EXPLAINED AT THE END OF THE POST > > which means that the array isn't contiguous with the mPoint struct. > You can certainly do that and then rearrange things to make it so > afterwards, but you're not doing so now. Personally though I'd avoid > having two different representations. You'd be better off with > > typedef struct { > int4 length; > int noOfUnits; > uPoint units[1]; /* actually, a variable length array */ > } mPoint; > > and then allocating or reallocating the result struct with a size > calculation like this: > > offsetof(mPoint, units) + noOfUnits * sizeof(uPoint) > > BTW, realloc (as opposed to repalloc) doesn't seem like a tremendously > good idea here. You are leaking that memory for the life of the session. > > regards, tom lane _ Hotmail: Trusted email with Microsoft’s powerful SPAM protection. https://signup.live.com/signup.aspx?id=60969