Re: [fpc-pascal] sorting and merging array of records
Am 12.01.2012 03:34, schrieb waldo kitty: On 1/11/2012 20:35, Tomas Hajny wrote: No, Insert doesn't do any copying by default. In your current code, the copying is performed by calling the PTLERec constructor (in "New(PTLERec, Init(...") but if the pointer isn't inserted, it is thrown away currently. thrown away by the default object insert code? why doesn't it properly wipe it out since it is tossing the pointer into the bitbucket? It can not make any assumption about your pointer. In theory your collection could contain Integers that were simply casted to Pointers. In that case a "disposing of the pointer's data" would be fatal! i just gotta figure out the way that the search and such work as in if they are returning pointers to the record or the record data itself... pointers are still very alien to me :? :/ The methods return Pointers which you can cast to PTLERec. This you can then access like a usual object (e.g. dereferencing, etc) Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] sorting and merging array of records
On Thu, January 12, 2012 03:34, waldo kitty wrote:
> On 1/11/2012 20:35, Tomas Hajny wrote:
>> On 11 Jan 12, at 17:46, waldo kitty wrote:
>> .
>> .
>>> 1. right now the compare is working on the catalog number
>>> (TTLERec.catnbr) and
>>> with duplicates:=FALSE there are no duplicates... however, i need to be
>>> able to
>>> choose which record to keep when there is a duplicate catnbr but the
>>> epoch
>>> (TTLERec.epoch) is different... right now it is throwing out all but
>>> the
>>> first... how can i tell it how to decide which one to throw away? i saw
>>> earlier
>>> that one of the parent objects has neat functions like AtPut which
>>> would easily
>>> allow me to overwrite an existing record that's too old... i just don't
>>> know if
>>> i can use that in the middle of an insert or a search or just exactly
>>> where i
>>> would even put code to do this...
>>
>> The best solution is probably overriding the Insert method to fit
>> your slightly modified logic. Copy the default implementation of
>> TSortedCollection.Insert as found in objects.pp in FPC source tree
>> and modify it according your needs (if Search returns true, perform
>> the additional check for the epoch and depending on the result either
>> use AtInsert (equally to the default implementation) or Dispose the
>> previously stored record at position I ("I" returned by Search, "At
>> (I)" used to access the previously stored record) and then use AtPut.
>
> i've been mulling this over for a few hours and i'm wondering if
> overriding the
> insert method or simply doing the lofic in my importation routine would be
> best... right now my code blindly inserts the record but i could do a
> search for
> a matching "key" (catnbr by keyof ??) and then if i find a match, look at
> the
> epoch of the two records, one in vars and the other in the collection...
> if the
> one in the vars is newer, then do an AtPut to overwrite the one currently
> stored... otherwise, just go on to read the next file record into the vars
> and
> check it from there...
Yes, you can certainly do that. The Insert method does both the Search
call and checking the Duplicates field already. If you do it on your side
(rather than overriding the Insert method), the same calls will be
performed twice in some cases (in particular if the inserting is really
necessary). However, both approaches are certainly possible.
> without looking at the "code to copy" if i want to override the insert
> method,
> it almost seems that there's a bug if it just throws away the record we're
> trying to insert... it would seem that if the code locates a "duplicate"
> record,
> it would properly dispose of unwanted data... unless i perform the
No. There are two tasks. One is dynamic allocation of the object, the
other is insertion. Although you perform both tasks on one line in your
program, these are two distinct tasks. The insert code cannot know whether
you still may need the (previously allocated) object or not in the very
general case. The insert code doesn't throw anything away - you do it by
not storing the result of the "New (PTLERec, Init (..." to some variable
and only send it as a parameter to a method which may or may not store it
somewhere else.
> previously
> mentioned logic checks manually to catch this, i don't see this
> happening... the
> number of records left in memory at the end of the program's execution is
> the
> same number as those not added because they are "dupes"...
There are certainly different approaches possible. Yes, you could e.g.
have a static object (allocated on stack or as global data), populate it
with the values read from the file first, search the collection using this
static object and only allocate a new one dynamically if you need to
insert it.
>>> 2. something else i'm running into is with duplicates:=FALSE, there's a
>>> whole
>>> bucket load of records that are not disposed of when i wipe out the
>>> collection... heaptrc hollers right nasty to me about'em on exit... i
>>> can only
>>> assume that these are the duplicates but i don't understand why they
>>> are still
>>> hanging around if insert or add threw them away already :/
>>
>> If you already had them in the collection, they're not added again.
>> You only dispose records added to the collection at the end, but
>> these are lost this way. You can also sort this out in the overridden
>> Insert method (if you don't want to use the newly created record,
>> dispose it).
>
> as above, i don't know that they are /in/ the collection... i'm
> (currently)
> simply calling the insert method and leaving the work up to it... if it
> should
> be handling this gracefully, it isn't... at least not in a way that
> heaptrc likes ;)
It behaves as specified; the responsibility for disposing the duplicates
is on your side (since you allocate them in your code also).
>>> [TRIM]
> data^.catnbr := Copy(data^.satdata[1],3,5);
> data^.epoch := Real_Value(data^.satdata[1],19,14);
> inc(sat_cn
Re: [fpc-pascal] sorting and merging array of records
On 1/12/2012 07:20, Tomas Hajny wrote: On Thu, January 12, 2012 03:34, waldo kitty wrote: [TRIM] without looking at the "code to copy" if i want to override the insert method, it almost seems that there's a bug if it just throws away the record we're trying to insert... it would seem that if the code locates a "duplicate" record, it would properly dispose of unwanted data... unless i perform the No. There are two tasks. One is dynamic allocation of the object, the other is insertion. Although you perform both tasks on one line in your program, these are two distinct tasks. The insert code cannot know whether you still may need the (previously allocated) object or not in the very general case. The insert code doesn't throw anything away - you do it by not storing the result of the "New (PTLERec, Init (..." to some variable and only send it as a parameter to a method which may or may not store it somewhere else. AHH! i think i understand this now... you are saying that instead of doing this... aTLEColl^.insert(New(PTLERec, Init(MySatName, MySatData1, MySatData2, MyCatNbr, MyEpoch))); i should do it in two steps like this... aTLERec := New(PTLERec, Init(MySatName, MySatData1, MySatData2, MyCatNbr, MyEpoch)); aTLEColl^.insert(aTLERec); but the the problem still comes of how do i know if the record was inserted into the list or not? insert doesn't seem to return a true or false on success... i guess this is where it is required(?) to override the insert method? does everyone have to override the insert method if they are in duplicates := FALSE mode? how do they handle the throwing away of the unneeded data?? [TRIM] 2. something else i'm running into is with duplicates:=FALSE, there's a whole bucket load of records that are not disposed of when i wipe out the collection... heaptrc hollers right nasty to me about'em on exit... i can only assume that these are the duplicates but i don't understand why they are still hanging around if insert or add threw them away already :/ If you already had them in the collection, they're not added again. You only dispose records added to the collection at the end, but these are lost this way. You can also sort this out in the overridden Insert method (if you don't want to use the newly created record, dispose it). i'm looking at this override of the insert method... as above, i don't know that they are /in/ the collection... i'm (currently) simply calling the insert method and leaving the work up to it... if it should be handling this gracefully, it isn't... at least not in a way that heaptrc likes ;) It behaves as specified; the responsibility for disposing the duplicates is on your side (since you allocate them in your code also). i think i see now... [TRIM] i'll probably have broken my code by the time you read this... but i'll very likely be attempting to implement the logic in my Input_Satellite_List routine ;) OB-) Possible, but likely resulting in some useless overhead (computing performance-wise) if you still intend to use the method Insert in that case. yeah, that went over like a lead balloon... i'm digging into the insert method override, instead... just gotta figure out how to access the epoch in the current record and the one in the passed record for the decision making comparison... ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] sorting and merging array of records
On 1/12/2012 11:23, waldo kitty wrote:
On 1/12/2012 07:20, Tomas Hajny wrote:
On Thu, January 12, 2012 03:34, waldo kitty wrote:
[TRIM]
i'll probably have broken my code by the time you read this... but i'll
very likely be attempting to implement the logic in my Input_Satellite_List
routine ;) OB-)
Possible, but likely resulting in some useless overhead (computing
performance-wise) if you still intend to use the method Insert in that
case.
yeah, that went over like a lead balloon... i'm digging into the insert method
override, instead... just gotta figure out how to access the epoch in the
current record and the one in the passed record for the decision making
comparison...
yeah, i know... bad form to reply to my own message but...
I GOT IT!! WHOOHOO! finally...
procedure TTLEColl.Insert (Item: Pointer);
var
I: Sw_Integer;
old: Pointer;
begin
If NOT Search(KeyOf(Item), I) OR Duplicates Then { Item valid }
AtInsert(I, Item){ Insert the item }
else // otherwise
// compare the epochs & replace existing record if < item^.epoch
if PTLERec(At(i))^.epoch < PTLERec(Item)^.epoch then
begin
old := At(i); // save existing pointer first!
AtPut(i, Item);// now put in the new record
PTLERec(old)^.Done;// time to remove the old data
dispose(old); // and finally dispose it
end
else
begin// we're tossing this one out
PTLERec(Item)^.Done; // so remove the data first
dispose(Item); // and then dispose the item
end;
end;
now the list is sorted, newer entries replace older ones and there's no more
whining from heaptrc about left over fluff on the heap... yeah!
and i said it before but dang this thing's fast... compared to the old tool that
i've been using for eons... i mean look at it like this... we're comparing a
turtle with a lightening bolt... somehow "fast" just doesn't seem to be strong
enough to describe the difference in speed :P
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] sorting and merging array of records
On Thu, January 12, 2012 17:23, waldo kitty wrote:
> On 1/12/2012 07:20, Tomas Hajny wrote:
>> On Thu, January 12, 2012 03:34, waldo kitty wrote:
> [TRIM]
>>> without looking at the "code to copy" if i want to override the insert
>>> method, it almost seems that there's a bug if it just throws away the
>>> record we're trying to insert... it would seem that if the code locates
>>> a "duplicate" record, it would properly dispose of unwanted data...
>>> unless i perform the
>>
>> No. There are two tasks. One is dynamic allocation of the object, the
>> other is insertion. Although you perform both tasks on one line in your
>> program, these are two distinct tasks. The insert code cannot know
>> whether
>> you still may need the (previously allocated) object or not in the very
>> general case. The insert code doesn't throw anything away - you do it by
>> not storing the result of the "New (PTLERec, Init (..." to some variable
>> and only send it as a parameter to a method which may or may not store
>> it
>> somewhere else.
>
> AHH! i think i understand this now... you are saying that instead of doing
> this...
>
> aTLEColl^.insert(New(PTLERec, Init(MySatName, MySatData1, MySatData2,
> MyCatNbr,
> MyEpoch)));
>
> i should do it in two steps like this...
>
> aTLERec := New(PTLERec, Init(MySatName, MySatData1, MySatData2, MyCatNbr,
> MyEpoch));
> aTLEColl^.insert(aTLERec);
>
> but the the problem still comes of how do i know if the record was
> inserted into
> the list or not? insert doesn't seem to return a true or false on
> success... i
> guess this is where it is required(?) to override the insert method? does
> everyone have to override the insert method if they are in duplicates :=
> FALSE
> mode? how do they handle the throwing away of the unneeded data??
There are multiple options. You can search before (as suggested by you)
and only call Insert if not finding the result, you can override the
Insert call as suggested by me, you can add another insertion method with
a different signature (e.g. a function instead of a procedure providing
information whether the record was inserted or not as its result), you
could also search for the record after calling Insert and compare the
pointer returned by At for the found index returned by Search to your
original pointer (i.e. checking that both pointers refer to the same
address, not just that they contain the same data), you could possibly
also ignore the lost memory altogether in certain scenarios (assuming that
the memory is deallocated anyway when the program is finished with its
work, the amount of data is limited to sensible values compared to overall
amount of memory in your machine and the program does just some one-time
processing and thus does not live for very long), etc.
.
.
> yeah, that went over like a lead balloon... i'm digging into the insert
> method
> override, instead... just gotta figure out how to access the epoch in the
> current record and the one in the passed record for the decision making
> comparison...
PROCEDURE TTLECollection.Insert (Item: Pointer);
VAR I: Sw_Integer;
BEGIN
If NOT Search(KeyOf(Item), I) OR Duplicates Then { Item valid }
AtInsert(I, Item){ Insert the item }
else
if PTLERec (At (I))^.Epoch > PTLERec (Item)^.Epoch then
begin
FreeItem (At (I));
AtPut (I, Item);
end
else
FreeItem (Item); (* or "Dispose (PTLERec (Item), Done)" instead *)
END;
Tomas
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] sorting and merging array of records
On 1/12/2012 12:42, Tomas Hajny wrote:
On Thu, January 12, 2012 17:23, waldo kitty wrote:
[TRIM]
but the the problem still comes of how do i know if the record was
inserted into the list or not? insert doesn't seem to return a true
or false on success... i guess this is where it is required(?) to
override the insert method? does everyone have to override the
insert method if they are in duplicates := FALSE mode? how do they
handle the throwing away of the unneeded data??
There are multiple options. You can search before (as suggested by you)
and only call Insert if not finding the result, you can override the
Insert call as suggested by me,
this is what i've done and it works as per my previous message in this thread...
however, i have found another problem... the program mostly works.. i mean that
if a dupe catnbr with an epoch >= to the existing record is found it is thrown
out but when another dupe catnbr is found it, too, is also inserted... this
seems to only be happening with entries that have more than one duplicate... it
doesn't make sense... i'm suspecting the compare routine to be the problem...
epochs are read from the file and converted to a double by this original routine
from TSKelso's SGP4-26A source code archive...
Function Real_Value(buffer : string; start,length : integer) : double;
Takes the segment of {buffer} beginning at {start} and having length
{length} and converts it to a double precision real number.
Function Real_Value(buffer : string; start,length : integer) : double;
var
MyResult : integer;
answer : double;
begin
buffer := Copy(buffer,start,length);
Convert_Blanks(buffer);
if buffer = '' then
buffer := '0';
Val(buffer,answer,MyResult);
if MyResult = 0 then
Real_Value := answer
else
Real_Value := 0.0;
end; {Function Real_Value}
but yet i find this in my debug output...
03597 5.04184917406000E+003 . ** inserting record
[...]
03597 1.20109577665600E+004 . ** inserting record
here's my compare and insert routines... i'm suspecting my use of CompareValue
is in error :/
Function CompareReal(num1,num2 : double) : integer;
begin
CompareReal := CompareValue(num1, num2);
end;
procedure TTLEColl.Insert (Item: Pointer);
var
I: Sw_Integer;
old: Pointer;
foo: integer;
begin
If NOT Search(KeyOf(Item), I) OR Duplicates Then { Item valid }
begin
{$IFDEF DEBUG}
write(PTLERec(Item)^.catnbr^);
write(' ');
write(PTLERec(Item)^.epoch);
write(' ');
write('.');
writeln(' ** inserting record');
{$ENDIF}
AtInsert(I, Item); { Insert the item }
end
else // otherwise
begin
{$IFDEF DEBUG}
write(PTLERec(At(i))^.catnbr^);
write(' ');
write(PTLERec(At(i))^.epoch);
write(' ');
write(PTLERec(Item)^.epoch);
{$ENDIF}
// compare the epochs
foo := CompareReal(PTLERec(At(i))^.epoch,PTLERec(Item)^.epoch);
case foo of
-1 : begin // replace existing record
{$IFDEF DEBUG}
writeln(' ** replacing record');
{$ENDIF}
old := At(i); // save existing pointer first!
AtPut(i, Item); // now put in the new record
PTLERec(old)^.Done; // time to remove the old data
dispose(old); // and finally dispose it
end;
0 : begin // we're tossing this one out
{$IFDEF DEBUG}
writeln(' ** epochs are same');
{$ENDIF}
PTLERec(Item)^.Done; // so remove the data first
dispose(Item);// and then dispose the item
end;
1 : begin // we're tossing this one out
{$IFDEF DEBUG}
writeln(' ** existing is newer');
{$ENDIF}
PTLERec(Item)^.Done; // so remove the data first
dispose(Item);// and then dispose the item
end;
end; {case}
end;
end;
[TRIM]
.
.
yeah, that went over like a lead balloon... i'm digging into the insert
method
override, instead... just gotta figure out how to access the epoch in the
current record and the one in the passed record for the decision making
comparison...
PROCEDURE TTLECollection.Insert (Item: Pointer);
VAR I: Sw_Integer;
BEGIN
If NOT Search(KeyOf(Item), I) OR Duplicates Then { Item valid }
AtInsert(I, Item){ Insert the item }
else
if PTLERec (At (I))^.Epoch> PTLERec (Item)^.Epoch then
begin
FreeItem (At (I));
AtPut (I, Item);
end
else
FreeItem (I
Re: [fpc-pascal] sorting and merging array of records
On 1/12/2012 14:11, waldo kitty wrote: but yet i find this in my debug output... 03597 5.04184917406000E+003 . ** inserting record [...] 03597 1.20109577665600E+004 . ** inserting record i was looking at the wrong compare... i had thought i'd be "smart" and use strcomp but it has bit me arse! :lol: function TTLEColl.Compare(key1, key2: pointer): sw_integer; begin if PString(key1)^ = PString(key2)^ then compare := 0 else if PString(key1)^ < PString(key2)^ then compare := -1 else compare := 1; // compare := strcomp(PChar(key1),PChar(key2)); end; i wonder why the difference?? reverting to the original code as above (note the commented out strcomp) and it is working... now, after scrolling thru a side by side diff) there are no dupes at all, YY! ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
[fpc-pascal] Installing on mac
Hi, I have done a sudo make clean all. The compiler and units are built correctly and installed in /usr/local/lib/fpc/2.7.1. The tools seems to be placed under /usr/local/bin. However ./fpc from command-line gives me the old 2.4.4, therefore I changed my ./profile file. So now I have 2.7.1. However Lazarus still compiles with 2.4.4, even if I compile it from commandline. This has to mean that I shouldn't have modified the .profile, but create a symlink??? Any help is appreciated. Regards, Darius___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Installing on mac
On 12 Jan 2012, at 21:40, Darius Blaszyk wrote: > I have done a sudo make clean all. The compiler and units are built correctly > and installed in /usr/local/lib/fpc/2.7.1. The tools seems to be placed under > /usr/local/bin. > > However ./fpc from command-line gives me the old 2.4.4, therefore I changed > my ./profile file. So now I have 2.7.1. However Lazarus still compiles with > 2.4.4, even if I compile it from commandline. This has to mean that I > shouldn't have modified the .profile, but create a symlink??? "make install" never changed the default installed version on Unix-like platforms. To change that one, you indeed have to change the ppc386/ppcppc/ppcx64/... symlinks in /usr/local/bin to point to the version that you want to become default. Alternatively, you can change the compiler binary that Lazarus uses in Tools->Options->Environment->Compiler path Jonas___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Installing on mac
On Jan 12, 2012, at 9:48 PM, Jonas Maebe wrote: > > On 12 Jan 2012, at 21:40, Darius Blaszyk wrote: > >> I have done a sudo make clean all. The compiler and units are built >> correctly and installed in /usr/local/lib/fpc/2.7.1. The tools seems to be >> placed under /usr/local/bin. >> >> However ./fpc from command-line gives me the old 2.4.4, therefore I changed >> my ./profile file. So now I have 2.7.1. However Lazarus still compiles with >> 2.4.4, even if I compile it from commandline. This has to mean that I >> shouldn't have modified the .profile, but create a symlink??? > > "make install" never changed the default installed version on Unix-like > platforms. To change that one, you indeed have to change the > ppc386/ppcppc/ppcx64/... symlinks in /usr/local/bin to point to the version > that you want to become default. Thanks Jonas, I missed the ppc symlink there for some reason. Now that it's updated everything works fine. Regards, Darius ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] sorting and merging array of records
On 12 Jan 12, at 14:11, waldo kitty wrote: . . > that's pretty close to what i did (see above) :lol: i might be able to > tighten > mine up a bit instead of doing the .done and then a dispose... or does that > matter? I believe that there isn't a difference from a technical point of view in this case (except for the fact that decoupling the destructor from the deallocation may be a bit dangerous if you forget about it later a try to do something with the object in between). Tomas ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] sorting and merging array of records
On 1/12/2012 17:49, Tomas Hajny wrote: On 12 Jan 12, at 14:11, waldo kitty wrote: . . that's pretty close to what i did (see above) :lol: i might be able to tighten mine up a bit instead of doing the .done and then a dispose... or does that matter? I believe that there isn't a difference from a technical point of view in this case (except for the fact that decoupling the destructor from the deallocation may be a bit dangerous if you forget about it later a try to do something with the object in between). i'm not quite sure i understand... i kinda konw what you are speaking of, though, because the HINTs and such (i don't know which since i just enabled them) tell me something about a plain "pointer being disposed makes no sense" or some such... when i put the dispose lines back, these reports were emitted by the compiler but heaptrc didn't bitch any more... removing the dispose lines to make the compiler made heaptrc whine and cry... also, i've a problem with os2260full that i'll be changing the subject of this thread for or starting another thread... yes, i bit the bullet and wiped my old os2 2.4.0(?? or 2.4.2???) install and none of my previous projects compile... they complain about "Error: Can't call the assembler, error -1 switching to external assembling"... :/ i've manually removed all the .o and .ppu files that i may have inadvertently transferred when i scp'd everything over to my os2 box but no joy :/ ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] sorting and merging array of records
On 1/12/2012 17:49, Tomas Hajny wrote: On 12 Jan 12, at 14:11, waldo kitty wrote: . . that's pretty close to what i did (see above) :lol: i might be able to tighten mine up a bit instead of doing the .done and then a dispose... or does that matter? I believe that there isn't a difference from a technical point of view in this case (except for the fact that decoupling the destructor from the deallocation i decoupled the destructor from the deallocation? hummm... no clue... ya got a cluex4 to smack me with, please? :( may be a bit dangerous if you forget about it later a try to do something with the object in between). in this case, that won't happen ;) ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
[fpc-pascal] MSEide+MSEgui 2.8 for FPC 2.6.0
Hi, MSEide+MSEgui 2.8 for FPC 2.6.0 has been released: http://sourceforge.net/projects/mseide-msegui/ Martin ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
