Hello David, > I don't allocate the memory up to the size of the "Designated size" but > keep realloc-ing until "remaining-byte" counter reaches zero..
I'm not sure I understeand exacly what you mean. But if you are reallocating memory whole the time, you end up with a non efficient program, if not talking about memory fragmentation. What Arno meant was to receive in a fixed buffer and copy that buffer to the place you want, eg a TFileStream. For the fixed buffer you even not need to allocate memory, just put it on the stack as this: procedure TApp.OnDataAvailable(Sender: TObject; Error: word); var Buf: array[0..MAX_BUF_SIZE - 1] of char; begin // receive here > Which one is more efficient performance-wise? I think you wants to save to a file. The most efficient is to receive in a fixed buffer, then write the buffer to a file with the lower level routines FileWrite(). Open the file just once in begin, and Close it only when all data is received, or session is closed. --- Rgds, Wilfried [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html http://www.mestdagh.biz Sunday, October 30, 2005, 15:42, Kei wrote: > I don't allocate the memory up to the size of the "Designated size" but > keep realloc-ing until "remaining-byte" counter reaches zero.. > Do you think I should use TStream (Stream.readbuffer, readbuffer, ...) > or just a pointer of buffer (malloc, realloc, and basic pointer > operations) ? Which one is more efficient performance-wise? > David > Arno Garrels wrote: >>Do you allocate memory up to the size of data the client intends to send? >>That would be a huge waste of memory and won't work upon large data. >>Instead receive into a buffer of let's say 8 kb and write it to your >>stream at once, receive next chunk and so on. You don't have to take care >>of packet size, it's something on a lower layer encapsulated by winsock. >> >>--- >>Arno Garrels [TeamICS] >> >> >> >> >>>Thanks! >>> >>>David >>> >>>Wilfried Mestdagh wrote: >>> >>> >>> >>>>Hello David, >>>> >>>> >>>> >>>> >>>> >>>>>How do I know the maximum size possible? >>>>> >>>>> >>>>> >>>>> >>>>at receiving side, specially depending on the speed of your own program >>>>you mostly never get above 8 kb. However I have seen receiving packets >>>>10 time as high. But you dont have to care mutch, if you receive not >>>>all, then OnDataAvailable is immediatly called again, meaning if you >>>>have a temporary buffer that is to small, however if you receive on the >>>>stack then you can make him (almost) as large you wants. >>>> >>>> >>>> >>>> >>>> >>>>>I certainly don't want to malloc 100KB for a TCP packet... >>>>> >>>>> >>>>> >>>>> >>>>See above, you dont need to. >>>> >>>>--- >>>>Rgds, Wilfried [TeamICS] >>>>http://www.overbyte.be/eng/overbyte/teamics.html >>>>http://www.mestdagh.biz >>>> >>>>Saturday, October 29, 2005, 20:25, Kei wrote: >>>> >>>> >>>> >>>> >>>> >>>>>Hi! >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >>>>>I think I'm going to accept the reality that... TCP packets are splitted >>>>>into arbitrary sizes.. but!!! >>>>>How do I know the maximum size possible? What is the maximum size that a >>>>>packet can be? I certainly don't want to malloc 100KB for a TCP >>>>>packet... >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >>>>>Thanks! >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >>>>>David >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >>>>>Wilfried Mestdagh wrote: >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >>>>>>Hello David, >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>Hi! I'm new to ICS! >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>Welcome to the group :) >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>A->>B: msg hello >>>>>>>B->>A: msg yo! how's it going? >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>Yes that's the way to go. Design a user made proto for what you intend >>>>>>to do. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>If A is larger than the default buffer size (256 chars) then the A >>>>>>>(sender) will warn B in advance >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>You can do that, but is not nececary. Also you will maybe have a very >>>>>>mutch allocation / deallocation of memory and you can eventually end up >>>>>>with fragmented memory where you have not a nice large block in it at >>>>>>the moment you need it. But it can work, just think over carefully. A >>>>>>better idea is often to make a receive buffer that grows automatically >>>>>>if (and only if) needed, and then just reuse that buffer over and over >>>>>>again. Then you have some (re)allocation in begin but then stable. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>TWSocket will automatically split it into packets, >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>Winsock will split in packets as large as the MTU (around 1500 bytes). >>>>>>Eventually data can (and will if high speed) arrive as 1 large packet >>>>>>but not necacarely. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>I really want the sender side to send the 1MB file all at once, since >>>>>>>I do the FileWrite() right after Receive().... >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>No you cannot. Winsock does not respect packet boundaries, but (see >>>>>>prior paragraph) there are no megabytes TCP packets. >>>>>> >>>>>>You have to receive all data chuncks into a buffer, and when you >>>>>>received them all then you save to file (or save every packet direct to >>>>>>disk). There is no problem to know the moment of close the file because >>>>>>you know the length of the data from your protocol. >>>>>> >>>>>>--- >>>>>>Rgds, Wilfried [TeamICS] >>>>>>http://www.overbyte.be/eng/overbyte/teamics.html >>>>>>http://www.mestdagh.biz >>>>>> >>>>>>Saturday, October 29, 2005, 11:01, Kei wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>Hi! I'm new to ICS! >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>I am designing a simple protocol that will be mainly used locally (as >>>>>>>a database server backend)..so I'm guessing I could send up to 2GB of >>>>>>>stuff without hassle (BLOBs, for example). Right now I'm just >>>>>>>experimenting with the facility for two parties to effectively "talk" >>>>>>>to each other, even with long long messages, and with binary data >>>>>>>transfer. For example, for sending a message, the command is "msg >>>>>>>[text]" >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>A->>B: msg hello >>>>>>B->>A: msg yo! how's it going? >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>If A is larger than the default buffer size (256 chars) then the A >>>>>>>(sender) will warn B in advance, like this: "hey! I'm gonna send you >>>>>>>10000 bytes of text" >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>A->>B: longmsg 10000 >>>>>>B->>A: ready msg >>>>>>A->>B: msg blahblahblah...........blah! >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>In this case, B will be notified of the text size, then when >>>>>>>OnClientDataAvailable() event comes, it will malloc a bigger buffer, >>>>>>>then Receive(CustomSized_Buffer, SizeHeToldMe). Similarly Im >>>>>>>considering the same mechanism to send binary data. But if the file >>>>>>>is slightly >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>larger (>>10KB) then TWSocket will automatically split it into packets, >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>which I don't want it to do: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>A->>B: upload 1048576 picture.jpg >>>>>>B->>A: ready upload >>>>>>A->>B: 01001010101010.. (10720 bytes) >>>>>>A->>B: 11111010101012.. (10720 bytes) >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>A->>B: 01001010101010.. (4023 bytes) >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>I really want the sender side to send the 1MB file all at once, since >>>>>>>I do the FileWrite() right after Receive().... >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>Could anybody please help me on this issue? >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>Thanks! >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>David >>>>>>> >>>>>>> -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
