Re: [fpc-pascal] Performance problems with Image Conversions
On Sat, 23 Feb 2013, Andrew Brunner wrote: Hi guys, I just finished (a beta) of a collage app for Aurawin. I'm seeing a huge load on my servers when scaling images using the Image factory classes for JPEG, and PNG. Prefix: 6 Core AMD 64 32GB RAM Ubuntu 12.10 x64 8 Core AMD 64 32GB RAM Ubuntu 12.10 x64 FPC/Lazarus daily svn/trunk Overview: 1.) end-user grabs references to files (stored on my server) 2.) back-end core object scales camera photos to something more net friendly 512px max to scale. 3.) server streams photos back to client over the web and 4.) server assembles XML stream of Mime encoded file data for things like https://aurawin.com/cb?3 Try to get rid of the XML, it is always going to be slow. Using XML can slow your applications with a factor 6. If you are mime-encoding an image, it'll be even more. Specifics The pics in this sample take about 30 seconds to go from their originals to 512px. The app waits for the collage to save to the cloud. The problem gets worse for larger collages with more images. iX,iY are no larger than 512 class function Image.Transform(Stream:TStream; var sContentType,srcKind,dstKind:string; var iX,iY:Integer):boolean; var FReader : TFPCustomImageReader; FWriter : TFPCustomImageWriter; cReader : TFPCustomImageReaderClass; cWriter : TFPCustomImageWriterClass; Factor : Double; FTransparent : boolean; dstImg : TFPMemoryImage; srcImg : TFPMemoryImage; srcCanvas: TFPImageCanvas; dstCanvas: TFPImageCanvas; procedure FreeImageData; begin if (srcCanvas<>nil) then srcCanvas.Free(); Do a FreeAndNil(srcCanvas) here. 1. Free already checks if the pointer is not nil. You do not need to check again. 2. FreeAndNil will make sure the pointer is Nil srcImg:=TFPMemoryImage.Create(0,0); Do not use TFPMemoryImage. It is a catch-all memory format, not optimized at all, using 64 bits for the images. Instead, use e.g. TFPCompactImgRGBA8Bit if you need alpha or TFPCompactImgRGB8Bit if you do not need Alpha. dstImg:=TFPMemoryImage.Create(iX,iY); dstImg.UsePalette:=false; dstCanvas:=TFPImageCanvas.create(dstImg); dstCanvas.StretchDraw(0,0,iX,iY,srcImg); Phew, this is also a real killer. Do not draw on a canvas with stretchdraw. That will be slow as well; Rather, attempt to manipulate the image in memory by doing a stretch directly. Choose an algorithm which is fast. The default interpolation will result in quite a lot of calculations. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Performance problems with Image Conversions
On 24.02.2013 11:44, Michael Van Canneyt wrote: srcImg:=TFPMemoryImage.Create(0,0); Do not use TFPMemoryImage. It is a catch-all memory format, not optimized at all, using 64 bits for the images. Instead, use e.g. TFPCompactImgRGBA8Bit if you need alpha or TFPCompactImgRGB8Bit if you do not need Alpha. Nice, I did not know we have more compact image types available. O.o dstImg:=TFPMemoryImage.Create(iX,iY); dstImg.UsePalette:=false; dstCanvas:=TFPImageCanvas.create(dstImg); dstCanvas.StretchDraw(0,0,iX,iY,srcImg); Phew, this is also a real killer. Do not draw on a canvas with stretchdraw. That will be slow as well; Rather, attempt to manipulate the image in memory by doing a stretch directly. Choose an algorithm which is fast. The default interpolation will result in quite a lot of calculations. Maybe a collection of algorithms for in memory rescaling would be nice... Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Performance problems with Image Conversions
On Sun, 24 Feb 2013, Sven Barth wrote: On 24.02.2013 11:44, Michael Van Canneyt wrote: srcImg:=TFPMemoryImage.Create(0,0); Do not use TFPMemoryImage. It is a catch-all memory format, not optimized at all, using 64 bits for the images. Instead, use e.g. TFPCompactImgRGBA8Bit if you need alpha or TFPCompactImgRGB8Bit if you do not need Alpha. Nice, I did not know we have more compact image types available. O.o If memory serves well, Mattias Gaertner made them. dstImg:=TFPMemoryImage.Create(iX,iY); dstImg.UsePalette:=false; dstCanvas:=TFPImageCanvas.create(dstImg); dstCanvas.StretchDraw(0,0,iX,iY,srcImg); Phew, this is also a real killer. Do not draw on a canvas with stretchdraw. That will be slow as well; Rather, attempt to manipulate the image in memory by doing a stretch directly. Choose an algorithm which is fast. The default interpolation will result in quite a lot of calculations. Maybe a collection of algorithms for in memory rescaling would be nice... There are a lot of them, that is what the interpolations are for, but they work with a canvas. The case of rescaling an image and simply send the resulting image as a file was not foreseen at the time of writing. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Performance problems with Image Conversions
On 24.02.2013 12:43, Michael Van Canneyt wrote: On Sun, 24 Feb 2013, Sven Barth wrote: On 24.02.2013 11:44, Michael Van Canneyt wrote: srcImg:=TFPMemoryImage.Create(0,0); Do not use TFPMemoryImage. It is a catch-all memory format, not optimized at all, using 64 bits for the images. Instead, use e.g. TFPCompactImgRGBA8Bit if you need alpha or TFPCompactImgRGB8Bit if you do not need Alpha. Nice, I did not know we have more compact image types available. O.o If memory serves well, Mattias Gaertner made them. According to the SVN log your memory is correct. :) dstImg:=TFPMemoryImage.Create(iX,iY); dstImg.UsePalette:=false; dstCanvas:=TFPImageCanvas.create(dstImg); dstCanvas.StretchDraw(0,0,iX,iY,srcImg); Phew, this is also a real killer. Do not draw on a canvas with stretchdraw. That will be slow as well; Rather, attempt to manipulate the image in memory by doing a stretch directly. Choose an algorithm which is fast. The default interpolation will result in quite a lot of calculations. Maybe a collection of algorithms for in memory rescaling would be nice... There are a lot of them, that is what the interpolations are for, but they work with a canvas. The case of rescaling an image and simply send the resulting image as a file was not foreseen at the time of writing. It seems that it would be good if we'd refactor the interpolations then so that they can be used with either a canvas or an image as destination... Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
[fpc-pascal] FPC 2.6.2 released!
Hello, Finally, FPC 2.6.2 has landed. FPC 2.6.2 is an update to 2.6.0 that contains most library progress in 2012 and some crucial compiler fixes, as well as a few new targets. Building is still in progress and some formats (deb,rpm) and targets might not be available yet. Changes that may break backwards compatibility are documented at: http://wiki.freepascal.org/User_Changes_2.6.2 For Downloads, please use the FTP server at ftp://freepascal.stack.nl/pub/fpc/dist/2.6.2/ and sourceforge https://sourceforge.net/projects/freepascal/files/ as much possible. Enjoy! The Free Pascal Compiler Team Free Pascal Compiler Version 2.6.2 ** What's New in 2.6.2 ** Free Pascal 2.6.2 is a point release from the 2.6.0 fixes branch. Please also see http://wiki.freepascal.org/User_Changes_2.6.2 for a list of changes that may affect the behaviour of previously working code, and how to cope with these changes. Some highlights are: Compiler: * Improvements and fixes for the ARM architecture Packages: * New package fpindexer (indexing engine) * Support for observer pattern added to fcl-base (and base classes in RTL) * Lots and lots fixes and improvements for fcl-db * Support for JSON dataset added among others * Fixes and improvements for fcl-passrc (and fpdoc) * Updates for PTCPas and gtk2 * fpmkunit improvements (better support for future switch to fpmake) * Several fixes for x11 * Several fixes for winunits (and winceunits) Platforms: * Improvements to the NativeNT target (newly introduced as alpha in 2.6.0) * Many fixes for OpenBSD and NetBSD (considered in beta state now) * Internal ELF writer supported for more BSD targets * Fixes and improvements for gba and nds See http://bugs.freepascal.org/changelog_page.php for the list of reported bugs which have been fixed in this release. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Performance problems with Image Conversions
On Sun, 24 Feb 2013, Sven Barth wrote: Maybe a collection of algorithms for in memory rescaling would be nice... There are a lot of them, that is what the interpolations are for, but they work with a canvas. The case of rescaling an image and simply send the resulting image as a file was not foreseen at the time of writing. It seems that it would be good if we'd refactor the interpolations then so that they can be used with either a canvas or an image as destination... Yes. I'll put it on my todo list. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Performance problems with Image Conversions
On 02/24/2013 04:44 AM, Michael Van Canneyt wrote: Try to get rid of the XML, it is always going to be slow. Using XML can slow your applications with a factor 6. If you are mime-encoding an image, it'll be even more. I was just thinking about that. I think JSON would be ok, especially if I can use the RTL to get the data into the fields. XML code takes about 33% of my dbModules for each implementation. That would take a lot of work for each DBModule and CoreObject, both client (js) and server(pas). Do a FreeAndNil(srcCanvas) here. 1. Free already checks if the pointer is not nil. You do not need to check again. 2. FreeAndNil will make sure the pointer is Nil Done. Do not use TFPMemoryImage. It is a catch-all memory format, not optimized at all, using 64 bits for the images. Instead, use e.g. TFPCompactImgRGBA8Bit if you need alpha or TFPCompactImgRGB8Bit if you do not need Alpha. Done. I'm preparing to check for performance gains... dstImg:=TFPMemoryImage.Create(iX,iY); dstImg.UsePalette:=false; dstCanvas:=TFPImageCanvas.create(dstImg); dstCanvas.StretchDraw(0,0,iX,iY,srcImg); Phew, this is also a real killer. Do not draw on a canvas with stretchdraw. That will be slow as well; Rather, attempt to manipulate the image in memory by doing a stretch directly. Choose an algorithm which is fast. The default interpolation will result in quite a lot of calculations. Ok, I noticed some units surrounding the graphic code. Is this something that is illustrated on the wiki? Where do I get the methods? Anyone else is welcome to comment on scaling methods here... I have a feeling this would yield the most performance gains. Digital Photos scaled go from 3264x2448 down to 512x384 so the scaling is somewhat slow. The streaming is slow too... If not, does anyone have lightning fast scaling methods in C++ or Java that I can port/contribute? ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
[fpc-pascal] Regression? Problem passing a packed record to a overriden, virtual, abstract method in fpc 2.7.1.
Hi,
see the following little program:
program argumentbug;
{$MODE ObjFpc}
uses classes;
type
TVector3 = packed record
X, Y, Z: Single;
end;
TClassA = class
protected
fVector: TVector3;
public
procedure SetVector(AVector: TVector3); virtual; abstract;
end;
{ TClassB }
TClassB = class(TClassA)
public
procedure SetVector(AVector: TVector3); override;
end;
{ TClassB }
procedure TClassB.SetVector(AVector: TVector3);
begin
writeln('TClassB: ',AVector.X,',',AVector.Y,',',AVector.Z);
fVector:=AVector;
end;
var
MyVector: TVector3;
MyClassB: TClassB;
begin
MyVector.X:=0;
MyVector.Y:=0;
MyVector.Z:=3;
MyClassB:=TClassB.Create;
MyClassB.SetVector(MyVector);
end.
Obviously the expected output is 0,0,3 but in fact the output is 0,3,3.
I am sure that this used to work as my real code showing the problem is many
years old.
(The problem stays even if I don't declare the method in TClassA abstract.)
OS: Ubuntu 12.10 64-Bit
Compiler: FPC 2.7.1 Rev. 23655
R.
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
[fpc-pascal] differences between .pp and .pas files
Good day, Please what is a difference between .pp and .pas files extension? -- Kind Regards Eric Kom System Administrator & Programmer - Metropolitan College _ / You are scrupulously honest, frank, and \ | straightforward. Therefore you have few | \ friends./ - \ \ .--. |o_o | |:_/ | // \ \ (| Kom | ) /'\_ _/`\ \___)=(___/ 2 Hennie Van Till, White River, 1240 Tel: 013 750 2255 | Fax: 013 750 0105 | Cell: 078 879 1334 eric...@kom.za.net | eric...@metropolitancollege.co.za www.kom.za.net | www.kom.za.org | www.erickom.co.za Key fingerprint: 513E E91A C243 3020 8735 09BB 2DBC 5AD7 A9DA 1EF5 ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] differences between .pp and .pas files
At 09:34 PM 2/24/2013, Eric Kom wrote: Good day, Please what is a difference between .pp and .pas files extension? two characters... ;-) _ / You are scrupulously honest, frank, and \ | straightforward. Therefore you have few | \ friends./ - ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
[fpc-pascal] Re: differences between .pp and .pas files
On 25-2-2013 6:34, Eric Kom wrote: > Good day, > > Please what is a difference between .pp and .pas files extension? As Ralf said: 2 letters. I have the idea .pp was used in the past to differentiate from other Pascal compilers, while .pas is used more commonly now. Groete, Reinier ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
[fpc-pascal] Re: Performance problems with Image Conversions
On 24-2-2013 18:35, Andrew Brunner wrote: > If not, does anyone have lightning fast scaling methods in C++ or Java > that I can port/contribute? You might want to look if e.g. ImageMagick is faster for your needs. There's bindings for it in Pascal: http://wiki.lazarus.freepascal.org/PascalMagick Apparently, exactimage was designed to be faster than imagemagick: http://www.exactcode.de/site/open_source/exactimage/ It's apparently GPL. No idea about bindings for that. Some more info about the built in FPC image support: http://wiki.lazarus.freepascal.org/fcl-image#Known_Issues_and_limitations ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Re: differences between .pp and .pas files
2013/2/25 Reinier Olislagers > On 25-2-2013 6:34, Eric Kom wrote: > > Good day, > > > > Please what is a difference between .pp and .pas files extension? > > As Ralf said: 2 letters. I have the idea .pp was used in the past to > differentiate from other Pascal compilers, while .pas is used more > commonly now. > > Groete, > Reinier ... and, .pas is commonly found code of units (e.g.: yeahbaby.pas -> unit yeahbaby;); .pp is commonly found code that generates the executable (e.g.: ohgod.pp -> program ohgod;). -- Silvio Clécio My public projects - github.com/silvioprog ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
[fpc-pascal] unsubscribe
Hello Please unsubscribe me (no more use of free pascal) Thanks a lot -- Jean-françois Jouvet OPALES 140 rue Désiré Monnier 39000 Lons-le-Saunier TEL 03 84 24 88 66 FAX 03 84 24 88 67 jf.jou...@opales.fr ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
