[fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Graeme Geldenhuys

Hi,

Does FPC have UTF-8 versions of the Copy() and Length() functions?

I've started implementing support for UTF-8 in my fpGUI widget set -
more specifically text painting in all components (the easy bit) and
text handling in the TFEdit (edit box widget).

For now I have used the UTF8Copy() and UTF8Length()  functions as
designed in the Lazarus LCL project and it works fine.  I'm pretty
sure FPC must have such functions, but I searched everywhere and
couldn't find anything.


--
Graeme Geldenhuys

** General error, hit any user to continue. **
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Daniël Mantione


Op Sat, 19 May 2007, schreef Graeme Geldenhuys:

> Does FPC have UTF-8 versions of the Copy() and Length() functions?

They don't exist. FPC has been designed to either use the system encoding 
(which can be utf8). In this case, the string routines from sysutils do 
what you want. The other option is to use widestrings; 
length(utf8decode(s)) will return the length of an utf-8 string.

Daniël___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Graeme Geldenhuys

On 5/19/07, Daniël Mantione <[EMAIL PROTECTED]> wrote:

> Does FPC have UTF-8 versions of the Copy() and Length() functions?

They don't exist. FPC has been designed to either use the system encoding
(which can be utf8). In this case, the string routines from sysutils do
what you want. The other option is to use widestrings;
length(utf8decode(s)) will return the length of an utf-8 string.


Sorry, I'm very new to Unicode support.  Wouldn't it be useful to have
UTF-8 and UTF-16 (and all the other encodings) functions in FPC?  For
example the Lazarus LCL (LCLProc unit) has loads of such functions.

The Length function is easy to get around, but the Copy, Pos ,etc
functions are not.



--
Graeme Geldenhuys

General error, hit any user to continue.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Daniël Mantione


Op Sat, 19 May 2007, schreef Graeme Geldenhuys:

> On 5/19/07, Daniël Mantione <[EMAIL PROTECTED]> wrote:
> > > Does FPC have UTF-8 versions of the Copy() and Length() functions?
> > 
> > They don't exist. FPC has been designed to either use the system encoding
> > (which can be utf8). In this case, the string routines from sysutils do
> > what you want. The other option is to use widestrings;
> > length(utf8decode(s)) will return the length of an utf-8 string.
>
> Sorry, I'm very new to Unicode support.  Wouldn't it be useful to have
> UTF-8 and UTF-16 (and all the other encodings) functions in FPC?  For
> example the Lazarus LCL (LCLProc unit) has loads of such functions.

I think a utf8 unit would be usefull.

> The Length function is easy to get around, but the Copy, Pos ,etc
> functions are not.

Why?

Daniël___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Graeme Geldenhuys

On 5/19/07, Daniël Mantione <[EMAIL PROTECTED]> wrote:


I think a utf8 unit would be usefull.


Glad you agree.  I searched through the mailing list and found quite a
few people that required such function and implemented their own.  So
yes, why don't we include a new unit with those functions for
everybody to use. The Lazarus project (LCLProc unit) has been tried
and tested for ages.  We can create a new unit based on the functions
in there - just remove any LCL specific functions.

Seeing that the LCLProc unit has been in use for ages in Lazarus, any
chance that we can get such a unit fast-tracked and included in trunk
and fixes_2_2 branch?  Ready for the next release.
It shouldn't interfere with any existing applications, because it will
be a new and separate unit. It's also been well tested in Lazarus.

So what name could we give such a unit.  unicodeutils.pas?  utfutils.pas?


> The Length function is easy to get around, but the Copy, Pos ,etc
> functions are not.

Why?


I meant not easy by inline'ing things like your Length example.  The
UTF8Copy() function requires a bit more coding, that's all.



--
Graeme Geldenhuys

General error, hit any user to continue.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Felipe Monteiro de Carvalho

On 5/19/07, Daniël Mantione <[EMAIL PROTECTED]> wrote:

I think a utf8 unit would be usefull.


Some things need to be choosen:

* which functions from lclproc would be included

* the position of the unit on fpc (maybe fcl? but then, which
directory from fcl)

* the exact name of the unit

thanks,
--
Felipe Monteiro de Carvalho
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Daniël Mantione


Op Sat, 19 May 2007, schreef Felipe Monteiro de Carvalho:

> On 5/19/07, Daniël Mantione <[EMAIL PROTECTED]> wrote:
> > I think a utf8 unit would be usefull.
> 
> Some things need to be choosen:
> 
> * which functions from lclproc would be included
> 
> * the position of the unit on fpc (maybe fcl? but then, which
> directory from fcl)

Not FCL. FCL is for classes based OOP code, while utf-8 has a much broader 
scope. I'd say RTL.
 
> * the exact name of the unit

It depends on the first point. If it is just utf-8, utf8 would be a good 
name.

Daniël___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Michael Van Canneyt


On Sat, 19 May 2007, Felipe Monteiro de Carvalho wrote:

> On 5/19/07, Daniël Mantione <[EMAIL PROTECTED]> wrote:
> > I think a utf8 unit would be usefull.
> 
> Some things need to be choosen:
> 
> * which functions from lclproc would be included
> 
> * the position of the unit on fpc (maybe fcl? but then, which
> directory from fcl)

No, it should go in the RTL. It's very basic.

> 
> * the exact name of the unit

utf8 seems appropriate, or utf if more than one encoding is supported.

Michael.___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Daniël Mantione


Op Sat, 19 May 2007, schreef Graeme Geldenhuys:

> On 5/19/07, Daniël Mantione <[EMAIL PROTECTED]> wrote:
> > 
> > I think a utf8 unit would be usefull.
> 
> Glad you agree.  I searched through the mailing list and found quite a
> few people that required such function and implemented their own.  So
> yes, why don't we include a new unit with those functions for
> everybody to use. The Lazarus project (LCLProc unit) has been tried
> and tested for ages.  We can create a new unit based on the functions
> in there - just remove any LCL specific functions.
> 
> Seeing that the LCLProc unit has been in use for ages in Lazarus, any
> chance that we can get such a unit fast-tracked and included in trunk
> and fixes_2_2 branch?  Ready for the next release.

As far as I am concerned, it can only go in fixes_2_2 after it has proven 
to be stable and should not disrupt the 2.2 release. In other words, it 
might not make 2.2.0, but could make 2.2.2.

> It shouldn't interfere with any existing applications, because it will
> be a new and separate unit. It's also been well tested in Lazarus.
> 
> So what name could we give such a unit.  unicodeutils.pas?  utfutils.pas?
> 
> > > The Length function is easy to get around, but the Copy, Pos ,etc
> > > functions are not.
> > 
> > Why?
> 
> I meant not easy by inline'ing things like your Length example.  The
> UTF8Copy() function requires a bit more coding, that's all.

It is mostly usefull if you use widestrings internally. If you use utf-8 
internally I agree an utf-8 would be more comfortable.

Daniël___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Felipe Monteiro de Carvalho

On 5/19/07, Michael Van Canneyt <[EMAIL PROTECTED]> wrote:

utf8 seems appropriate, or utf if more than one encoding is supported.


I think that having a generic name is better, even if at first we only
add utf-8 functions, because when we need more functions for other
encodings we would need to create another unit, or rename this one,
and that's unnecessary work

--
Felipe Monteiro de Carvalho
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Marco van de Voort
> Op Sat, 19 May 2007, schreef Felipe Monteiro de Carvalho:
> 
> > On 5/19/07, Dani?l Mantione <[EMAIL PROTECTED]> wrote:
> > > I think a utf8 unit would be usefull.
> > 
> > Some things need to be choosen:
> > 
> > * which functions from lclproc would be included
> > 
> > * the position of the unit on fpc (maybe fcl? but then, which
> > directory from fcl)
> 
> Not FCL. FCL is for classes based OOP code, while utf-8 has a much broader 
> scope. I'd say RTL.

Outer RTL? It is not needed for cycling :_)
  
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Rimgaudas Laucius


- Original Message - 
From: "Graeme Geldenhuys" <[EMAIL PROTECTED]>

To: "FPC-Pascal users discussions" 
Sent: Saturday, May 19, 2007 11:58 AM
Subject: Re: [fpc-pascal] UTF-8 versions of Copy() and Length()



On 5/19/07, Daniël Mantione <[EMAIL PROTECTED]> wrote:

> Does FPC have UTF-8 versions of the Copy() and Length() functions?

They don't exist. FPC has been designed to either use the system encoding
(which can be utf8). In this case, the string routines from sysutils do
what you want. The other option is to use widestrings;
length(utf8decode(s)) will return the length of an utf-8 string.


Sorry, I'm very new to Unicode support.  Wouldn't it be useful to have
UTF-8 and UTF-16 (and all the other encodings) functions in FPC?  For
example the Lazarus LCL (LCLProc unit) has loads of such functions.



You can find info on Unicode standard at 
http://www.unicode.org/versions/Unicode4.0.0/. 
http://www.unicode.org/versions/Unicode4.0.0/ch02.pdf presents Unicode 
encoding forms.


It is not useful to have functions for both encodings, because these 
encodings are interconvertable and it is more effective to use UTF-16 for 
data processing. Actually, UTF-8 is suitable only for storing of external 
dada, because it is more compact. It expresses characters that are outside 
ASCII as sequences of 8-bit code points (actually 2 or 3) while UTF-16 
expesses them using single (~actually) 16-bit code points. Thus processing 
of internal data (iterating, counting, etc.) using UTF-16 encoding may be 
done more effectivelly and easy.






The Length function is easy to get around, but the Copy, Pos ,etc
functions are not.



--
Graeme Geldenhuys

General error, hit any user to continue.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal




___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Felipe Monteiro de Carvalho

On 5/19/07, Rimgaudas Laucius <[EMAIL PROTECTED]> wrote:

It is not useful to have functions for both encodings, because these
encodings are interconvertable and it is more effective to use UTF-16 for
data processing


I disagree. The conversion impacts performance heavely. It will also
require memory to store the converted string, and after you perform a
operation you need to convert back.

Further, UTF-16 contains both 2-byte characters and 4-byte characters,
so I don't see how it would be any faster to process it in comparison
to process a utf-8 string.

About being easier to implement, that's irrelevant, because the
functions are already done.

--
Felipe Monteiro de Carvalho
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread ik

Hi,

I've started to create a unit for UTF-8, but lack of time is keeping
me from developing it at the moment. I have some implementation
already if you are interested, and wish to continue to develop it.

Ido

On 5/19/07, Graeme Geldenhuys <[EMAIL PROTECTED]> wrote:

Hi,

Does FPC have UTF-8 versions of the Copy() and Length() functions?

I've started implementing support for UTF-8 in my fpGUI widget set -
more specifically text painting in all components (the easy bit) and
text handling in the TFEdit (edit box widget).

For now I have used the UTF8Copy() and UTF8Length()  functions as
designed in the Lazarus LCL project and it works fine.  I'm pretty
sure FPC must have such functions, but I searched everywhere and
couldn't find anything.


--
Graeme Geldenhuys

** General error, hit any user to continue. **
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal




--
http://ik.homelinux.org/
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Rimgaudas Laucius

Storage:
UTF8UTF16.

Performance:
Length (UTF8) = UTF8->UTF16
2*Lenth(UTF8)> UTF8->UTF16

4-byte characters are used by UTF32. UTF16 uses sequences of 2 code points 
from surrogates area to expess charactes outside basic multilingual plane 
that are very rarely used (actully i do not know any program that implements 
that).




- Original Message - 
From: "Felipe Monteiro de Carvalho" <[EMAIL PROTECTED]>

To: "FPC-Pascal users discussions" 
Sent: Saturday, May 19, 2007 12:57 PM
Subject: Re: [fpc-pascal] UTF-8 versions of Copy() and Length()



On 5/19/07, Rimgaudas Laucius <[EMAIL PROTECTED]> wrote:

It is not useful to have functions for both encodings, because these
encodings are interconvertable and it is more effective to use UTF-16 for
data processing


I disagree. The conversion impacts performance heavely. It will also
require memory to store the converted string, and after you perform a
operation you need to convert back.

Further, UTF-16 contains both 2-byte characters and 4-byte characters,
so I don't see how it would be any faster to process it in comparison
to process a utf-8 string.

About being easier to implement, that's irrelevant, because the
functions are already done.

--
Felipe Monteiro de Carvalho
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal




___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Felipe Monteiro de Carvalho

On 5/19/07, Rimgaudas Laucius <[EMAIL PROTECTED]> wrote:

Performance:
Length (UTF8) = UTF8->UTF16
2*Lenth(UTF8)> UTF8->UTF16


I don't understand what you are trying to say here.

--
Felipe Monteiro de Carvalho
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Rimgaudas Laucius
I mean that any operation done on UTF8 string (deleting or inserting of part 
of string) consume as much as converting it to UTF16. So it will be less 
effective to do that operation twice with UTF8 string than to convert it to 
UTF16 where these operations may be done instantly.



- Original Message - 
From: "Felipe Monteiro de Carvalho" <[EMAIL PROTECTED]>

To: "FPC-Pascal users discussions" 
Sent: Saturday, May 19, 2007 1:36 PM
Subject: Re: [fpc-pascal] UTF-8 versions of Copy() and Length()



On 5/19/07, Rimgaudas Laucius <[EMAIL PROTECTED]> wrote:

Performance:
Length (UTF8) = UTF8->UTF16
2*Lenth(UTF8)> UTF8->UTF16


I don't understand what you are trying to say here.

--
Felipe Monteiro de Carvalho
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal




___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Graeme Geldenhuys

I just grabbed what I needed from the LCLProc unit (part of Lazarus)
which has a lot of UTF-8 and UTF-16 functions.

I'll put all those functions in a single unit and post it to fpc-devel
mailing list for review before we include it into FPC.

If you have any more functions, please pass them on, so I can include
them as well.

Regards,
 - Graeme -


On 5/19/07, ik <[EMAIL PROTECTED]> wrote:

Hi,

I've started to create a unit for UTF-8, but lack of time is keeping
me from developing it at the moment. I have some implementation
already if you are interested, and wish to continue to develop it.

Ido



--
Graeme Geldenhuys

General error, hit any user to continue.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Daniël Mantione


Op Sat, 19 May 2007, schreef Felipe Monteiro de Carvalho:

> On 5/19/07, Rimgaudas Laucius <[EMAIL PROTECTED]> wrote:
> > It is not useful to have functions for both encodings, because these
> > encodings are interconvertable and it is more effective to use UTF-16 for
> > data processing
> 
> I disagree. The conversion impacts performance heavely. It will also
> require memory to store the converted string, and after you perform a
> operation you need to convert back.
>
> Further, UTF-16 contains both 2-byte characters and 4-byte characters,
> so I don't see how it would be any faster to process it in comparison
> to process a utf-8 string.

For most operations, it is not necessary to process characters outside 
the BMP separately, i.e.:

for i:=1 to length(s) do
  s[i]:=upcase(i);

... is valid UTF-16 code, and much faster than the same operation in 
UTF-8.

Daniël___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread ik

Hi,

Here is what I made so far,
http://ik.homelinux.org/projects/files/ikunicode.tar.gz

You can take anything you wish. Please note that when I'll have time,
I'll continue working on the unit itself. I also wanted it to be
included on the rtl of FPC, but I don't see it happens anytime soon...
:(

Ido

On 5/19/07, Graeme Geldenhuys <[EMAIL PROTECTED]> wrote:

I just grabbed what I needed from the LCLProc unit (part of Lazarus)
which has a lot of UTF-8 and UTF-16 functions.

I'll put all those functions in a single unit and post it to fpc-devel
mailing list for review before we include it into FPC.

If you have any more functions, please pass them on, so I can include
them as well.

Regards,
  - Graeme -


On 5/19/07, ik <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I've started to create a unit for UTF-8, but lack of time is keeping
> me from developing it at the moment. I have some implementation
> already if you are interested, and wish to continue to develop it.
>
> Ido
>

--
Graeme Geldenhuys

General error, hit any user to continue.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal




--
http://ik.homelinux.org/
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Florian Klaempfl
Graeme Geldenhuys schrieb:
> On 5/19/07, Daniël Mantione <[EMAIL PROTECTED]> wrote:
>> > Does FPC have UTF-8 versions of the Copy() and Length() functions?
>>
>> They don't exist. FPC has been designed to either use the system encoding
>> (which can be utf8). In this case, the string routines from sysutils do
>> what you want. The other option is to use widestrings;
>> length(utf8decode(s)) will return the length of an utf-8 string.
> 
> Sorry, I'm very new to Unicode support.  Wouldn't it be useful to have
> UTF-8 and UTF-16 (and all the other encodings) functions in FPC? 

It's not that easy, for example upper/lower casing, comparing etc. of
unicode requires usually external big tables which we don't want to pull
into fpc.

> For
> example the Lazarus LCL (LCLProc unit) has loads of such functions.
> 
> The Length function is easy to get around, but the Copy, Pos ,etc
> functions are not.
> 
> 
> 

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Rimgaudas Laucius


- Original Message - 
From: "Florian Klaempfl" <[EMAIL PROTECTED]>

To: "FPC-Pascal users discussions" 
Sent: Saturday, May 19, 2007 4:45 PM
Subject: Re: [fpc-pascal] UTF-8 versions of Copy() and Length()



Graeme Geldenhuys schrieb:

On 5/19/07, Daniël Mantione <[EMAIL PROTECTED]> wrote:

> Does FPC have UTF-8 versions of the Copy() and Length() functions?

They don't exist. FPC has been designed to either use the system 
encoding

(which can be utf8). In this case, the string routines from sysutils do
what you want. The other option is to use widestrings;
length(utf8decode(s)) will return the length of an utf-8 string.


Sorry, I'm very new to Unicode support.  Wouldn't it be useful to have
UTF-8 and UTF-16 (and all the other encodings) functions in FPC?


It's not that easy, for example upper/lower casing, comparing etc. of
unicode requires usually external big tables which we don't want to pull
into fpc.



I agree. Though smartly build Unicode table is not such big (it may increase 
executable size by ~80KB), but it is absolutely not necessary. All 
contemporary platforms have sufficient Unicode services. It is easy to 
employ them. 



___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Felipe Monteiro de Carvalho

On 5/19/07, Florian Klaempfl <[EMAIL PROTECTED]> wrote:

It's not that easy, for example upper/lower casing, comparing etc. of
unicode requires usually external big tables which we don't want to pull
into fpc.


Yes, but if the unicode functions are all in a separate unit, they
won´t be included in your executable, unless you really want it.

Plus I don´t remember any tables on the utf function on Lclproc, but
maybe I didn´t pay enougth attention

--
Felipe Monteiro de Carvalho
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Mattias Gaertner
On Sat, 19 May 2007 18:07:44 +0200
"Felipe Monteiro de Carvalho" <[EMAIL PROTECTED]> wrote:

> On 5/19/07, Florian Klaempfl <[EMAIL PROTECTED]> wrote:
> > It's not that easy, for example upper/lower casing, comparing etc.
> > of unicode requires usually external big tables which we don't want
> > to pull into fpc.
> 
> Yes, but if the unicode functions are all in a separate unit, they
> won´t be included in your executable, unless you really want it.
> 
> Plus I don´t remember any tables on the utf function on Lclproc, but
> maybe I didn´t pay enougth attention

Only simple helper functions are in LCLProc.
A lower/uppercase Unicode function is not simple and when eventually
the LCL needs it, it would simply delegate the implementation to
the LCL interfaces, which can simply call the appropriate library
functions.


Mattias
 
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Florian Klaempfl
All these unicode functions should code through the widestring mananger
so they get plugable.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Graeme Geldenhuys

On 5/19/07, Felipe Monteiro de Carvalho
<[EMAIL PROTECTED]> wrote:

Yes, but if the unicode functions are all in a separate unit, they
won´t be included in your executable, unless you really want it.


Yup, I agree on this one...


--
Graeme Geldenhuys

General error, hit any user to continue.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Graeme Geldenhuys

On 5/19/07, Mattias Gaertner <[EMAIL PROTECTED]> wrote:


Only simple helper functions are in LCLProc.
A lower/uppercase Unicode function is not simple and when eventually
the LCL needs it, it would simply delegate the implementation to
the LCL interfaces, which can simply call the appropriate library
functions.


Which is why you must love Open Source software. :-)
Another Object Pascal based GUI Framework has already done all the
work for us and they used a BSD license so we are free to use what we
need.

Have a look at the LPTK project on Source Forge. Two units called
schar16.pas and unitkeys.pas

It contains a lot of functions: Upper, Lower, Pos, Length, conversions
from UTF-8 to UTF-16 and vice versa, character translations for
Latin1-Latin4, Arabic, Kana, Hungarian, Cyrillic, Hebrew, Thai, etc...
Lots we can use!


--
Graeme Geldenhuys

General error, hit any user to continue.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread ik

On 5/19/07, Graeme Geldenhuys <[EMAIL PROTECTED]> wrote:

On 5/19/07, Mattias Gaertner <[EMAIL PROTECTED]> wrote:
>
> Only simple helper functions are in LCLProc.
> A lower/uppercase Unicode function is not simple and when eventually
> the LCL needs it, it would simply delegate the implementation to
> the LCL interfaces, which can simply call the appropriate library
> functions.

Which is why you must love Open Source software. :-)
Another Object Pascal based GUI Framework has already done all the
work for us and they used a BSD license so we are free to use what we
need.

Have a look at the LPTK project on Source Forge. Two units called
schar16.pas and unitkeys.pas

It contains a lot of functions: Upper, Lower, Pos, Length, conversions
from UTF-8 to UTF-16 and vice versa, character translations for
Latin1-Latin4, Arabic, Kana, Hungarian, Cyrillic, Hebrew, Thai, etc...
 Lots we can use!


The reason I started writing my own library is because I wanted pure
Pascal libraries to use things. I dislike bindings to other libraries
written in different languages (such as QT). "I have a dream" to see
native tools for all common needs written in pure (Object) Pascal.

Secondly, I'm non Latin native speaker, so I find the need for Unicode
on all of it's glory (and bad things) a must rather then something
that can only be "good" additional support.

On my code, you can see that I prepared the ground for working with
TWideStringManager.
Oh, and it's MIT license (not as restrictive as BSD license ;))

Ido




--
Graeme Geldenhuys

General error, hit any user to continue.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal




--
http://ik.homelinux.org/
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] UTF-8 versions of Copy() and Length()

2007-05-19 Thread Graeme Geldenhuys

On 5/19/07, ik <[EMAIL PROTECTED]> wrote:

The reason I started writing my own library is because I wanted pure
Pascal libraries to use things. I dislike bindings to other libraries
written in different languages (such as QT). "I have a dream" to see
native tools for all common needs written in pure (Object) Pascal.


I like that dream... Hence the reason I work on the fpGUI Widget Set.
A GUI Framework implemented in Object Pascal, no huge 3rd Party
library requirements and easy portable to other systems. Currently
runs on Linux (Xlib) and Windows (GDI) and the graphics backend tested
on Windows Mobile.
See http://opensoft.homeip.net/fpgui/


Secondly, I'm non Latin native speaker, so I find the need for Unicode
on all of it's glory (and bad things) a must rather then something
that can only be "good" additional support.


I don't require Unicode support in fpGUI but others do, like Felipe. I
also don't want one platform having features another doesn't. Hence
the reason I started with Unicode support while running under X11
(Felipe already did it for GDI).

--
Graeme Geldenhuys

General error, hit any user to continue.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


[fpc-pascal] Release of fpc-2.1.4 aka 2.2.0-beta

2007-05-19 Thread Joost van der Sluis
Hello everybody,

I'm happy to announce that release 2.1.4 aka 2.2.0 beta is out. We ask
our users to test the changes made in the last few years. This beta will
be available for about two months, whereafter 2.2.0 will be released.
Helping us to test version 2.1.4 now, can avoid problems when you
finally update your fpc-version to 2.2.0.

Greetings

Joost
(on behalf of the whole FPC team)

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal