Re: [R-pkg-devel] Package Encoding and Literal Strings

[jo...@jorisgoosen.nl]

Hello Tomas,

On Mon, 21 Dec 2020 at 21:21, Tomas Kalibera 
wrote:

> Hi Joris,
>
> On 12/21/20 7:33 PM, jo...@jorisgoosen.nl wrote:
>
> Hello Tomas,
>
> Thank you for the feedback and your summary of how things now work and
> what goes wrong for the tao- and mathot-string confirms all of my
> suspicions. And it also describes my exact problem fairly well.
>
> It seems it does come down to R not keeping the UTF-8 encoding of the
> literal strings on Windows with a "typical codepage" when loading a
> package.
> This despite reading it from file in that particular encoding and also
> specifying the same in DESCRIPTION.
> While `eval(parse(..., encoding="UTF-8"))` *does* keep the encoding on the
> literal strings. Which means there is some discrepancy between the two.
> That means the way a package is loaded it uses a different path then when
> using `eval(parse(..., encoding="UTF-8"))`?
>
> Yes, it must be a different path. The DESCRIPTION field defines what
> encoding is the input in, so that R can read it. It does not tell R how it
> should represent the strings internally. The behavior is ok, well except
> for non-representable characters.
>
> You mention:
> > Strings that cannot be represented in the native encoding like tao will
> get the escapes, and so cannot be converted back to UTF-8. This is not
> great, but I  see it was the case already in 3.6 (so not a recent
> regression) and I don't think it would be worth the time trying to fix that
> - as discussed earlier, only switching to UTF-8 would fix all of these
> translations, not just one.
>
> Not a recent regression means it used to work the same for both and
> keeping the UTF-8 encoding?
> I've tried R 3 and it already doesnt work there, I also tried 2.8 but
> couldnt get my testpkg (simplified to use "charToRaw" instead of a C-call)
> to install there.
> However, having this work would already be quite useful as our custom GUI
> on top of R is fully UTF-8 anyhow.
>
> By "not a recent regression" I meant it wasn't broken recently. It
> probably never worked the way you (and me and probably everyone else) would
> like it to work, that is it probably always translated to native encoding,
> because that was the only option except rewriting all of our code,
> packages, external libraries to use UTF-16LE (as discussed before).
>

Too bad, but that was what I was afraid of in the first place.

> And I would certainly be up for figuring out how to fix the regression so
> that we can use this until your work on the UTF-8 version with UCRT is
> released.
> On the other hand, maybe this would not be the wisest investment of my
> time.
>
> I bet your applications do more than just load a package and then access
> string literals in the code. And as soon as you do anything with those
> strings, R may translate them to native encoding (well unless we document
> this does not happen, typically some code around connections, file paths,
> etc). So, providing a shortcut for this case I am afraid wouldn't help you
> much. If the problem was just parsing, you could also use "\u" escapes as
> workaround in the literals. Remember, the parse(,encoding="UTF-8") only
> could work in single-byte encodings.
>
Ah yeah, the original problem with that was that the `xgettext`
parsingscript doesn't know how to handle those escapes. But that means we
will just have to fix that then.

> I've tried using the installer and toolchain you linked to in
> https://developer.r-project.org/Blog/public/2020/07/30/windows/utf-8-build-of-r-and-cran-packages/index.html
> and use that to compile our software.
> This normally works with the Rtools toolchain, but it seems that "make" is
> missing from your toolchain. When I build (our project with Riniside in it)
> using your toolchain in the beginning of PATH and using mingw32-make from
> rtools40 I run into problems of a missing "cc1plus".
>
> Sorry, building native code is still involved with that demo. You would
> have to set PATHs and well maybe alter the installation or build from
> source, as described in
>
> https://svn.r-project.org/R-dev-web/trunk/WindowsBuilds/winutf8/winutf8.html
>
> What might be actually easier, you could try a current development
> version, I will send you a link.
>

Cheers,
Joris

I got the link and will have a go at that and reply there with any remarks
or questions.

> If I read https://mxe.cc/ it seems it is meant for cross-compiling, not
> locally on Windows?
> Maybe that is what is going wrong.
> But despite trying for quite a bit I couldn't get our software to compile
> in such a way it could link with R.
> Which means I couldn't test if it solves our problem...
>
> You can compile native code locally on Windows, the toolchain includes a
> native compiler and I build R packages natively as well. Cross-compilation
> is used to build the compiler toolchain and external libraries for
> packages.Cheers
> Tomas
>

> Cheers,
> Joris
>
>
> On Fri, 18 Dec 2020 at 18:05, Tomas Kalibera 
> wrote:
>
>> Hi Joris,
>>
>

Re: [R-pkg-devel] Question about package dependency

[Neal Fultz]

Unless there is something Oracle-specific for your database code, I'd
recommend using the DBI interface instead - "enterprise" users can
provide an Oracle object to your functions, and other people could use
their own particular databases.

DBI itself could be a Suggests dependency if it isn't strictly required.


On Tue, Dec 22, 2020 at 5:09 PM Sebastien Bihorel
 wrote:
>
> Hi
>
> I am working on modifying a package originally developed for use within my 
> company, with the intent to distribute it to users external to our company.
> The package currently depends on ROracle to extract data from an Oracle 
> database. The modifications I am implementing would allow the package to 
> operate under 2 modes of data extraction: from this database for internal 
> users or from locally stored data files for external users. Beyond the data 
> extraction step, the package would provide functionality to manipulate the 
> data which is independent of their origin.
>
> I was wondering if and how I could set up my package code and DESCRIPTION 
> file so that external users don't have to install ROracle to install and use 
> my package (since they would operate only in the 2nd mode of data extraction 
> described above).
>
> Thank in advance for your suggestions.
>
>
> [[alternative HTML version deleted]]
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

[R-pkg-devel] Question about package dependency

[Sebastien Bihorel]

Hi

I am working on modifying a package originally developed for use within my 
company, with the intent to distribute it to users external to our company.
The package currently depends on ROracle to extract data from an Oracle 
database. The modifications I am implementing would allow the package to 
operate under 2 modes of data extraction: from this database for internal users 
or from locally stored data files for external users. Beyond the data 
extraction step, the package would provide functionality to manipulate the data 
which is independent of their origin.

I was wondering if and how I could set up my package code and DESCRIPTION file 
so that external users don't have to install ROracle to install and use my 
package (since they would operate only in the 2nd mode of data extraction 
described above).

Thank in advance for your suggestions.


[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Question about package dependency

[Sebastien Bihorel]

Thanks David,

I will look into your suggestion more.

Sebastien


From: David Bosak 
Sent: Tuesday, December 22, 2020 20:27
To: Sebastien Bihorel ; 
r-package-devel@r-project.org 
Subject: RE: [R-pkg-devel] Question about package dependency


Hi Sebastian:



What you describe sounds like it could be handled by the �suggests� section of 
the Description file.   Then you can conditionally require it for your 
different user groups, perhaps based on a configuration setting.  See if this 
helps:  https://r-pkgs.org/description.html



David





Sent from Mail for Windows 10



From: Sebastien Bihorel
Sent: Tuesday, December 22, 2020 8:09 PM
To: r-package-devel@r-project.org
Subject: [R-pkg-devel] Question about package dependency



Hi



I am working on modifying a package originally developed for use within my 
company, with the intent to distribute it to users external to our company.

The package currently depends on ROracle to extract data from an Oracle 
database. The modifications I am implementing would allow the package to 
operate under 2 modes of data extraction: from this database for internal users 
or from locally stored data files for external users. Beyond the data 
extraction step, the package would provide functionality to manipulate the data 
which is independent of their origin.



I was wondering if and how I could set up my package code and DESCRIPTION file 
so that external users don't have to install ROracle to install and use my 
package (since they would operate only in the 2nd mode of data extraction 
described above).



Thank in advance for your suggestions.





[[alternative HTML version deleted]]



__

R-package-devel@r-project.org mailing list

https://stat.ethz.ch/mailman/listinfo/r-package-devel



[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Question about package dependency

[Sebastien Bihorel]

Hi Neal,

The database is beyond specific in our case and for internal use only. External 
users should not be able to interact with it.

I will look into the Suggests approach.

Thanks


From: Neal Fultz 
Sent: Tuesday, December 22, 2020 20:34
To: Sebastien Bihorel 
Cc: r-package-devel@r-project.org 
Subject: Re: [R-pkg-devel] Question about package dependency

Unless there is something Oracle-specific for your database code, I'd
recommend using the DBI interface instead - "enterprise" users can
provide an Oracle object to your functions, and other people could use
their own particular databases.

DBI itself could be a Suggests dependency if it isn't strictly required.


On Tue, Dec 22, 2020 at 5:09 PM Sebastien Bihorel
 wrote:
>
> Hi
>
> I am working on modifying a package originally developed for use within my 
> company, with the intent to distribute it to users external to our company.
> The package currently depends on ROracle to extract data from an Oracle 
> database. The modifications I am implementing would allow the package to 
> operate under 2 modes of data extraction: from this database for internal 
> users or from locally stored data files for external users. Beyond the data 
> extraction step, the package would provide functionality to manipulate the 
> data which is independent of their origin.
>
> I was wondering if and how I could set up my package code and DESCRIPTION 
> file so that external users don't have to install ROracle to install and use 
> my package (since they would operate only in the 2nd mode of data extraction 
> described above).
>
> Thank in advance for your suggestions.
>
>
> [[alternative HTML version deleted]]
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel