Re: [Rd] Modification-proposal for %% (modulo) when supplied with double
Okay, thanks for your reactions.
I realized it's not something being "broken", just that it's a situation where
my intuition messes things up, and that in this situation it's not as easy as
using all.equal, which is my usual approach when working with floats.
But maybe I underestimated the impact a change would have, thanks for the
example Frederick.
Best,
Emil
On 11/09/2018, 21:10, "frede...@ofb.net" wrote:
Duncan, I think Emil realizes that the floating point format isn't
able to represent certain numbers, that's why he is suggesting this
change rather than complaining about our arithmetic being broken.
However, I agree with you that we should not adopt his proposal. It
would not make things more "user friendly" for people. Everyone has a
different application and a different use of %% and they just need to
keep in mind that they are talking to a computer and not a blackboard.
Here is an example of a feature that was meant to help users get more
intuitive results with floating point numbers, but which actually
caused headaches instead:
https://github.com/Rdatatable/data.table/issues/1642 It is a slightly
different scenario to this one, but I think it is still a good example
of how we can end up creating unforeseen problems for people if we
change core functionality to do unsolicited rounding behind the
scenes.
Best wishes,
Frederick
On Tue, Sep 11, 2018 at 12:11:29PM -0400, Duncan Murdoch wrote:
> On 11/09/2018 11:23 AM, Emil Bode wrote:
> > Hi all,
> >
> >
> >
> > Could we modify the "%%" (modulo)-operator to include some tolerance
for rounding-errors when supplied with doubles?
> >
> > It's not much work (patch supplied on the bottom), and I don't think it
would break anything, only if you were really interested in analysing rounding
differences.
> >
> > Any ideas about implementing this and overwriting base::`%%`, or would
we want another method (as I've done for the moment)?
>
> I think this is a bad idea. Your comments say "The
> \code{\link[base:Arithmetic]{`\%\%`}} operator calculates the modulo, but
> sometimes has rounding errors, e.g. "\code{(9.1/.1) \%\% 1}" gives ~ 1,
> instead of 0."
>
> This is false. The %% calculation is exactly correct. The rounding error
> happened in your input: 9.1/0.1 is not equal to 91, it is a little bit
> less:
>
> > options(digits=20)
> > 9.1/.1
> [1] 90.985789
>
> And %% did not return 1, it returned the correct value:
>
> > (9.1/.1) %% 1
> [1] 0.98578915
>
> So it makes no sense to change %%.
>
> You might argue that the division 9.1/.1 is giving the wrong answer, but
in
> fact that answer is correct too. The real problem is that in double
> precision floating point the numbers 9.1 and .1 can't be represented
> exactly. This is well known, it's in the FAQ (question 7.31).
>
> Duncan Murdoch
>
> >
> >
> >
> > Background
> >
> > I was writing some code where something has to happen at a certain
interval, with progress indicated, something like this:
> >
> > interval <- .001
> >
> > progress <- .1
> >
> > for(i in 1:1000*interval) {myFun(i); Sys.sleep(interval); if(i %%
progress, 0))) cat(i, '\n')}
> >
> > without interval and progress being known in advance. I could work
around it and make i integer, or do something like
> >
> > isTRUE(all.equal(i %% progress,0)) || isTRUE(all.equal(i %% progress,
progress),
> >
> > but I think my code is clearer as it is. And I like the idea behind
all.equal: we want double to approximately identical.
> >
> >
> >
> > So my patch (with roxygen2-markup):
> >
> > #' Modulo-operator with near-equality
> >
> > #'
> >
> > #' The \code{\link[base:Arithmetic]{`\%\%`}} operator calculates the
modulo, but sometimes has rounding errors, e.g. "\code{(9.1/.1) \%\% 1}" gives
~ 1, instead of 0.\cr
> >
> > #' Comparable to what all.equal does, this operator has some tolerance
for small rounding errors.\cr
> >
> > #' If the answer would be equal to the divisor within a small
tolerance, 0 is returned instead.
> >
> > #'
> >
> > #' For integer x and y, the normal \%\%-operator is used
> >
> > #'
> >
> > #' @usage `\%mod\%`(x, y, tolerance = sqrt(.Machine$double.eps))
> >
> > #' x \%mod\% y
> >
> > #' @param x,y numeric vectors, similar to those passed on to \%\%
> >
> > #' @param tolerance numeric, maximum difference, see
\code{\link[base]{all.equal}}. The default is ~ \code{1.5e-8}
> >
> > #' @return identical to the result for \%\%, unless the answer would be
really close to y, in which case 0 is returned
> >
> > #'
Re: [Rd] Bug 17432 in readLines with R >= 3.5.0 still a problem
Thanks, I responded to this on bugzilla.
On Wed, Sep 12, 2018 at 9:04 AM Chris Culnane
wrote:
>
> Bug 17432 (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17432) is
> still a problem when using pipes for IPC.
>
> The bug is evident when calling R from another process and trying to
> communicate via StdIn. R will buffer the input and not read lines until the
> buffer is exceeded or StdIn is closed by the sending process. This prevents
> interactive communication between a calling process and a child R process.
>
> From a quick look at the source code, it looks like the bug is caused by only
> disabling buffering when isatty() returns true for a file descriptor
> (connections.c). This fixes the original bug when the script is run in a
> terminal, but doesn't help for pipes, which will return false for isatty().
>
> An example R script and python script are provided to demonstrate the problem:
>
> R script (example.r):
>
> f <- file("stdin")
> open(f)
> while(length(line <- readLines(f,n=1)) > 0) {
> write(line, stderr())
> }
>
> Python3 script:
>
> import sys, os, subprocess
> process = subprocess.Popen(['Rscript', 'example.r'], stdin=subprocess.PIPE,
> stdout=subprocess.PIPE)
> for line in sys.stdin:
> process.stdin.write((line + '\n').encode('utf-8'))
> process.stdin.flush()
>
>
> Expected Behaviour:
> Run python script, each line entered is echoed back immediately by the R
> script - which is what happens on 3.4.4
>
> Observed Behaviiour on >=3.5.0 (include devel):
> The R script does not process lines as they are sent, it only receives them
> when StdIn is closed.
>
>
> Best Regards
>
> Chris
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Environments and parallel processing
+1 to what Gabor and Ralf said. In this case the memory address can be misleading. My understanding is that the environments in all the processes, 1 parent and 2 child, have the *same* memory address, but once you write to them the operating system makes copies and maps the address to a *different* physical address. This is the copy-on-write UNIX fork model. The example below shows 3 processes that all use the same address for the environment env1, yet there must be 3 different environments, because env1$x has 3 different values simultaneously. -Clark library(parallel) env1 <- new.env() env1$x = 0 cl <- makeCluster(2, type="FORK") # Now we write to the environment, and env1$x has two distinct values clusterEvalQ(cl, env1$x <- rnorm(1)) # [[1]] # [1] -1.296702 # # [[2]] # [1] -0.4001104 # The environments in the 2 child processes still have the same address, # which is the same as the original address parLapply(cl, 1:4, function(x) capture.output(str(env1))) env1 # The original x is unchanged env1$x stopCluster(cl) On Wed, Sep 12, 2018 at 11:31 AM Ralf Stubner wrote: > On 12.09.2018 20:20, Gábor Csárdi wrote: > > This is all normal, a fork cluster works with processes, that do not > > share memory. > > And if you are after shared-memory parallelism, you can try the 'Rdsm' > package: https://cran.r-project.org/package=Rdsm > > Greetings > Ralf > > -- > Ralf Stubner > Senior Software Engineer / Trainer > > daqana GmbH > Dortustraße 48 > 14467 Potsdam > > T: +49 331 23 61 93 11 > F: +49 331 23 61 93 90 > M: +49 162 20 91 196 > Mail: ralf.stub...@daqana.com > > Sitz: Potsdam > Register: AG Potsdam HRB 27966 P > Ust.-IdNr.: DE300072622 > Geschäftsführer: Prof. Dr. Dr. Karl-Kuno Kunze > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Possible bug with chromatic adaptation in grDevices::convertColor
It appears that the chromatic adaptation feature of `grDevices::convertColor`is
broken, and likely has been for many years. While a little surprising, it is
an obscure enough feature that there is some possibility this is actually
broken, as opposed to user error on my part. If it turns out to the latter, I
apologize in advance for spamming this list.
Consider:
rgb.in <- c("#CC", "#EE") clr <- t(col2rgb(rgb.in)) / 255
clr.out <- convertColor(clr, "sRGB", "sRGB") rgb(clr.out) ## [1]
"#CC" "#EE"
convertColor(clr, "sRGB", "sRGB", "D65", "D50") ## Error in
match.arg(from, nWhite) : ## 'arg' must be NULL or a character vector
This appears to be because `grDevices:::chromaticAdaptation` expects the
whitepoints to be provided in the character format (e.g. "D65"), but they are
already converted by `convertColor` into the tristimulus values. After
applying the patch at the end of this e-mail, we get:
clr.out <- convertColor(clr, "sRGB", "sRGB", "D65", "D50") rgb(clr.out)
## [1] "#DACAB0" "#FEECCE"
I do not have a great way of confirming that the conversion is correct with my
changes, but I did verify that the `M` matrix computed
within`grDevics:::chromaticAdaptation` for the "D65"->"D50" conversion
(approximately) matches the corresponding matrix from brucelindbloom.com
chromatic adaptation page:
http://www.brucelindbloom.com/Eqn_ChromAdapt.html
Additionally visual inspection via
scales::show_col(c(rgb.in, rgb(clr.out)))
is consistent with a shift from bluer indirect daylight ("D65") to yellower
direct daylight ("D50") illuminant.
It is worth noting that the adaption method currently
in`grDevices:::chromaticAdaptation` appears to be the "Von Kries" method, not
the "Bradford" method as documented in `?convertColor` and in the comments of
thesources. I base this on comparing the cone response domain matrices on the
aforementioned brucelindbloom.com page to the `Ma` matrix defined
in`grDevics:::chromaticAdaptation`.
Given that brucelindbloom.com appears to recommend "Bradford", that the sources
suggest that was the intent, that `chromaticAdaptation` is only used
by`convertColor` in the R sources, that `chromaticAdapation` is not exported,
and that that feature appears currently inaccessible via `convertColor`, it may
be worth using this opportunity to change the adaptation method to "Bradford".
A suggested patch follows. It is intended to minimize the required changes,
although doing so requires a double transposition. The transpositions could be
easily avoided, but it would require reformulating the calculations
in`chromaticAdaption`.
Best,
Brodie.
Index: src/library/grDevices/R/convertColor.R
===
--- src/library/grDevices/R/convertColor.R (revision 75298)
+++ src/library/grDevices/R/convertColor.R (working copy)
@@ -81,7 +81,7 @@
}
chromaticAdaptation <- function(xyz, from, to) {
- ## bradford scaling algorithm
+ ## Von Kries scaling algorithm
Ma <- matrix(c( 0.40024, -0.22630, 0.,
0.70760, 1.16532, 0.,
-0.08081, 0.04570, 0.91822), nrow = 3L, byrow = TRUE)
@@ -242,8 +242,8 @@
if (is.null(from.ref.white))
from.ref.white <- to.ref.white
- from.ref.white <- c2to3(white.points[, from.ref.white])
- to.ref.white <- c2to3(white.points[, to.ref.white])
+ from.ref.white.3 <- c2to3(white.points[, from.ref.white])
+ to.ref.white.3 <- c2to3(white.points[, to.ref.white])
if (is.null(nrow(color)))
color <- matrix(color, nrow = 1L)
@@ -262,19 +262,19 @@
rgb
}
- xyz <- apply(color, 1L, from$toXYZ, from.ref.white)
+ xyz <- apply(color, 1L, from$toXYZ, from.ref.white.3)
if (is.null(nrow(xyz)))
xyz <- matrix(xyz, nrow = 1L)
- if (!isTRUE(all.equal(from.ref.white, to.ref.white))) {
+ if (!isTRUE(all.equal(from.ref.white.3, to.ref.white.3))) {
mc <- match.call()
if (is.null(mc$from.ref.white) || is.null(mc$to.ref.white))
warning("color spaces use different reference whites")
- xyz <- chromaticAdaptation(xyz, from.ref.white, to.ref.white)
+ xyz <- t(chromaticAdaptation(t(xyz), from.ref.white, to.ref.white))
}
- rval <- apply(xyz, 2L, to$fromXYZ, to.ref.white)
+ rval <- apply(xyz, 2L, to$fromXYZ, to.ref.white.3)
if (inherits(to,"RGBcolorConverter"))
rval <- trim(rval)
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Poor documentation for "adj" and text()
Hello Core Team,
I sent this patch over a year ago. It looks like it was sent in
response to another user's complaint which echoed some of my own
observations about problems in the documentation for 'text'. Did
anyone have a chance to look it over? I'd like to get it out of my
queue.
Thanks,
Frederick
On Tue, Apr 11, 2017 at 12:01:05PM -0700, frede...@ofb.net wrote:
> Thanks Ulrich for sharing your experience.
>
> I'm attaching a patch which tries to address the issues you raised.
>
> I agree with you in principle, but I think it makes sense to leave
> some details under "Details". However, the descriptions in "Arguments"
> should give enough information that a user can get the function to do
> something predictable in at least one situation, and I feel this is
> not the case at present.
>
> I tried to fix the wording so that 'adj' and 'offset' are no longer
> confusing to new users (or to me, every time I forget what they mean).
>
> I also fixed the paragraph on rotated text; it is more correct now, at
> least for X11-cairo.
>
> I hope that someone in the Core Team can look this over and apply it.
>
> Thank you,
>
> Frederick
>
> On Tue, Apr 11, 2017 at 09:23:50AM +0200, Ulrich Windl wrote:
> > Hi!
> >
> > (I'd like to be able to access your bugzilla, BTW)
> > The documentation for parameter "adj" of text() in R 3.3.3 is hard to
> > understand (unless you know what it does already):
> >
> > "adj
> > one or two values in [0, 1] which specify the x (and optionally y)
> > adjustment of the labels. On most devices values outside that interval will
> > also work."
> >
> > What is the meaning of the values? I think the description ("adj allows
> > adjustment of the text with respect to (x, y). Values of 0, 0.5, and 1
> > specify left/bottom, middle and right/top alignment, respectively. The
> > default is for centered text, i.e., adj = c(0.5, NA). Accurate vertical
> > centering needs character metric information on individual characters which
> > is only available on some devices. Vertical alignment is done slightly
> > differently for character strings and for expressions: adj = c(0,0) means
> > to left-justify and to align on the baseline for strings but on the bottom
> > of the bounding box for expressions. This also affects vertical centering:
> > for strings the centering excludes any descenders whereas for expressions
> > it includes them. Using NA for strings centers them, including
> > descenders.") should be moved to the parameter.
> >
> > In general I'd suggest to describe the range, meaning and default of every
> > parameter where the parameter is listed. "Details" should only give an
> > overview of the functions.
> >
> > Likewise "offset": Will the direction be influenced by "pos"? The
> > description is quite silent on that.
> >
> > Documentation should be structured to help the user to find the facts
> > easily without having to read the whole page.
> >
> > Regards,
> > Ulrich Windl
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
> --- text.Rd 2016-11-27 18:33:26.541516325 -0800
> +++ new-text.Rd 2017-04-11 11:48:32.668926075 -0700
> @@ -26,16 +26,18 @@
> If \code{labels} is longer than \code{x} and
> \code{y}, the coordinates are recycled to the length of \code{labels}.}
>\item{adj}{one or two values in \eqn{[0, 1]} which specify the x
> -(and optionally y) adjustment of the labels. On most devices values
> -outside that interval will also work.}
> +(and optionally y) justification of the labels, with 0 for
> +left/bottom, 1 for right/top, and 0.5 for centered.
> +On most devices values
> +outside \eqn{[0, 1]} will also work. See below.}
>\item{pos}{a position specifier for the text. If specified this
> overrides any \code{adj} value given. Values of \code{1},
> \code{2}, \code{3} and \code{4}, respectively indicate
> positions below, to the left of, above and to the right of
> -the specified coordinates.}
> - \item{offset}{when \code{pos} is specified, this value gives the
> -offset of the label from the specified coordinate in fractions
> -of a character width.}
> +\code{(x, y)}.}
> + \item{offset}{when \code{pos} is specified, this value controls the
> +distance of the text label from \code{(x, y)}, in fractions of a
> +character width.}
>\item{vfont}{\code{NULL} for the current font family, or a character
> vector of length 2 for Hershey vector fonts. The first element of
> the vector selects a typeface and the second element selects a
> @@ -62,10 +64,11 @@
>mathematical notation is available such as sub- and superscripts,
>greek letters, fractions, etc.
>
> - \code{adj} allows \emph{adj}ustment of the text with respect to
> + \code{adj} allows \emph{adj}ustment of the text position with respect to
>\code{(x, y)}.
> - Values of
