On Tue, 8 Nov 2016 05:47 am, jlada...@itu.edu wrote: > On Saturday, November 5, 2016 at 6:39:52 PM UTC-7, Steve D'Aprano wrote: >> On Sun, 6 Nov 2016 09:17 am, Mr. Wrobel wrote: >> >> >> I don't have any experience with GPU processing. I expect that it will be >> useful for somethings, but for number-crushing and numeric work, I am >> concerned that GPUs rarely provide correctly rounded IEEE-754 maths. That >> means that they are accurate enough for games where a few visual glitches >> don't matter, but they risk being inaccurate for serious work. >> >> I fear that doing numeric work in GPUs will be returning to the 1970s, >> when every computer was incompatible with every other computer, and it >> was almost impossible to write cross-platform, correct, accurate numeric >> code. > > Hi Steve, > > You, Jason Swails, myself, and several others had a discussion about the > state of GPU arithmetic and IEEE-754 compliance just over a year ago.
I don't know why you think I was part of this discussion -- I made one comment early in the thread, and took part in none of the subsequent comments. If I had read any of the subsequent comments in the thread, I don't remember them. > https://groups.google.com/forum/#!msg/comp.lang.python/Gt_FzFlES8A/r_3dbW5XzfkJ;context-place=forum/comp.lang.python For those who dislike GoogleGroups, here's the official archive: https://mail.python.org/pipermail/python-list/2015-February/686683.html > It has been very important for the field of computational molecular > dynamics (and probably several other fields) to get floating-point > arithmetic working right on GPU architecture. I don't know anything about > other manufacturers of GPU's, but NVidia announced IEEE-754, > double-precision arithmetic for their GPU's in 2008, and it's been > included in the standard since CUDA 2.0. That's excellent news, and well-done to NVidia. But as far as I know, they're not the only manufacturer of GPUs, and they are the only ones who support IEEE 754. So this is *exactly* the situation I feared: incompatible GPUs with varying support for IEEE 754 making it difficult or impossible to write correct numeric code across GPU platforms. Perhaps it doesn't matter? Maybe people simply don't bother to use anything but Nvidia GPUs for numeric computation, and treat the other GPUs as toys only suitable for games. > If floating-point math wasn't working on GPU's, I suspect that a lot of > people in the scientific community would be complaining. I don't. These are scientists, not computational mathematics computer scientists. In the 1980s, the authors of the "Numeric Recipes in ..." books, William H Press et al, wrote a comment about the large number of scientific papers and simulations which should be invalidated due to poor numeric properties of the default pseudo-random number generators available at the time. I see no reason to think that the numeric programming sophistication of the average working scientist or Ph.D. student has improved since then. The average scientist cannot even be trusted to write an Excel spreadsheet without errors that invalidate their conclusion: https://www.washingtonpost.com/news/wonk/wp/2016/08/26/an-alarming-number-of-scientific-papers-contain-excel-errors/ let alone complex floating point numeric code. Sometimes those errors can change history: the best, some might say *only*, evidence for the austerity policies which have been destroying the economies in Europe for almost a decade now is simply a programming error. http://www.bloomberg.com/news/articles/2013-04-18/faq-reinhart-rogoff-and-the-excel-error-that-changed-history These are not new problems: dubious numeric computations have plagued scientists and engineers for decades, there is still a huge publication bias against negative results, most papers are written but not read, and even those which are read, most are wrong. http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 Especially in fast moving fields of science where there is money to be made, like medicine and genetics. There the problems are much, much worse. Bottom line: I'm very glad that Nvidia now support IEEE 754 maths, and that reduces my concerns: at least users of one common GPU can be expected to have correctly rounded results of basic arithmetic operations. -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
