Guest firebird68 Posted October 13, 2008 Posted October 13, 2008 PRODUCT Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007 (2007-02-22) QUESTIONS OR COMMENTS Message: When multiplying 111,111,111 by itself, the Excel result is: 12345678987654300. The real answer when multiplying by hand is : 12345678987654321. Why does Excel round off the last two digits to zero? Excel appears to rounds off to 15 digits. Why not 16 or more? -- The Firebird Man
Guest VanguardLH Posted October 13, 2008 Posted October 13, 2008 Re: Digit limits firebird68 wrote: > PRODUCT > Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007 > (2007-02-22) > > QUESTIONS OR COMMENTS > Message: When multiplying 111,111,111 by itself, the Excel result is: > 12345678987654300. The real answer when multiplying by hand is : > 12345678987654321. Why does Excel round off the last two digits to zero? > Excel appears to rounds off to 15 digits. Why not 16 or more? "excel" is not in the name of this newsgroup. http://www.j-walk.com/SS/excel/usertips/tip032.htm http://support.microsoft.com/kb/65903 http://excel.tips.net/Pages/T001983_Thoughts_and_Ideas_on_Significant_Digits_in_Excel.html http://precisioncalc.com/What_is_xlPrecision.html
Guest Ken Blake, MVP Posted October 13, 2008 Posted October 13, 2008 Re: Digit limits On Mon, 13 Oct 2008 04:49:01 -0700, firebird68 <firebird68@discussions.microsoft.com> wrote: > PRODUCT > Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007 > (2007-02-22) > > QUESTIONS OR COMMENTS > Message: When multiplying 111,111,111 by itself, the Excel result is: > 12345678987654300. The real answer when multiplying by hand is : > 12345678987654321. Why does Excel round off the last two digits to zero? > Excel appears to rounds off to 15 digits. Why not 16 or more? You're asking an Excel question in a Windows newsgroup. You are far more likely to get the help you are looking for if you would ask in an Excel newsgroup; that's where the Excel experts hang out. -- Ken Blake, Microsoft MVP - Windows Desktop Experience Please Reply to the Newsgroup
Guest AlmostBob Posted October 13, 2008 Posted October 13, 2008 Re: Digit limits http://www.smokeylake.com/excel/Data.htm explains how excel stores numbers, and the precision lost after the 15th (decimal) digit -- Adaware http://www.lavasoft.de spybot http://www.safer-networking.org AVG free antivirus http://www.grisoft.com Etrust/Vet/CA.online Antivirus scan http://www3.ca.com/securityadvisor/virusinfo/scan.aspx Panda online AntiVirus scan http://www.pandasoftware.com/ActiveScan/ Catalog of removal tools (1) http://www.pandasoftware.com/download/utilities/ Catalog of removal tools (2) http://www3.ca.com/securityadvisor/newsinfo/collateral.aspx?CID=40387 Blocking Unwanted Parasites with a Hosts file http://mvps.org/winhelp2002/hosts.htm links provided as a courtesy, read all instructions on the pages before use Grateful thanks to the authors and webmasters _ "firebird68" <firebird68@discussions.microsoft.com> wrote in message news:44915B4D-2BC9-4CD3-AC45-F23B904E37F2@microsoft.com... > PRODUCT > Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007 > (2007-02-22) > > QUESTIONS OR COMMENTS > Message: When multiplying 111,111,111 by itself, the Excel result is: > 12345678987654300. The real answer when multiplying by hand is : > 12345678987654321. Why does Excel round off the last two digits to zero? > Excel appears to rounds off to 15 digits. Why not 16 or more? > -- > The Firebird Man
Guest RobertVA Posted October 13, 2008 Posted October 13, 2008 Re: Digit limits firebird68 wrote: > PRODUCT > Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007 > (2007-02-22) > > QUESTIONS OR COMMENTS > Message: When multiplying 111,111,111 by itself, the Excel result is: > 12345678987654300. The real answer when multiplying by hand is : > 12345678987654321. Why does Excel round off the last two digits to zero? > Excel appears to rounds off to 15 digits. Why not 16 or more? In most systems the memory available to store a quantity is limited. When asked to represent large numbers many software applications store the number as in a manner resembling scientific notation (like 1.23456789876543 X 10^14 BUT there's a significant possibility the application uses a power of two instead of a power of ten). Since the space to store the digits is limited, the number gets rounded off. If the programmer thinks the matter is important enough the problem can be eliminated, but that would probably at the expense of slower operation and/or greatly enlarged storage requirements. Note that for similar reasons there are numbers that cannot be precisely stated as fractions. These irrational numbers include pi and the square root of many prime numbers like two and three.
Guest Larry(LJL269) Posted October 13, 2008 Posted October 13, 2008 Re: Digit limits where u really get in trouble is with small #'s that approach 0 (like 1/n as n approaches infinity) which r eventually rounded to 0 & 0's propogate thru the model or r multiplied by a very large # to get 0. Solving partial differential eqs numerically which leads to more deadly A-bomb is 1 example I know of. Big concern in Numerical Analysis. Hope that helps. Larry On Mon, 13 Oct 2008 14:06:36 -0400, RobertVA <robert_c72athotmail@invalid.com> wrote: >firebird68 wrote: >> PRODUCT >> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007 >> (2007-02-22) >> >> QUESTIONS OR COMMENTS >> Message: When multiplying 111,111,111 by itself, the Excel result is: >> 12345678987654300. The real answer when multiplying by hand is : >> 12345678987654321. Why does Excel round off the last two digits to zero? >> Excel appears to rounds off to 15 digits. Why not 16 or more? > >In most systems the memory available to store a quantity is limited. >When asked to represent large numbers many software applications store >the number as in a manner resembling scientific notation (like >1.23456789876543 X 10^14 BUT there's a significant possibility the >application uses a power of two instead of a power of ten). Since the >space to store the digits is limited, the number gets rounded off. If >the programmer thinks the matter is important enough the problem can be >eliminated, but that would probably at the expense of slower operation >and/or greatly enlarged storage requirements. > >Note that for similar reasons there are numbers that cannot be precisely >stated as fractions. These irrational numbers include pi and the square >root of many prime numbers like two and three. ---------------------------------------------------------------------- A working unsecure OS is infinitely better than non-working secure OS. Just spent 1 week cleaning up the mess WUpdate made preventing hypothetical security problems. http://microscum.com/comsense/
Guest HeyBub Posted October 13, 2008 Posted October 13, 2008 Re: Digit limits firebird68 wrote: > PRODUCT > Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007 > (2007-02-22) > > QUESTIONS OR COMMENTS > Message: When multiplying 111,111,111 by itself, the Excel result is: > 12345678987654300. The real answer when multiplying by hand is : > 12345678987654321. Why does Excel round off the last two digits to > zero? Excel appears to rounds off to 15 digits. Why not 16 or more? Excel uses IEEE double-precision floating point numbers. These numbers are limited to about 15 significant digits. As for 16 or more, a limit had to be set somewhere. If the developers had coded extended precision arithmetic to, oh, say 75 digits of precision, sure enough somebody would come along and say "why not 76?"
Recommended Posts