Digit limits

[Guest firebird68]

PRODUCT

Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007

(2007-02-22)

 

QUESTIONS OR COMMENTS

Message: When multiplying 111,111,111 by itself, the Excel result is:

12345678987654300. The real answer when multiplying by hand is :

12345678987654321. Why does Excel round off the last two digits to zero?

Excel appears to rounds off to 15 digits. Why not 16 or more?

--

The Firebird Man

[Guest VanguardLH]

Re: Digit limits

 

firebird68 wrote:

> PRODUCT

> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007

> (2007-02-22)

>

> QUESTIONS OR COMMENTS

> Message: When multiplying 111,111,111 by itself, the Excel result is:

> 12345678987654300. The real answer when multiplying by hand is :

> 12345678987654321. Why does Excel round off the last two digits to zero?

> Excel appears to rounds off to 15 digits. Why not 16 or more?

 

"excel" is not in the name of this newsgroup.

 

http://www.j-walk.com/SS/excel/usertips/tip032.htm

http://support.microsoft.com/kb/65903

http://excel.tips.net/Pages/T001983_Thoughts_and_Ideas_on_Significant_Digits_in_Excel.html

http://precisioncalc.com/What_is_xlPrecision.html

[Guest Ken Blake, MVP]

Re: Digit limits

 

On Mon, 13 Oct 2008 04:49:01 -0700, firebird68

<firebird68@discussions.microsoft.com> wrote:

> PRODUCT

> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007

> (2007-02-22)

>

> QUESTIONS OR COMMENTS

> Message: When multiplying 111,111,111 by itself, the Excel result is:

> 12345678987654300. The real answer when multiplying by hand is :

> 12345678987654321. Why does Excel round off the last two digits to zero?

> Excel appears to rounds off to 15 digits. Why not 16 or more?

 

 

You're asking an Excel question in a Windows newsgroup. You are far

more likely to get the help you are looking for if you would ask in an

Excel newsgroup; that's where the Excel experts hang out.

 

--

Ken Blake, Microsoft MVP - Windows Desktop Experience

Please Reply to the Newsgroup

[Guest AlmostBob]

Re: Digit limits

 

http://www.smokeylake.com/excel/Data.htm

explains how excel stores numbers, and the precision lost after the 15th

(decimal) digit

 

--

Adaware http://www.lavasoft.de

spybot http://www.safer-networking.org

AVG free antivirus http://www.grisoft.com

Etrust/Vet/CA.online Antivirus scan

http://www3.ca.com/securityadvisor/virusinfo/scan.aspx

Panda online AntiVirus scan http://www.pandasoftware.com/ActiveScan/

Catalog of removal tools (1)

http://www.pandasoftware.com/download/utilities/

Catalog of removal tools (2)

http://www3.ca.com/securityadvisor/newsinfo/collateral.aspx?CID=40387

Blocking Unwanted Parasites with a Hosts file

http://mvps.org/winhelp2002/hosts.htm

links provided as a courtesy, read all instructions on the pages before use

 

Grateful thanks to the authors and webmasters

_

 

"firebird68" <firebird68@discussions.microsoft.com> wrote in message

news:44915B4D-2BC9-4CD3-AC45-F23B904E37F2@microsoft.com...

> PRODUCT

> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007

> (2007-02-22)

>

> QUESTIONS OR COMMENTS

> Message: When multiplying 111,111,111 by itself, the Excel result is:

> 12345678987654300. The real answer when multiplying by hand is :

> 12345678987654321. Why does Excel round off the last two digits to zero?

> Excel appears to rounds off to 15 digits. Why not 16 or more?

> --

> The Firebird Man

[Guest RobertVA]

Re: Digit limits

 

firebird68 wrote:

> PRODUCT

> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007

> (2007-02-22)

>

> QUESTIONS OR COMMENTS

> Message: When multiplying 111,111,111 by itself, the Excel result is:

> 12345678987654300. The real answer when multiplying by hand is :

> 12345678987654321. Why does Excel round off the last two digits to zero?

> Excel appears to rounds off to 15 digits. Why not 16 or more?

 

In most systems the memory available to store a quantity is limited.

When asked to represent large numbers many software applications store

the number as in a manner resembling scientific notation (like

1.23456789876543 X 10^14 BUT there's a significant possibility the

application uses a power of two instead of a power of ten). Since the

space to store the digits is limited, the number gets rounded off. If

the programmer thinks the matter is important enough the problem can be

eliminated, but that would probably at the expense of slower operation

and/or greatly enlarged storage requirements.

 

Note that for similar reasons there are numbers that cannot be precisely

stated as fractions. These irrational numbers include pi and the square

root of many prime numbers like two and three.

[Guest Larry(LJL269)]

Re: Digit limits

 

where u really get in trouble is with small #'s that approach 0 (like

1/n as n approaches infinity) which r eventually rounded to 0 & 0's

propogate thru the model or r multiplied by a very large # to get 0.

 

Solving partial differential eqs numerically which leads to more

deadly A-bomb is 1 example I know of. Big concern in Numerical

Analysis.

 

Hope that helps. Larry

 

On Mon, 13 Oct 2008 14:06:36 -0400, RobertVA

<robert_c72athotmail@invalid.com> wrote:

>firebird68 wrote:

>> PRODUCT

>> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007

>> (2007-02-22)

>>

>> QUESTIONS OR COMMENTS

>> Message: When multiplying 111,111,111 by itself, the Excel result is:

>> 12345678987654300. The real answer when multiplying by hand is :

>> 12345678987654321. Why does Excel round off the last two digits to zero?

>> Excel appears to rounds off to 15 digits. Why not 16 or more?

>

>In most systems the memory available to store a quantity is limited.

>When asked to represent large numbers many software applications store

>the number as in a manner resembling scientific notation (like

>1.23456789876543 X 10^14 BUT there's a significant possibility the

>application uses a power of two instead of a power of ten). Since the

>space to store the digits is limited, the number gets rounded off. If

>the programmer thinks the matter is important enough the problem can be

>eliminated, but that would probably at the expense of slower operation

>and/or greatly enlarged storage requirements.

>

>Note that for similar reasons there are numbers that cannot be precisely

>stated as fractions. These irrational numbers include pi and the square

>root of many prime numbers like two and three.

 

----------------------------------------------------------------------

 

A working unsecure OS is infinitely better than non-working secure OS.

Just spent 1 week cleaning up the mess WUpdate made preventing

hypothetical security problems. http://microscum.com/comsense/

[Guest HeyBub]

Re: Digit limits

 

firebird68 wrote:

> PRODUCT

> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007

> (2007-02-22)

>

> QUESTIONS OR COMMENTS

> Message: When multiplying 111,111,111 by itself, the Excel result is:

> 12345678987654300. The real answer when multiplying by hand is :

> 12345678987654321. Why does Excel round off the last two digits to

> zero? Excel appears to rounds off to 15 digits. Why not 16 or more?

 

Excel uses IEEE double-precision floating point numbers. These numbers are

limited to about 15 significant digits.

 

As for 16 or more, a limit had to be set somewhere. If the developers had

coded extended precision arithmetic to, oh, say 75 digits of precision, sure

enough somebody would come along and say "why not 76?"