Scribd
Upload
68 views

Precision (Computer Science)

Precision in computer science refers to the level of detail in which a numerical quantity is expressed, usually measured in bits. A higher precision allows for less rounding error but uses more storage. Common sources of precision loss include floating point arithmetic and integer representation.

Uploaded by

Alex Chen
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
68 views

Precision (Computer Science)

Precision in computer science refers to the level of detail in which a numerical quantity is expressed, usually measured in bits. A higher precision allows for less rounding error but uses more storage. Common sources of precision loss include floating point arithmetic and integer representation.

Uploaded by

Alex Chen
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 1

Precision (computer science) - Wikipedia, the free encyclopedia

1 of 1

https://en.wikipedia.org/wiki/Precision_(computer_science)

From Wikipedia, the free encyclopedia

In computer science, the precision of a numerical quantity is a measure of the detail in which the
quantity is expressed. This is usually measured in bits, but sometimes in decimal digits. It is related to
precision in mathematics, which describes the number of digits that are used to express a value.

Precision is often the source of rounding errors in computation. The number of bits used to store a
number will often cause some loss of accuracy. An example would be to store "sin(0.1)" in IEEE
single precision floating point standard. The error is then often magnified as subsequent
computations are made using the data (although it can also be reduced).

Arbitrary-precision arithmetic
Extended precision
IEEE754 (IEEE floating point standard)
Integer (computer science)
Significant figures
Truncation
Retrieved from "https://en.wikipedia.org/w/index.php?title=Precision_(computer_science)&
oldid=708962480"
Categories: Computer data
This page was last modified on 8 March 2016, at 12:30.
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms
may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia is a
registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.

2016/4/26 12:18

You might also like