A Computer Science Theory Class Was Actually Useful At My Job

A Computer Science Theory Class Was Actually Useful At My Job

I got a Computer Science & Engineering degree a few years ago. Most of the things I learned in school I have never used in any job. I never need to draw a finite state machine, or know how the path a hard drive takes to get at non-continuous data, or understand the caching mechanism of the L1 cache is, or Von Neumann architecture. Most of the skills I have needed in my job, I have learned on my own, or I have learned because the problem came up at work, and I had to learn the skill extremely fast. However, this week, I ran into a problem that would take me a long time to figure out the problem if I had not had my computer theory classes.

An employee was trying to get a sum total of several record together to equal 0, but even though the math was right it was coming up with 7.13249493E-14. This looks like a big number, but it is really just the computer’s way of writing in scientific notation  7.13249493 x 10-14, which is really 0.0000000000000713249493. This is a very small number, when I saw this I knew what it was.

Floating Point Errors

For computers to do math, all numbers have to be converted to base-2 from our base-10 number system. So 4+7=11, for a computer is 0100+0111=1011. Converting whole numbers is easy enough, but to convert decimal numbers is a little harder. Most programming languages represent decimals as floating point when they are turned into base-2. This causes a problem though. Just like if you were given a piece of paper and a pen, and asked to write out π, you could not right out a fully accurate number, but you could get really close(for most applications, getting within the millionth decimal place is really close in my mind). Certain decimal numbers can not be written out completely accurately in floating point base-2. Oddly enough 0.1 is one of these numbers. So the data set just happened to cause this error, and the program was not rounding, but showing the extremely small deviation caused by numbers like 0.1 being slightly off.

Errors not Because of the Code

It is extremely rare that a bug is not caused by some stupid piece of code that I wrote. Having a bug crop up because of a limitation of the system is crazy rare, and I had to bask in it for a little while. Obviously the easiest way to fix this bug is to just round, but it was interesting to see this, since I do not do crazy complex math programs that come into this problem way more often.

No Comments

Leave a Comment

Please be polite. We appreciate that.
Your email address will not be published and required fields are marked