Email or username:

Password:

Forgot your password?
Julia Evans

"How Integers and Floats Work" is coming out later this week! Here's the about page:

title: computers do math weird

Weird things happen when your computer does math.

computer (thinking): 0.1 + 0.2 = 0.30000000000000004
computer (thinking): 4294967295 + 1 = 0
person (unhappy): uh, that's not what they taught me in math class...

The reason it gets so weird is that your computer has to cram each number into a limited number of bits (8, 16, 32, or 64 bits).

When your computer does math, it's running CPU instructions. And there are only 2 kinds of CPU math instructions: those that work on integers, and those that work on floating point numbers.

person (talking): let's go learn how your computer handles integers and floating point numbers!
17 comments
Julia Evans

I'm really happy with this “the way computers do math is weird" framing we arrived at -- I think it gets at why floating point weirdnesses and integer overflow can feel so upsetting (“it's MATH! it's supposed to behave how I EXPECT!”)

Michael W Lucas¹ :flan_mail:

@b0rk there's nothing like the joy of developing a great mental framework for thinking about a complex thing.

benjohn

@b0rk I suppose it is weird, but I kind of like integer overflow. I like how it’s a big circle that goes round and round for ever. I remember it being useful sometimes for computer graphics in assembly. But it is mostly useless when we usually want numbers that do normal things!

Julia Evans

@benjohn yeah absolutely, it's fun to see something that I spent a lot of time learning about in math class ("the numbers modulo n”) have such a mainstream real-life application

Ayke van Laethem

@benjohn yes indeed, with some clever tricks it can be quite useful in graphics! This is also what FastLED does:
github.com/FastLED/FastLED/wik

I like it when data is defined in a way that it naturally leads to very efficient code.

Ben

@b0rk
That was my favourite part of my numerical analysis and later discrete functions courses—the aspect of "so the rules we normally expect to apply _don't_, and here's how we can cope with that" helped understand so much about why things go wrong.

marcorobotics

@b0rk I feel very, very, very proud that nothing about those examples feels weird to me anymore 😂 🤣

Jeremy Kahn

@b0rk

this header inspired some linguistic rumination about how you are able to drop small words ("[the way] computers do math [is] weird") but couldn't sacrifice "do", despite being the smallest of small words

because one has to do some extra work to make "math" work out as a verb in "[computers NN] [math VB] [weird ADV]" and instead it reads as three substantives "[computers NN] [math NN] [weird ADJ]", which reads like a list of tags #computers #math #weird

Jeremy Kahn

@b0rk

and also I love your helper cards; even when I think "oh I know that one" I always learn something new when I read them.

They are a gift.

Ridley @ WATCH LYCORECO

@b0rk some CPUs do support binary-coded decimal, or even decimal floating-point! they're very uncommon in modern consumer systems, though

Emelia/Emi

@rcombs @b0rk HP's oddball "Saturn" core says hi! It's *really* weird, being a 4-bit processor with mostly 64-bit registers. It's what ran their older graphing calculators, and a few of the newer ones were just a Saturn emulator on a small ARM SoC.

FCLC

@b0rk Where will it be published/available? Really loved all the posts along the way, would be interested in picking up a copy 😊

JR Buckley 🏳️‍🌈⚛️

@b0rk
🤣
And I got in trouble for sneaking a SLIDE RULE into class.

Go Up