diff --git a/notes/ANS Theory.md b/notes/ANS Theory.md index 18b404e..84ebb58 100644 --- a/notes/ANS Theory.md +++ b/notes/ANS Theory.md @@ -1,8 +1,8 @@ -In standard numeral systems, different digits are treated as containing the same amount of information. A 7 stores the same amount of info as a 9, which stores the same amount of info as a 1. +In standard numeral systems, different digits are treated as containing the same amount of information. A 7 is stored using the same amount of info as a 9, which is stored using the same amount of info as a 1, that is, 1 digit. -This makes the amount of information a single digit stores *uniform* across all digits. However, that's far from the most efficient way to represent most datasets. +This makes the amount of information a single digit stores *uniform* across all digits. However, that's far from the most efficient way to represent most datasets, because real world data rarely follows a uniform distribution. ANS theory is based around the idea that digits that occur more often can be stored in a way that requires less information, and digits that occur less often can be stored using more information. @@ -12,4 +12,5 @@ Taking a look at the standard binary numeral system, there are two digits in the Given that $x$ represents a natural number, and $s$ is the digit we're adding. In a standard binary system, adding $s$ to the least significant position means that in the new number $x$ (before the addition) now represents the Nth appearance of an even (when $s = 0$ ), or odd (when $s = 1$). With ANS, the goal is is to make that asymmetrical, so that you can represent more common values with a denser representation. - +# Arithmetic Coding +Arithmetic coding works