vault backup: 2024-08-01 16:13:11

notes/ANS Theory.md

@@ -0,0 +1,13 @@
+<https://arxiv.org/abs/1311.2540>
+
+In standard numeral systems, different digits are treated as containing the same amount of information. A 7 stores the same amount of info as a 9, which stores the same amount of info as a 1. 
+
+This makes the amount of information a single digit stores *uniform* across all digits. However, that's far from the most efficient way to represent most datasets. 
+
+ANS theory is based around the idea that digits that occur more often can be stored in a way that requires less information, and digits that occur less often can be stored using more information.
+
+Taking a look at the standard binary numeral system, there are two digits in the set (0 and 1). Given a natural number represented in binary, eg `1010`, there are two different ways to *add information to that number*:
+1. We can add a digit to the most significant position. As an example, adding a `1` to the above value would result in `11010`. Doing this means that the added digit stores information about *large ranges*. In the provided example, this means that setting that digit changes the value by 16.
+2. We can add a digit to the least significant position. As an example, adding a `1` to the above value would result in `10101`. Changing the added digit will only change the resulting natural number by 1.
+
+Given that $x$ represents a natural number, and $s$ is the digit we're adding. In a standard binary system, adding $s$ to the least significant position means that in the new number $x$ (before the addition) now represents the Nth appearance of an even (when $s = 0$ ), or odd (when $s = 1$).