notes/notes/ANS Theory.md

https://arxiv.org/abs/1311.2540

In standard numeral systems, different digits are treated as containing the same amount of information. A 7 stores the same amount of info as a 9, which stores the same amount of info as a 1.

This makes the amount of information a single digit stores uniform across all digits. However, that's far from the most efficient way to represent most datasets.

ANS theory is based around the idea that digits that occur more often can be stored in a way that requires less information, and digits that occur less often can be stored using more information.

Taking a look at the standard binary numeral system, there are two digits in the set (0 and 1). Given a natural number represented in binary, eg 1010, there are two different ways to add information to that number:

1. We can add a digit to the most significant position. As an example, adding a 1 to the above value would result in 11010. Doing this means that the added digit stores information about large ranges. In the provided example, this means that setting that digit changes the value by 16.
2. We can add a digit to the least significant position. As an example, adding a 1 to the above value would result in 10101. Changing the added digit will only change the resulting natural number by 1.

Given that x represents a natural number, and s is the digit we're adding. In a standard binary system, adding s to the least significant position means that in the new number x (before the addition) now represents the Nth appearance of an even (when s = 0 ), or odd (when s = 1). With ANS, the goal is is to make that asymmetrical, so that you can represent more common values with a denser representation.