notes/notes/ANS Theory.md

12 lines
1.3 KiB
Markdown
Raw Normal View History

2024-08-01 20:57:36 +00:00
<https://arxiv.org/abs/1311.2540>
2024-08-01 21:31:21 +00:00
2024-08-01 21:36:21 +00:00
In standard numeral systems, different digits are treated as containing the same amount of information. A 7 stores the same amount of info as a 9, which stores the same amount of info as a 1.
This makes the amount of information a single digit stores *uniform* across all digits. However, that's far from the most efficient way to represent most datasets.
2024-08-01 21:46:21 +00:00
ANS theory is based around the idea that digits that occur more often can be stored in a way that requires less information, and digits that occur less often can be stored using more information.
2024-08-01 21:51:21 +00:00
Taking a look at the standard binary numeral system, there are two digits in the set (0 and 1). Given a natural number represented in binary, eg `1010`, there are two different ways to *add information to that number*:
1. We can add a digit to the most significant position. As an example, adding a `1` to the above value would result in `11010`. Doing this means that the added digit stores information about *large ranges*. In the provided example, this means that setting that digit changes the value by 16.
2024-08-01 21:57:07 +00:00
2. We can add a digit to the least significant position. As an example, adding a `1` to the above value would result in `10101`. Changing the added digit will only change the resulting natural number by 1.