arc/notes
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: arc:1052e71a6d76894dd189f142581ef8b0230977ec
Branches Tags
arc:main
...
compare: arc:bff7640d0a56bb98efaaaa31927846c2e5a92893
Branches Tags
arc:main

2 Commits
1052e71a6d ... bff7640d0a

Author SHA1 Message Date
arc
bff7640d0a vault backup: 2025-05-09 12:12:48 2025-05-09 12:12:48 -06:00
arc
27b25ff6dd vault backup: 2025-05-09 12:07:48 2025-05-09 12:07:48 -06:00
1 changed files with 7 additions and 2 deletions
Show Stats Expand all files Collapse all files

9
notes/ANS Theory.md
View File

@ -1,8 +1,8 @@
<https://arxiv.org/abs/1311.2540>
In standard numeral systems, different digits are treated as containing the same amount of information. A 7 stores the same amount of info as a 9, which stores the same amount of info as a 1.
In standard numeral systems, different digits are treated as containing the same amount of information. A 7 is stored using the same amount of info as a 9, which is stored using the same amount of info as a 1, that is, 1 digit.
This makes the amount of information a single digit stores *uniform* across all digits. However, that's far from the most efficient way to represent most datasets.
This makes the amount of information a single digit stores *uniform* across all digits. However, that's far from the most efficient way to represent most datasets, because real world data rarely follows a uniform distribution.
ANS theory is based around the idea that digits that occur more often can be stored in a way that requires less information, and digits that occur less often can be stored using more information.
@ -12,4 +12,9 @@ Taking a look at the standard binary numeral system, there are two digits in the
Given that $x$ represents a natural number, and $s$ is the digit we're adding. In a standard binary system, adding $s$ to the least significant position means that in the new number $x$ (before the addition) now represents the Nth appearance of an even (when $s = 0$ ), or odd (when $s = 1$). With ANS, the goal is is to make that asymmetrical, so that you can represent more common values with a denser representation.
# Arithmetic Coding
Arithmetic coding works by taking a stream of data, and converting it into an infinitely precise number between $0.00$, and $1.00$. This is based off of the idea that the sum of the probability of all events happening will always amount to $100\%$.
For example, the probability of a coin flip resulting in tails is 50%, and the probability of a coin flip resulting in heads is 50%. The probability of a coin flip resulting in heads *or* tails is %100.
If we wanted to keep track of the result of a series of coin flips, this could be done by subdividing a range. If the coin flip is between $0$ and $0.5$, then we know that the first flip must