arc/notes
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: arc:1052e71a6d76894dd189f142581ef8b0230977ec
Branches Tags
arc:main
...
compare: arc:bff7640d0a56bb98efaaaa31927846c2e5a92893
Branches Tags
arc:main

2 Commits
1052e71a6d ... bff7640d0a

Author SHA1 Message Date
arc
bff7640d0a vault backup: 2025-05-09 12:12:48 2025-05-09 12:12:48 -06:00
arc
27b25ff6dd vault backup: 2025-05-09 12:07:48 2025-05-09 12:07:48 -06:00
1 changed files with 7 additions and 2 deletions
Show Stats Expand all files Collapse all files

9
notes/ANS Theory.md
View File

@ -1,8 +1,8 @@
<https://arxiv.org/abs/1311.2540> <https://arxiv.org/abs/1311.2540>
In standard numeral systems, different digits are treated as containing the same amount of information. A 7 stores the same amount of info as a 9, which stores the same amount of info as a 1. In standard numeral systems, different digits are treated as containing the same amount of information. A 7 is stored using the same amount of info as a 9, which is stored using the same amount of info as a 1, that is, 1 digit.
This makes the amount of information a single digit stores *uniform* across all digits. However, that's far from the most efficient way to represent most datasets. This makes the amount of information a single digit stores *uniform* across all digits. However, that's far from the most efficient way to represent most datasets, because real world data rarely follows a uniform distribution.
ANS theory is based around the idea that digits that occur more often can be stored in a way that requires less information, and digits that occur less often can be stored using more information. ANS theory is based around the idea that digits that occur more often can be stored in a way that requires less information, and digits that occur less often can be stored using more information.
@ -12,4 +12,9 @@ Taking a look at the standard binary numeral system, there are two digits in the
Given that $x$ represents a natural number, and $s$ is the digit we're adding. In a standard binary system, adding $s$ to the least significant position means that in the new number $x$ (before the addition) now represents the Nth appearance of an even (when $s = 0$ ), or odd (when $s = 1$). With ANS, the goal is is to make that asymmetrical, so that you can represent more common values with a denser representation. Given that $x$ represents a natural number, and $s$ is the digit we're adding. In a standard binary system, adding $s$ to the least significant position means that in the new number $x$ (before the addition) now represents the Nth appearance of an even (when $s = 0$ ), or odd (when $s = 1$). With ANS, the goal is is to make that asymmetrical, so that you can represent more common values with a denser representation.
# Arithmetic Coding
Arithmetic coding works by taking a stream of data, and converting it into an infinitely precise number between $0.00$, and $1.00$. This is based off of the idea that the sum of the probability of all events happening will always amount to $100\%$.
For example, the probability of a coin flip resulting in tails is 50%, and the probability of a coin flip resulting in heads is 50%. The probability of a coin flip resulting in heads *or* tails is %100.
If we wanted to keep track of the result of a series of coin flips, this could be done by subdividing a range. If the coin flip is between $0$ and $0.5$, then we know that the first flip must