Virtualization
/
brotli
mirror of https://github.com/google/brotli
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Generate new .txt version of the spec. 

Based on the changes in the .nroff source in PR #229
This commit is contained in:
Zoltan Szabadka 2015-10-20 12:27:09 +02:00
parent b7a613fd51
commit 676bc91cc4
1 changed files with 100 additions and 100 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

200
docs/draft-alakuijala-brotli-07.txt
View File

@ -93,7 +93,7 @@ Table of Contents
12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34
13. Informative References . . . . . . . . . . . . . . . . . . . 35 13. Informative References . . . . . . . . . . . . . . . . . . . 35
14. Source code . . . . . . . . . . . . . . . . . . . . . . . . . 35 14. Source code . . . . . . . . . . . . . . . . . . . . . . . . . 35
15. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 35 15. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 35
Appendix A. Static dictionary data . . . . . . . . . . . . . . . 35 Appendix A. Static dictionary data . . . . . . . . . . . . . . . 35
Appendix B. List of word transformations . . . . . . . . . . . . 116 Appendix B. List of word transformations . . . . . . . . . . . . 116
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 119 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 119
@ -549,11 +549,11 @@ Internet-Draft Brotli October 2015
significant bit. The code lengths are initially in tree[I].Len; the significant bit. The code lengths are initially in tree[I].Len; the
codes are produced in tree[I].Code. codes are produced in tree[I].Code.
1) Count the number of codes for each code length. Let 1) Count the number of codes for each code length. Let
bl_count[N] be the number of codes of length N, N >= 1. bl_count[N] be the number of codes of length N, N >= 1.
2) Find the numerical value of the smallest code for each 2) Find the numerical value of the smallest code for each
code length: code length:
@ -564,26 +564,26 @@ Alakuijala & Szabadka Expires April 6, 2016 [Page 10]
Internet-Draft Brotli October 2015 Internet-Draft Brotli October 2015
code = 0; code = 0;
bl_count[0] = 0; bl_count[0] = 0;
for (bits = 1; bits <= MAX_BITS; bits++) { for (bits = 1; bits <= MAX_BITS; bits++) {
code = (code + bl_count[bits-1]) << 1; code = (code + bl_count[bits-1]) << 1;
next_code[bits] = code; next_code[bits] = code;
} }
3) Assign numerical values to all codes, using consecutive 3) Assign numerical values to all codes, using consecutive
values for all codes of the same length with the base values for all codes of the same length with the base
values determined at step 2. Codes that are never used values determined at step 2. Codes that are never used
(which have a bit length of zero) must not be assigned a (which have a bit length of zero) must not be assigned a
value. value.
for (n = 0; n <= max_code; n++) { for (n = 0; n <= max_code; n++) {
len = tree[n].Len; len = tree[n].Len;
if (len != 0) { if (len != 0) {
tree[n].Code = next_code[len]; tree[n].Code = next_code[len];
next_code[len]++; next_code[len]++;
} }
} }
Example: Example:
@ -1161,10 +1161,10 @@ Internet-Draft Brotli October 2015
There are four methods, called context modes, to compute the Context There are four methods, called context modes, to compute the Context
ID: ID:
* MSB6, where the Context ID is the value of six most
significant bits of p1,
* LSB6, where the Context ID is the value of six least * LSB6, where the Context ID is the value of six least
significant bits of p1, significant bits of p1,
* MSB6, where the Context ID is the value of six most
significant bits of p1,
* UTF8, where the Context ID is a complex function of p1, p2, * UTF8, where the Context ID is a complex function of p1, p2,
optimized for text compression, and optimized for text compression, and
* Signed, where Context ID is a complex function of p1, p2, * Signed, where Context ID is a complex function of p1, p2,
@ -1353,20 +1353,20 @@ Internet-Draft Brotli October 2015
language function: language function:
void InverseMoveToFrontTransform(uint8_t* v, int v_len) { void InverseMoveToFrontTransform(uint8_t* v, int v_len) {
uint8_t mtf[256]; uint8_t mtf[256];
int i; int i;
for (i = 0; i < 256; ++i) { for (i = 0; i < 256; ++i) {
mtf[i] = (uint8_t)i; mtf[i] = (uint8_t)i;
} }
for (i = 0; i < v_len; ++i) { for (i = 0; i < v_len; ++i) {
uint8_t index = v[i]; uint8_t index = v[i];
uint8_t value = mtf[index]; uint8_t value = mtf[index];
v[i] = value; v[i] = value;
for (; index; --index) { for (; index; --index) {
mtf[index] = mtf[index - 1]; mtf[index] = mtf[index - 1];
} }
mtf[0] = value; mtf[0] = value;
} }
} }
8. Static dictionary 8. Static dictionary
@ -1434,16 +1434,13 @@ Internet-Draft Brotli October 2015
where the _i subscript denotes the transform_id above. Each T_i is where the _i subscript denotes the transform_id above. Each T_i is
one of the following 21 elementary transforms: one of the following 21 elementary transforms:
Identity, OmitLast1, ..., OmitLast9, UppercaseFirst, UppercaseAll, Identity, UppercaseFirst, UppercaseAll,
OmitFirst1, ..., OmitFirst9 OmitFirst1, ..., OmitFirst9, OmitLast1, ..., OmitLast9
The form of these elementary transforms are as follows: The form of these elementary transforms are as follows:
Identity(word) = word Identity(word) = word
OmitLastk(word) = the first (length(word) - k) bytes of word, or
empty string if length(word) < k
UppercaseFirst(word) = first UTF-8 character of word upper-cased UppercaseFirst(word) = first UTF-8 character of word upper-cased
UppercaseAll(word) = all UTF-8 characters of word upper-cased UppercaseAll(word) = all UTF-8 characters of word upper-cased
@ -1451,6 +1448,9 @@ Internet-Draft Brotli October 2015
OmitFirstk(word) = the last (length(word) - k) bytes of word, or OmitFirstk(word) = the last (length(word) - k) bytes of word, or
empty string if length(word) < k empty string if length(word) < k
OmitLastk(word) = the first (length(word) - k) bytes of word, or
empty string if length(word) < k
For the purposes of UppercaseAll, word is parsed into UTF-8 For the purposes of UppercaseAll, word is parsed into UTF-8
@ -1544,25 +1544,25 @@ Internet-Draft Brotli October 2015
length of the meta-block, and a bit signaling if the meta-block is length of the meta-block, and a bit signaling if the meta-block is
the last one. The format of the meta-block header is the following: the last one. The format of the meta-block header is the following:
1 bit: ISLAST, set to 1 if this is the last meta-block 1 bit: ISLAST, set to 1 if this is the last meta-block
1 bit: ISLASTEMPTY, if set to 1, the meta-block is empty; 1 bit: ISLASTEMPTY, if set to 1, the meta-block is empty;
this field is only present if ISLAST bit is set -- if this field is only present if ISLAST bit is set -- if
it is 1, then the meta-block and the brotli stream ends it is 1, then the meta-block and the brotli stream ends
at that bit, with any remaining bits in the last byte at that bit, with any remaining bits in the last byte
of the compressed stream filled with zeros (if the of the compressed stream filled with zeros (if the
fill bits are not zero, then the stream should be fill bits are not zero, then the stream should be
rejected as invalid) rejected as invalid)
2 bits: MNIBBLES, # of nibbles to represent the uncompressed 2 bits: MNIBBLES, # of nibbles to represent the uncompressed
length, encoded as follows: if set to 3, MNIBBLES is 0, length, encoded as follows: if set to 3, MNIBBLES is 0,
otherwise MNIBBLES is the value of this field plus 4. otherwise MNIBBLES is the value of this field plus 4.
If MNIBBLES is 0, the meta-block is empty, i.e. it does If MNIBBLES is 0, the meta-block is empty, i.e. it does
not generate any uncompressed data. In this case, the not generate any uncompressed data. In this case, the
rest of the meta-block has the following format: rest of the meta-block has the following format:
1 bit: reserved, must be zero 1 bit: reserved, must be zero
2 bits: MSKIPBYTES, # of bytes to represent metadata 2 bits: MSKIPBYTES, # of bytes to represent metadata
length length
@ -1572,37 +1572,37 @@ Alakuijala & Szabadka Expires April 6, 2016 [Page 28]
Internet-Draft Brotli October 2015 Internet-Draft Brotli October 2015
MSKIPBYTES x 8 bits: MSKIPLEN - 1, where MSKIPLEN is MSKIPBYTES x 8 bits: MSKIPLEN - 1, where MSKIPLEN is
the number of metadata bytes; this field is the number of metadata bytes; this field is
only present if MSKIPBYTES is positive, only present if MSKIPBYTES is positive,
otherwise MSKIPLEN is 0 (if MSKIPBYTES is otherwise MSKIPLEN is 0 (if MSKIPBYTES is
greater than 1, and the last byte is all greater than 1, and the last byte is all
zeros, then the stream should be rejected zeros, then the stream should be rejected
as invalid) as invalid)
0 - 7 bits: fill bits until the next byte boundary, 0 - 7 bits: fill bits until the next byte boundary,
must be all zeros must be all zeros
MSKIPLEN bytes of metadata, not part of the MSKIPLEN bytes of metadata, not part of the
uncompressed data or the sliding window uncompressed data or the sliding window
MNIBBLES x 4 bits: MLEN - 1, where MLEN is the length MNIBBLES x 4 bits: MLEN - 1, where MLEN is the length
of the meta-block uncompressed data in bytes (if the of the meta-block uncompressed data in bytes (if the
number of nibbles is greater than 4, and the last number of nibbles is greater than 4, and the last
nibble is all zeros, then the stream should be nibble is all zeros, then the stream should be
rejected as invalid) rejected as invalid)
1 bit: ISUNCOMPRESSED, if set to 1, any bits of compressed 1 bit: ISUNCOMPRESSED, if set to 1, any bits of compressed
data up to the next byte boundary are ignored, and data up to the next byte boundary are ignored, and
the rest of the meta-block contains MLEN bytes of the rest of the meta-block contains MLEN bytes of
literal data; this field is only present if the literal data; this field is only present if the
ISLAST bit is not set (if the ignored bits are not ISLAST bit is not set (if the ignored bits are not
all zeros, the stream should be rejected as invalid) all zeros, the stream should be rejected as invalid)
1-11 bits: NBLTYPESL, # of literal block types, encoded with 1-11 bits: NBLTYPESL, # of literal block types, encoded with
the following variable length code (as it appears in the following variable length code (as it appears in
the compressed data, where the bits are parsed from the compressed data, where the bits are parsed from
right to left, so 0110111 has the value 12): right to left, so 0110111 has the value 12):
Value Bit Pattern Value Bit Pattern
----- ----------- ----- -----------
@ -1657,13 +1657,13 @@ Internet-Draft Brotli October 2015
Block count code + Extra bits for first distance block Block count code + Extra bits for first distance block
count, only if NBLTYPESD >= 2 count, only if NBLTYPESD >= 2
2 bits: NPOSTFIX, parameter used in the distance coding 2 bits: NPOSTFIX, parameter used in the distance coding
4 bits: four most significant bits of NDIRECT, to get the 4 bits: four most significant bits of NDIRECT, to get the
actual value of the parameter NDIRECT, left-shift actual value of the parameter NDIRECT, left-shift
this four bit number by NPOSTFIX bits this four bit number by NPOSTFIX bits
NBLTYPESL x 2 bits: context mode for each literal block type NBLTYPESL x 2 bits: context mode for each literal block type
1-11 bits: NTREESL, # of literal prefix trees, encoded with 1-11 bits: NTREESL, # of literal prefix trees, encoded with
the same variable length code as NBLTYPESL the same variable length code as NBLTYPESL
@ -1687,11 +1687,11 @@ Internet-Draft Brotli October 2015
appears only if NTREESD >= 2, otherwise the context map appears only if NTREESD >= 2, otherwise the context map
has only zero values has only zero values
NTREESL prefix codes for literals NTREESL prefix codes for literals
NBLTYPESI prefix codes for insert-and-copy lengths NBLTYPESI prefix codes for insert-and-copy lengths
NTREESD prefix codes for distances NTREESD prefix codes for distances
9.3. Format of the meta-block data 9.3. Format of the meta-block data
@ -1727,8 +1727,8 @@ Internet-Draft Brotli October 2015
described in Paragraph 7.3. described in Paragraph 7.3.
Block type code for next distance block type, appears only Block type code for next distance block type, appears only
if NBLTYPESD >= 2 and the previous distance block count if NBLTYPESD >= 2 and the previous distance block count
is zero is zero
Block count code + Extra bits for next distance block Block count code + Extra bits for next distance block
length, appears only if NBLTYPESD >= 2 and the previous length, appears only if NBLTYPESD >= 2 and the previous
@ -1831,7 +1831,7 @@ Internet-Draft Brotli October 2015
save previous block type save previous block type
read block count using HTREE_BLEN_I and set BLEN_I read block count using HTREE_BLEN_I and set BLEN_I
decrement BLEN_I decrement BLEN_I
read insert and copy length, ILEN, CLEN with HTREEI[BTYPE_I] read insert and copy length, ILEN, CLEN using HTREEI[BTYPE_I]
loop for ILEN loop for ILEN
if BLEN_L is zero if BLEN_L is zero
read block type using HTREE_BTYPE_L and set BTYPE_L read block type using HTREE_BTYPE_L and set BTYPE_L
@ -1862,9 +1862,9 @@ Internet-Draft Brotli October 2015
read block count using HTREE_BLEN_D and set BLEN_D read block count using HTREE_BLEN_D and set BLEN_D
decrement BLEN_D decrement BLEN_D
compute context ID, CIDD from CLEN compute context ID, CIDD from CLEN
read distance code with HTREED[CMAPD[4 * BTYPE_D + CIDD]] read distance code using HTREED[CMAPD[4 * BTYPE_D + CIDD]]
compute distance by distance short code substitution compute distance by distance short code substitution
move backwards distance bytes in the uncompressed data and move backwards distance bytes in the uncompressed data and
copy CLEN bytes from this position to the uncompressed copy CLEN bytes from this position to the uncompressed
stream, or look up the static dictionary word, transform stream, or look up the static dictionary word, transform
the word as directed, and copy the result to the the word as directed, and copy the result to the
@ -1942,7 +1942,7 @@ Internet-Draft Brotli October 2015
available in the brotli open-source project: available in the brotli open-source project:
https://github.com/google/brotli https://github.com/google/brotli
15. Acknowledgements 15. Acknowledgments
The authors would like to thank Mark Adler for providing helpful The authors would like to thank Mark Adler for providing helpful
review comments, validating the specification by writing an review comments, validating the specification by writing an