Common:
* wrap dictionary data into `BrotliDictionary` structure
* replace public constant with getter `BrotliGetDictionary`
* reformat dictionary data
Decoder:
* adopt common changes
* clarify acceptable instance usage patterns
* hold reference to dictionary in state
Encoder:
* adopt common changes
* eliminate PIC spots in `CreateBackwardReferences`
* add per-chunk ratio guards for q0 and q1
* precompute relative distances to avoid repeated calculations
* prostpone hasher allocation/initialization
* refactor Hashers to be class-like structure
* further improvements for 1MiB+ inputs
* added new hasher type; made hashers more configurable
Java:
* Pull byte->int magic to `IntReader` from `BitReader`
* pull `BROTLI_MAX_BACKWARD_LIMIT` to constants
* split generic and Zopfli backward references code
* pull hashers init and stitch invocation to encoder
* make `dictionary_hash` a compilation unit
* add `size hint` parameter
* add new hasher
* use `size hint` to pick new hasher for q4
* modernize clz guard (fix#495)
* move `hash to binary tree` to separate file
* add `Initialize` and `Cleanup` to all hashers
* do not raise OOM if malloc(0) == NULL (fix#500)
Python 2.7 for Windows is compiled using MS Visaul C++ 9.0 (Visual Studio 2008).
However the latter does not support many modern C++ features which are
required to compile the Brotli encoder. So we monkey-patch distutils
to always look for MSVC version 10.0 instead of 9.0.
This is used for quality 11, for qualities <= 9 we already
have a simpler hash table.
The static data size is 252 kB, and this removes the
need to initialize a huge hash map at startup, which was
the reason why transforms had to be disabled by default.
In comparison, the static dictionary itself is 120 kB.
This supports every transform, except the kOmitFirstN.
* Change order of members of bit reader state structure.
* Remove unused includes for assert. Add BROTLI_DCHECK
macros and use it instead of assert.
* Do not calculate nbits in common case of ReadSymbol.
* Introduce and use PREDICT_TRUE / PREDICT_FALSE macros.
* Allocate less memory in the brotli decoder if it knows
the result size beforehand. Before this, the decoder
would always allocate 16MB if the encoder annotated the
window size as 22 bit (which is the default), even if the
file is only a few KB uncompressed. Now, it'll only
allocate a ringbuffer as large as needed for the result file.
But only if it can know the filesize, it's not possible
to know that if there are multiple metablocks or too large
uncompressed metablock.
Eugene Kliuchnikov