Commit Graph

69 Commits

Author SHA1 Message Date
Brotli bb809ac908 Use a hash table header and SIMD to speed up hash table operations (similar to [Swiss Tables](https://abseil.io/about/design/swisstables)).
PiperOrigin-RevId: 631982664
2024-05-08 17:59:58 -07:00
Brotli d01a4caaa8 Internal change
PiperOrigin-RevId: 626960053
2024-04-22 02:07:25 -07:00
Brotli 1b3a5ccb6e Prefetch the backreference hashtable bucket.
Place the prefetch before the last distance checks, to give the prefetch enough time to work.

PiperOrigin-RevId: 626228820
2024-04-18 20:00:02 -07:00
Evgenii Kliuchnikov 443af10a80 add (assumption) check
PiperOrigin-RevId: 625632989
2024-04-17 04:10:04 -07:00
Evgenii Kliuchnikov c1c76e993f Don't check `cur_ix_masked` against `ring_buffer_mask`.
`cur_ix_masked` isn't changing from iteration to iteration, and `max_length` ensures we never find a match long enough to walk off the ring buffer.

PiperOrigin-RevId: 624701948
2024-04-14 06:36:02 -07:00
Brotli 709c4672d4 Fix minor syntax issues.
Missing semicolons.
Move checks below variable declarations for c89.

PiperOrigin-RevId: 624199887
2024-04-12 09:16:00 -07:00
Brotli a76d96e730 Don't check `cur_ix_masked` against `ring_buffer_mask`.
`cur_ix_masked` isn't changing from iteration to iteration, and `max_length` ensures we never find a match long enough to walk off the ring buffer.

PiperOrigin-RevId: 624162764
2024-04-12 06:50:51 -07:00
Brotli a813a6a1e4 Update the `H5` hasher to use the `H6`'s `FN(STORE)`.
PiperOrigin-RevId: 623885589
2024-04-11 11:23:47 -07:00
Brotli f964a1e8ac Internal change
PiperOrigin-RevId: 623073126
2024-04-09 00:19:11 -07:00
Brotli cdbe7fc739 Internal change
PiperOrigin-RevId: 622802698
2024-04-08 04:30:44 -07:00
Brotli 9351fa7ffb Compare 4 bytes when checking if a longer match is possible.
Loading and comparing 4 bytes is ~as fast as 1 byte, but allows us to avoid more full match length calculation.

PiperOrigin-RevId: 617556847
2024-03-20 10:30:00 -07:00
Brotli 9717649c31 Use BROTLI_MAX_STATIC_CONTEXTS instead of magic constants in encode.c
PiperOrigin-RevId: 615341475
2024-03-13 02:47:52 -07:00
Evgenii Kliuchnikov 9b83be233e fix wording
PiperOrigin-RevId: 576788685
2023-10-26 02:03:20 -07:00
Evgenii Kliuchnikov e7313b0c4e tune memory manager for BROTLI_EXPERIMENTAL
PiperOrigin-RevId: 560703386
2023-08-28 07:20:49 -07:00
Evgenii Kliuchnikov 2a5a088b03 more tuning for BROTLI_EXPERIMENTAL + clean-on-oom
PiperOrigin-RevId: 558771745
2023-08-21 06:36:24 -07:00
Evgenii Kliuchnikov 3ebb2d30ab Move serialized dictionary feature behind the flag.
BROTLI_SHARED_DICTIONARY_SERIALIZED enum value is a part of API,
but it should not be used (will cause failures).
Changing how serialized dictionaries work won't be considered as an API change, until this feature is enabled.
Enabling this feature in the future will be considered as a "compatible" change.

PiperOrigin-RevId: 558091676
2023-08-18 02:55:33 -07:00
Evgenii Kliuchnikov 0f2157cc5e Update comment; fixes #1061
PiperOrigin-RevId: 557501089
2023-08-16 08:55:14 -07:00
Evgenii Kliuchnikov a560089843 speedup q5-9 on large files
PiperOrigin-RevId: 553440457
2023-08-03 04:58:52 -07:00
Evgenii Kliuchnikov 117b68b745 speedup encoder on q5-9 / 1MB+ files
PiperOrigin-RevId: 553087469
2023-08-02 03:05:57 -07:00
Evgenii Kliuchnikov 779a49bfd6 bake in runtime constant
PiperOrigin-RevId: 549590409
2023-07-20 04:18:46 -07:00
Evgenii Kliuchnikov 04f294b18a Fix emitting 1-byte long metadata block
PiperOrigin-RevId: 505484299
2023-01-30 09:10:28 +00:00
Brotli 1e61e972fb speed up encoding by ~5 %
PiperOrigin-RevId: 505061835
2023-01-30 09:10:14 +00:00
Aron Parker 71fe6cac06
Fix BrotliEncoderEstimatePeakMemoryUsage (#1002)
Fixes https://github.com/google/brotli/issues/1001
2023-01-07 22:01:47 +01:00
Eugene Kliuchnikov 0ea4603880
Fix MSVC warning (#998)
Fix #875
2023-01-04 12:10:29 +01:00
Evgenii Kliuchnikov a8f5813b84 Update
Documentation:
  - add note that brotli is a "stream" format, not an archive-like
  - regenerate .1 with Pandoc
Build:
  - drop legacy "BROTLI_BUILD_PORTABLE" option
  - drop "BROTLI_SANITIZED" definition
Code:
  - c: comb includes
  - c/enc: extract encoder state into separate header
  - c/enc: drop designated q10 codepath
  - c/enc: dealing better with flushing of empty stream
  - fix MSVC compilation
API:
  - py: use library version instead of one in version.h
  - c: add plugable API to report consumed input / produced output
  - c/java: support "lean" prepared dictionaries (without copy of source)
2022-11-17 13:03:09 +00:00
Jyrki Alakuijala 4ec67035c0
Merge pull request #929 from jbms/fix-vla-parameter
Fix -Werror=vla-parameter errors with GCC 11.2.0
2021-12-07 01:47:07 +01:00
Eugene Kliuchnikov 8376f72ed6
Prepare for copybara (#939)
Co-authored-by: Eugene Kliuchnikov <eustas@chromium.org>
2021-11-10 10:34:39 +01:00
Jeremy Maitin-Shepard 27dd726540 Fix -Werror=vla-parameter errors with GCC 11.2.0 2021-09-14 12:27:45 -07:00
Eugene Kliuchnikov 62662f87cd
Strip "./" in includes (#925)
Co-authored-by: Eugene Kliuchnikov <eustas@chromium.org>
2021-09-08 09:18:45 +02:00
Eugene Kliuchnikov 68f1b90ad0
Update (#918)
Prepare to use copybara worklow.
2021-08-18 19:15:07 +02:00
Eugene Kliuchnikov 19d86fb9a6
Merge-in SharedDictionary feature (#916)
Co-authored-by: Eugene Kliuchnikov <eustas@chromium.org>
2021-08-04 14:42:02 +02:00
Eugene Kliuchnikov 630b5084ee
Update (#914)
* slimmer stack frames in encoder
 * fix MSAN problem in hasher_composite
   (not dangerous, only in large_window mode)
 * fix JNI decoder wrapper - power-of-two payloads fail to decode sometimes
 * reformat polyfil.js and decode_test.js
2021-07-29 22:29:43 +02:00
Adrián Herrera Arcila 0a3944c8c9
Fix VLA parameter warning (#893)
Make VLA buffer types consistent in declarations and definitions.
Resolves build crash when using -Werror due to "vla-parameter" warning.

Signed-off-by: Adrian Herrera <adr.her.arc.95@gmail.com>
2021-06-23 09:53:59 +02:00
Ikko Ashimine bdcfb123c8
Fix typo in hash_composite_inc.h (#903)
defered -> deferred
2021-06-23 09:42:28 +02:00
Eugene Kliuchnikov 5692e422da
Update (#852)
* Update

 * comments and clarifications in block_splitter
 * power-of-2 aligned allocations for Hasher
 * refresh decode.js from Java sources
 * disable JS build
2021-01-18 10:56:39 +01:00
Dmitry Rozhkov 7e8e207ce2
Fix clang-10 compilation issue (#839)
clang-10 throws the following error:
In file included from external/org_brotli/c/enc/bit_cost.c:9:
external/org_brotli/c/enc/./bit_cost.h:48:16: error: implicit conversion
from 'size_t' (aka 'unsigned long') to 'double' may lose precision
[-Werror,-Wimplicit-int-float-conversion]
  if (retval < sum) {
             ~ ^~~
1 error generated.

Make the conversion explicit.
2020-09-07 09:40:03 +02:00
Eugene Kliuchnikov 223d80cfbe
Update (#826)
* IMPORTANT: decoder: fix potential overflow when input chunk is >2GiB
 * simplify max Huffman table size calculation
 * eliminate symbol duplicates (static arrays in .h files)
 * minor combing in research/ code
2020-08-26 12:32:27 +02:00
Eugene Kliuchnikov 7f740f1308
Update (#807)
- fix formatting
 - fix type conversion
 - fix no-op arithmetic with null-pointer
 - improve performance of hash_longest_match64
 - go: detect read after close
 - java decoder: support compound dictionary
 - remove executable flag on non-scripts
2020-05-15 11:06:21 +02:00
Clinton Ingram 924b2b2b9d
Move TZCNT and BSR intrinsics to platform.h, add MSVC versions (#636) 2020-03-19 11:57:56 +01:00
Cristi Vîjdea f503cb709c
Add HAVE_LOG2 build macro (#783)
* Add HAVE_LOG2 build macro

Fixes #781

* Rename macro to BROTLI_HAVE_LOG2 and move comment for visibility
2020-03-19 10:46:52 +01:00
Eugene Kliuchnikov 78e7bbc3c3
Update (#753)
* fix executable mode of decode.js
 * explain clang-analyser about non-nullability
 * fix "dead assignment"
 * rename proguard.cfg -> proguard.pgcfg
2019-05-03 11:51:11 +02:00
Eugene Kliuchnikov 4b2b2d4f83
Update (#749)
Update:

 * Bazel: fix MSVC configuration
 * C: common: extended documentation and helpers around distance codes
 * C: common: enable BROTLI_DCHECK in "debug" builds
 * C: common: fix implicit trailing zero in `kPrefixSuffix`
 * C: dec: fix possible bit reader discharge for "large-window" mode
 * C: dec: simplify distance decoding via lookup table
 * C: dec: reuse decoder state members memory via union with lookup table
 * C: dec: add decoder state diagram
 * C: enc: clarify access to static dictionary
 * C: enc: improve static dictionary hash
 * C: enc: add "stream offset" parameter for parallel encoding
 * C: enc: reorganize hasher; now Q2-Q3 require exactly 256KiB
           to avoid global TCMalloc lock
 * C: enc: fix rare access to uninitialized data in ring-buffer
 * C: enc: reorganize logging / checks in `write_bits.h`
 * Java: dec: add "large-window" support
 * Java: dec: improve speed
 * Java: dec: debug and 32-bit mode are now activated via system properties
 * Java: dec: demystify some state variables (use better names)
 * Dictionary generator: add single input mode
 * Java: dec: modernize tests
 * Bazel: js: pick working commit for closure rules
2019-04-12 13:57:42 +02:00
Eugene Kliuchnikov d0ffe60b87
Verbose CLI + start pulling "Shared-Brotli" (#722)
* Verbose CLI + start pulling "Shared-Brotli"

 * vesbose CLI output; fix #666
 * pull `SHIFT` transforms; currently this is semantically dead code;
   later it will be used by "Shared-Brotli"
2018-10-24 16:06:09 +02:00
Eugene Kliuchnikov f7cbc97c96
Fix typo / minor formatting (#716)
* Fix typo / minor formatting / pull computable constant to the place of use.
2018-10-16 16:46:54 +02:00
Eugene Kliuchnikov 2216a0dd63
Update (#706)
Update
 * add ASAN/MSAN unaligned read specializations
 * add "brotli" prefix to u_uint64 type
 * increment version to 1.0.06
 * fix CoverityScan "unused assignment" warning
 * fix JDK 8<->9 incompatibility
 * add encoder optimization for empty input
 * regenerate JS decoder
 * unbreak Travis builds
2018-09-13 08:09:32 -04:00
Eugene Kliuchnikov eb12ec04eb
Update (#688)
* add rolling-composite-hasher for large-window mode
* make API methods explicitly public
2018-06-20 15:14:10 +02:00
Eugene Kliuchnikov 8544ae858d
Update (#680)
* fix MSVC warnings
 * cleanups
2018-06-09 11:17:13 +02:00
Eugene Kliuchnikov 48a25b3fa4
Fix #671 (#672) 2018-05-18 22:07:52 +02:00
Eugene Kliuchnikov f5ed35d065
Update (#664)
* Update
 * fix ifdef style
 * get back to fine-compiler-version-based-macros (use Hedley)
 * fix q=0 histogram collection for very long copy/insert commands
2018-05-03 11:16:21 +02:00
Eugene Kliuchnikov 6000396155
Remove unprefixed macros from public headers (#662) 2018-04-20 14:10:55 +02:00