Brotli
Original author(s) | Jyrki Alakuijala, Zoltán Szabadka |
---|---|
Developer(s) | Jyrki Alakuijala, Eugene Kliuchnikov, Robert Obryk, Zoltán Szabadka, Lode Vandevenne |
Initial release | 15 October 2013 |
Stable release | 1.1.0[1]
/ 31 August 2023 |
Repository | |
Written in | Cross-platform |
Platform | Portable |
Type | Data compression |
License | MIT License |
Website | brotli |
Brotli is a lossless data compression algorithm developed by Google. It uses a combination of the general-purpose LZ77 lossless compression algorithm, Huffman coding and 2nd-order context modelling. Brotli is primarily used by
History
Google employees
The Brotli specification was generalized in September 2015 for HTTP stream compression (content-encoding type "br"). This generalized iteration also improved the compression ratio by using a predefined dictionary of frequently used words and phrases. The version of Brotli released in September 2015 by the Google software engineers contained enhancements in generic
The
Brotli support has been added over the years to web browsers, with 96% of worldwide users using a browser that supports the format, as of July 2022.[8]
In 2016 Dropbox reimplemented Brotli in Rust to fulfill their requirement to be more secure against a malicious client. In 2018 they implemented the missing feature so one can append to a Brotli compressed file.[9][10][11]
Algorithm
Brotli's new file format allows its authors to improve upon Deflate by several algorithmic and format-level improvements: the use of context models for literals and copy distances, describing copy distances through past distances, use of move-to-front queue in entropy code selection, joint-entropy coding of literal and copy lengths, the use of graph algorithms in block splitting, and a larger backward reference window are example improvements.
Unlike most general-purpose compression algorithms, Brotli uses a predefined dictionary, roughly 120 KiB in size, in addition to the dynamically populated ("sliding window") dictionary. The predefined dictionary contains over 13000 common words, phrases and other substrings derived from a large corpus of text and HTML documents.[12][3] Using a predefined dictionary has been shown to increase compression where a file mostly contains commonly used words.[13] However, according to Alakuijala, the predefined dictionary does not distract from Brotli's general-ness, and is not the main reason for improved compression. Brotli with an all-zero dictionary still performs well on web content due to the algorithmic advances, he claims.[14]
Brotli's sliding window is limited to 16
Name
While Google's zopfli implementation of the deflate compression algorithm is named after Zöpfli, the
Applications
Brotli compression is generally used as an alternative to gzip on the web, as Brotli provides better overall compression.[17] Compared to gzip compression, JavaScript files compressed with Brotli are roughly 15% smaller, HTML files are around 20% smaller, and CSS files are around 16% smaller.[18]
The reference implementation does ship a command-line program brotli
similar to gzip
,[19] but use in the Unix-like world as a simple compressor is scarce. Libarchive developers find the raw stream format of .br
files difficult to support, as there is no magic number to indicate the file format.[20]
Industry support
Browsers and other clients
- Mozilla Firefox introduced support for the "br" content-encoding method in version 44 (released on 26 January 2016).[21]
- Google Chrome has supported the "br" content-encoding method since version 50 (released on 20 April 2016).[22]
- Opera has supported the "br" content-encoding method since version 38 (released 8 June 2016).[22]
- Microsoft Edge has supported the "br" content-encoding method since version 15 (released on 5 April 2017).[23]
- Safari has supported the "br" content-encoding method since version 11 (released on 5 October 2017).
- cURL has a compile-time option to support the "br" content-encoding method using libbrotli as of version 7.57, released on 29 November 2017.[24]
- 7zip is available extended with Brotli by 7zip-zstd.[25]
- PeaZip supports Brotli .BR format for compression and extraction[26]
Web servers
- For Apache HTTP Server, the "br" content-encoding method has been supported by the mod_brotli module since version 2.4.26.[27]
- Microsoft IIS has a supported extension since May 2018 that adds support for the "br" content-encoding method.[28]
- Microsoft Azure Front Door can dynamically compress content on the edge using Brotli[29] since its launch on April 17, 2019.[30]
- nginx has a ngx_brotli module provided by Google since December 2016.
- Node.js features a built-in native en- and decoder since version 11.7.0, which can be used to support the "br" content-encoding.
- Amazon CloudFront can automatically compress cacheable responses at the edge using Brotli, as of September 2020.[31]
- LiteSpeed Web Server has included the "br" content-encoding method for static files only since version 5.2 in July 2017.
- Cloudflare CDN offers a brotli option to compress data between its edge node and the user.[32]
- NaviServer added support in version 4.99.17b1
- Caddy serves statically compressed .br files since version 0.9.4 from December 21st, 2016.
- lighttpd mod_deflate supports .br since 1.4.56[33] from November 2020.
References
- ^ "Release 1.1.0". 31 August 2023. Retrieved 18 September 2023.
- ^ Sheeter, Rod (February 18, 2015), "Smaller Fonts with WOFF 2.0 and unicode-range", Google Open Source Blog, Mountain View, CA: opensource.googleblog.com.
- ^ a b Alakuijala, Jyrki; Kliuchnikov, Evgenii; Szabadka, Zoltan; Vandevenne, Lode (22 September 2015), "Comparison of Brotli, Deflate, Zopfli, LZMA, LZHAM and Bzip2 Compression Algorithms" (PDF), The Comprehensive R Archive Network, r-project.org.
- ^ a b c Szabadka, Zoltan (September 22, 2015), "Introducing Brotli: a new compression algorithm for the internet", Google Open Source Blog, Mountain View, CA: opensource.googleblog.com.
- ^ a b c Alakuijala, Jyrki; Szabadka, Zoltan (2016), RFC 7932: Brotli Compressed Data Format, Internet Engineering Task Force Request for Comments, Fremont, CA: IETF Trust.
- ^ "W3C Receives Emmy Award for Standardizing Font Technology". 2022-06-01.
- ^ "Changing the face of the web: W3C Web Fonts Working Group and MPEG recognized with a Technology & Engineering Emmy Award". 2022-06-01.
- ^ "Can I use... - Brotli". 2022-06-28.
- ^ Lossless compression with Brotli in Rust for a bit of Pied Piper on the backend, Daniel Reiter Horn and Mehant Baid, 2016-06-29.
- ^ [1], Rishabh Jain and Daniel Reiter Horn, 2020-08-04
- ^ append to brotli compressed file, github ticket to google Brotli, listing implementation ideas, 2017-12-06
- ^ Chirgwin, Richard (September 23, 2015), "Google's new squeeze: Brotli compression open-sourced", The Register, theregister.co.uk.
- S2CID 8707991..
- ^ Alakuijala, Jyrki (May 15, 2021). "Static dictionary is not why Brotli reaches excellent compression density. Much ..." Hacker News.
- ^ Kliuchnikov, Eugene. "How to use large window sizes? · Issue #639 · google/brotli". GitHub.
Currently we are testing "Large Window Brotli" extension that will allow up to 1GiB window. [...] "Large Window Brotli" is landed.
- ^ Adler, Mark (Jan 26, 2015), "Brotli specification review and verification", Adler brotli, San Francisco: GitHub.
- ^ Calvano, Paul (2018-07-25). "Brotli Compression: How Much Will It Reduce Your Content?". Retrieved 2021-03-07.
- ^ Pandjarov, Hristo (2021-01-13). "More Site Speed Gains with Brotli Compression Algorithm". SiteGround. Retrieved 2021-03-07.
- ^ "brotli(1) manual page". manned.org.
- ^ "Brotli support · Issue #1238 · libarchive/libarchive". GitHub.
Without a magic signature, libarchive cannot automatically recognize the file type, so it cannot automatically decompress. (Libarchive does not consider the file name, only the contents.)
- ^ Goodger, Ben; et al. (26 January 2016), "Firefox 44 release notes", Mozilla Firefox, Mozilla Foundation.
- ^ a b Baheux, Kenji (15 January 2016), "Accept-encoding: br on HTTPS connection", Chrome Platform Status, chromestatus.com.
- ^ Trace, Rob (December 20, 2016), "Introducing Brotli compression in Microsoft Edge", Microft Edge Developer, blogs.windows.com
- ^ Stenberg, Daniel; et al. "curl - Changes". curl.haxx.se. Retrieved 14 January 2018.
- ^ "README". GitHub. 15 October 2021.
- ^ "Google Brotli: How to compress, open, extract BR files".
- ^ "Changes with Apache 2.4.26", Apache HTTPD repository, svn.apache.org.
- ^ "Higher Compression Ratio with Brotli compression". 6 Oct 2023.
- ^ "Caching with Azure Front Door". docs.microsoft.com. 15 June 2023.
- ^ "Azure Front Door Service is now available". azure.microsoft.com.
- ^ "Amazon CloudFront announces support for Brotli compression". aws.amazon.com.
- ^ "What will Cloudflare compress?". support.cloudflare.com.
- ^ "lighttpd 1.4.56 release info". redmine.lighttpd.net.
- Notes
- - Finley, Klint (22 September 2015), "Hooli, I Mean Google, Gives Away Compression Code for Free", Wired Online, wired.com.