Chunking (computing)
In computer programming, chunking has multiple meanings.
In memory management
Typical modern
In HTTP message transmission
Chunking is a specific feature of the HTTP 1.1 protocol.[1] Here, the meaning is the opposite of that used in memory management. It refers to a facility that allows inconveniently large messages to be broken into conveniently-sized smaller "chunks".
In data deduplication, data synchronization and remote data compression
In data deduplication, data synchronization and remote data compression, Chunking is a process to split a file into smaller pieces called chunks by the chunking algorithm. It can help to eliminate duplicate copies of repeating data on storage, or reduces the amount of data sent over the network by only selecting changed chunks. The Content-Defined Chunking (CDC) algorithm like Rolling hash and its variants have been the most popular data deduplication algorithms for the last 15 years.[2]
See also
- Chunk (information), a fragment of data within certain file formats
References
- ^ "HTTP/1.1: Protocol Parameters". Retrieved 2019-12-10.
- ^ FastCDC: a Fast and Efficient Content-Defined Chunking Approach for Data Deduplication (PDF). USENIX ATC ’16. 2016. Retrieved 2019-12-10.