Video coding format

Source: Wikipedia, the free encyclopedia.

A video coding format

video compression algorithm, most commonly based on discrete cosine transform (DCT) coding and motion compensation. A specific software, firmware, or hardware implementation capable of compression or decompression in a specific video coding format is called a video codec
.

Some video coding formats are documented by a detailed

standardization organizations as technical standards, and are thus known as a video coding standard. There are de facto standards
and formal standards.

Video content encoded using a particular video coding format is normally bundled with an audio stream (encoded using an

MPEG-2 Part 2 or H.264. Another example is the initial specification for the file type WebM, which specifies the container format (Matroska), but also exactly which video (VP8) and audio (Vorbis) compression format is inside the Matroska container, even though Matroska is capable of containing VP9 video, and Opus audio support was later added to the WebM
specification.

Distinction between format and codec

A format is the layout plan for data produced or consumed by a codec.

Although video coding formats such as H.264 are sometimes referred to as codecs, there is a clear conceptual difference between a specification and its implementations. Video coding formats are described in specifications, and software,

H.264), there can be many codecs implementing that specification (e.g. x264, OpenH264, H.264/MPEG-4 AVC products and implementations
).

This distinction is not consistently reflected terminologically in the literature. The H.264 specification calls

H.264 video coding standards and does not contain the word codec.[2] The Alliance for Open Media clearly distinguishes between the AV1 video coding format and the accompanying codec they are developing, but calls the video coding format itself a video codec specification.[3] The VP9 specification calls the video coding format VP9 itself a codec.[4]

As an example of conflation, Chromium's[5] and Mozilla's[6] pages listing their video format support both call video coding formats such as H.264 codecs. As another example, in Cisco's announcement of a free-as-in-beer video codec, the press release refers to the H.264 video coding format as a codec ("choice of a common video codec"), but calls Cisco's implementation of a H.264 encoder/decoder a codec shortly thereafter ("open-source our H.264 codec").[7]

A video coding format does not dictate all

NP-hard problem,[8] meaning that it is practically impossible to find an optimal solution. Though the video coding format must support such compression across frames in the bitstream format, by not needlessly mandating specific algorithms for finding such block-matches and other encoding steps, the codecs implementing the video coding specification have some freedom to optimize and innovate in their choice of algorithms. For example, section 0.5 of the H.264 specification says that encoding algorithms are not part of the specification.[2] Free choice of algorithm also allows different space–time complexity trade-offs for the same video coding format, so a live feed can use a fast but space-inefficient algorithm, and a one-time DVD
encoding for later mass production can trade long encoding-time for space-efficient encoding.

History

The concept of

temporal dimension.[9] In 1967, University of London researchers A.H. Robinson and C. Cherry proposed run-length encoding (RLE), a lossless compression scheme, to reduce the transmission bandwidth of analog television signals.[10]

The earliest digital video coding algorithms were either for

Gbit/s, significantly greater than the bandwidth available in the 2000s.[13]

Motion-compensated DCT

Practical

temporal dimension.[9]

DCT coding is a

University of Texas in 1973, and was published in 1974.[14][15][16]

The other key development was motion-compensated hybrid coding.

videotelephone scene with image quality comparable to a typical intra-frame coder requiring 2-bit per pixel.[22][21]

The DCT was applied to video encoding by Wen-Hsiung Chen,

Compression Labs to commercialize DCT technology.[23] In 1979, Anil K. Jain and Jaswant R. Jain further developed motion-compensated DCT video compression.[26][9] This led to Chen developing a practical video compression algorithm, called motion-compensated DCT or adaptive scene coding, in 1981.[9] Motion-compensated DCT later became the standard coding technique for video compression from the late 1980s onwards.[11][27]

Video coding standards

The first digital video coding standard was

MPEG formats) that followed.[11][27]

MPEG-4/H.263, which was a major leap forward for video compression technology.[28] It uses patents licensed from a number of companies, primarily Mitsubishi, Hitachi and Panasonic.[32]

The most widely used video coding format as of 2019[update] is

ISDB-T, DVB-T or DVB-T2), cable (DVB-C), and satellite (DVB-S2).[36]

A main problem for many video coding formats has been

HTML5 video
tag.

The current-generation video coding format is

AV1 format, intended for free license. As of 2019, AVC is by far the most commonly used format for the recording, compression, and distribution of video content, used by 91% of video developers, followed by HEVC which is used by 43% of developers.[33]

List of video coding standards

See also: List of codecs § Video compression formats
Timeline of international video compression standards
Basic algorithm Video coding standard Year Publishers Committees Licensors Market presence (2019)[33] Popular implementations
DPCM H.120 1984 CCITT
VCEG
 — — Un­known
DCT H.261 1988 CCITT VCEG
BT, Toshiba, etc.[29]
 —
Videoconferencing, videotelephony
Motion JPEG (MJPEG) 1992 JPEG JPEG
Open Source does NOT mean free! [40]
 — QuickTime
MPEG-1 Part 2
 1993 ISO, IEC
MPEG
Matsushita, etc.[41]
 — Video CD, Internet video
H.262 / MPEG-2 Part 2 (MPEG-2 Video) 1995 ISO, IEC, ITU-T MPEG, VCEG
Thomson, Mitsubishi, etc.[30]
 29%
SDTV
DV 1995 IEC IEC Sony, Panasonic Un­known
Camcorders, digital cassettes
H.263 1996 ITU-T VCEG Mitsubishi, Hitachi, Panasonic, etc.[32] Un­known Videoconferencing, videotelephony,
MPEG-4 Visual
MPEG-4 Part 2 (MPEG-4 Visual) 1999 ISO, IEC MPEG Mitsubishi, Hitachi, Panasonic, etc.[32] Un­known Internet video, DivX, Xvid
DWT Motion JPEG 2000 (MJ2) 2001 JPEG[44] JPEG[45] — Un­known Digital cinema[46]
DCT Advanced Video Coding (H.264 / MPEG-4 AVC) 2003 ISO, IEC, ITU-T MPEG, VCEG Panasonic,
etc.[34]
 91%
Silverlight, VOD
Theora 2004 Xiph Xiph — Un­known Internet video,
web browsers
VC-1 2006
SMPTE
SMPTE
 Microsoft, Panasonic, LG, Samsung, etc.[47] Un­known Blu-ray, Internet video
Apple ProRes 2007 Apple Apple Apple Un­known Video production, post-production
High Efficiency Video Coding (H.265 / MPEG-H HEVC) 2013 ISO, IEC, ITU-T MPEG, VCEG Samsung, GE, NTT, JVCKenwood, etc.[39][48] 43%
UHD streaming, High Efficiency Image File Format, macOS High Sierra, iOS 11
AV1 2018
AOMedia
AOMedia
 — 7%
HTML5 video
Versatile Video Coding (VVC / H.266) 2020 JVET JVET Un­known — —

Lossless, lossy, and uncompressed

Consumer video is generally compressed using

H.264 support both.[49]

Uncompressed video formats, such as Clean HDMI, is a form of lossless video used in some circumstances such as when sending video to a display over a HDMI connection. Some high-end cameras can also capture video directly in this format.

Intra-frame

Interframe compression complicates editing of an encoded video sequence.[50] One subclass of relatively simple video coding formats are the

intra-frame video formats, such as DV, in which each frame of the video stream is compressed independently without referring to other frames in the stream, and no attempt is made to take advantage of correlations between successive pictures over time for better compression. One example is Motion JPEG, which is simply a sequence of individually JPEG-compressed images. This approach is quick and simple, at the expense of the encoded video being much larger than a video coding format supporting Inter frame
coding.

Because interframe compression copies data from one frame to another, if the original frame is simply cut out (or lost in transmission), the following frames cannot be reconstructed properly. Making 'cuts' in intraframe-compressed video while video editing is almost as easy as editing uncompressed video: one finds the beginning and ending of each frame, and simply copies bit-for-bit each frame that one wants to keep, and discards the frames one does not want. Another difference between intraframe and interframe compression is that, with intraframe systems, each frame uses a similar amount of data. In most interframe systems, certain frames (such as "I frames" in MPEG-2) are not allowed to copy data from other frames, so they require much more data than other frames nearby.[51]

It is possible to build a computer-based video editor that spots problems caused when I frames are edited out while other frames need them. This has allowed newer formats like HDV to be used for editing. However, this process demands a lot more computing power than editing intraframe compressed video with the same picture quality. But, this compression is not very effective to use for any audio format.[52]

Profiles and levels

A video coding format can define optional restrictions to encoded video, called profiles and levels. It is possible to have a decoder which only supports decoding a subset of profiles and levels of a given video format, for example to make the decoder program/hardware smaller, simpler, or faster.[53]

A profile restricts which encoding techniques are allowed. For example, the H.264 format includes the profiles baseline, main and high (and others). While P-slices (which can be predicted based on preceding slices) are supported in all profiles, B-slices (which can be predicted based on both preceding and following slices) are supported in the main and high profiles but not in baseline.[54]

A level is a restriction on parameters such as maximum resolution and data rates.[54]

See also

Notes

  1. ^ The term video coding includes Advanced Video Coding, High Efficiency Video Coding, and Video Coding Experts Group.[1]

References

  1. ^ Thomas Wiegand; Gary J. Sullivan; Gisle Bjontegaard & Ajay Luthra (July 2003). "Overview of the H.264 / AVC Video Coding Standard" (PDF). IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY.
  2. ^ a b "SERIES H: AUDIOVISUAL AND MULTIMEDIA SYSTEMS : Infrastructure of audiovisual services – Coding of moving video : Advanced video coding for generic audiovisual services". Itu.int. Retrieved January 6, 2015.
  3. ^ "Front Page". Alliance for Open Media. Retrieved May 23, 2016.
  4. ^ Adrian Grange; Peter de Rivaz & Jonathan Hunt. "VP9 Bitstream & Decoding Process Specification" (PDF).
  5. ^ "Audio/Video". The Chromium Projects. Retrieved May 23, 2016.
  6. ^ "Media formats supported by the HTML audio and video elements". Mozilla. Retrieved May 23, 2016.
  7. ^ Rowan Trollope (October 30, 2013). "Open-Sourced H.264 Removes Barriers to WebRTC". Cisco. Archived from the original on May 14, 2019. Retrieved May 23, 2016.
  8. ^ "Chapter 3 : Modified A* Prune Algorithm for finding K-MCSP in video compression" (PDF). Shodhganga.inflibnet.ac.in. Retrieved January 6, 2015.
  9. ^ a b c d e f g h i j "History of Video Compression". ITU-T. Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6). July 2002. pp. 11, 24–9, 33, 40–1, 53–6. Retrieved November 3, 2019.
  10. doi:10.1109/PROC.1967.5493
    .
  11. ^
    ISBN 9780852967102
    .
  12. ^ a b c d e f Lea, William (1994). Video on demand: Research Paper 94/68. House of Commons Library. Retrieved September 20, 2019.
  13. ISBN 9780470857649
    .
  14. doi:10.1016/1051-2004(91)90086-Z
    .
  15. S2CID 149806273
  16. ISBN 978-0-12-580203-1
  17. ^
    doi:10.1109/TCOM.1974.1092258
    .
  18. S2CID 13743007
    .
  19. ISBN 9780120145720
    . A significant advance in image coding methodology occurred with the introduction of the concept of hybrid transform/DPCM coding (Habibi, 1974).
  20. ISBN 9783662466919
    .
  21. ^
    S2CID 62725808
    .
  22. ISBN 9783642870378
    .
  23. ^ a b c Stanković, Radomir S.; Astola, Jaakko T. (2012). "Reminiscences of the Early Work in DCT: Interview with K.R. Rao" (PDF). Reprints from the Early Days of Information Sciences. 60. Retrieved October 13, 2019.
  24. doi:10.1109/TCOM.1977.1093941
    .
  25. CCITT
    . September 1992. Retrieved July 12, 2019.
  26. ISBN 9780786487974
    .
  27. ^
    ISBN 9789812709998
    .
  28. ^ a b c d e f g "The History of Video File Formats Infographic". RealNetworks. April 22, 2012. Retrieved August 5, 2019.
  29. ^ a b "ITU-T Recommendation declared patent(s)". ITU. Retrieved July 12, 2019.
  30. ^ a b "MPEG-2 Patent List" (PDF). MPEG LA. Retrieved July 7, 2019.
  31. ISBN 9781483298511
    .
  32. ^ a b c "MPEG-4 Visual - Patent List" (PDF). MPEG LA. Retrieved July 6, 2019.
  33. ^ a b c "Video Developer Report 2019" (PDF). Bitmovin. 2019. Retrieved November 5, 2019.
  34. ^ a b "AVC/H.264 – Patent List" (PDF). MPEG LA. Retrieved July 6, 2019.
  35. S2CID 2060937
    .
  36. ^ "Digital Video Broadcasting (DVB); Specification for the use of video and audio coding in DVB services delivered directly over IP" (PDF).
  37. ^ "World, Meet Thor – a Project to Hammer Out a Royalty Free Video Codec". August 11, 2015.
  38. ^ Thomson, Gavin; Shah, Athar (2017). "Introducing HEIF and HEVC" (PDF). Apple Inc. Retrieved August 5, 2019.
  39. ^ a b "HEVC Patent List" (PDF). MPEG LA. Retrieved July 6, 2019.
  40. ^ ISO. "Home". International Standards Organization. ISO. Retrieved August 3, 2022.
  41. ^ "ISO Standards and Patents". ISO. Retrieved July 10, 2019.
  42. ^ Davis, Andrew (June 13, 1997). "The H.320 Recommendation Overview". EE Times. Retrieved November 7, 2019.
  43. ISBN 9780780341470
    . H.263 is similar to, but more complex than H.261. It is currently the most widely used international video compression standard for video telephony on ISDN (Integrated Services Digital Network) telephone lines.
  44. ^ "Motion JPEG 2000 Part 3". Joint Photographic Experts Group, JPEG, and Joint Bi-level Image experts Group, JBIG. Archived from the original on October 5, 2012. Retrieved June 21, 2014.
  45. ISBN 9781461507994
    .
  46. ISBN 9780240806174
    .
  47. ^ "VC-1 Patent List" (PDF). MPEG LA. Retrieved July 11, 2019.
  48. ^ "HEVC Advance Patent List". HEVC Advance. Archived from the original on August 24, 2020. Retrieved July 6, 2019.
  49. ^ Filippov, Alexey; Norkin, Aney; Alvarez, José Roberto (April 2020). "RFC 8761 - Video Codec Requirements and Evaluation Methodology". datatracker.ietf.org. Retrieved February 10, 2022.
  50. ^ Bhojani, D.R. "4.1 Video Compression" (PDF). Hypothesis. Retrieved March 6, 2013.
  51. ISBN 9788190639675
    .
  52. ^ "WebCodecs". www.w3.org. Retrieved February 10, 2022.
  53. ^ "Video Rendering - an overview | ScienceDirect Topics". www.sciencedirect.com. Retrieved February 10, 2022.
  54. ^ a b Jan Ozer. "Encoding options for H.264 video". Adobe.com. Retrieved January 6, 2015.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Video_coding_format&oldid=1217579602"