Information Theory & Error-Correcting Codes: Shannon Entropy to Hamming & Turbo Codes
Encyclopedia/1. The Cosmos & The Natural World/1. Mathematics & Formal Systems/10. Differential Equations & Dynamical Systems • Curated by Admin Timeline.sg
Information theory and error-correcting codes have revolutionized digital communication. From Shannon's entropy formula to modern turbo and LDPC codes, these mathematical tools enable reliable data transmission over noisy channels.
Chronological Storyline (41 Milestones)
1948 CE
Shannon Publishes Information Theory
Claude Shannon's landmark paper 'A Mathematical Theory of Communication' introduces the concepts of entropy, channel capacity, and the noisy-channel coding theorem, laying the foundation for the entire field. #informationtheory #mathematics
Shannon Publishes Information Theory By Claude E. Shannon and Warren Weaver - https://archive.org/details/in.ernet.dli.2015.503815/page/n5/mode/2up, Public domain, https://commons.wikimedia.org/w/index.php?curid=171159167
1949 CE
Golay Codes Introduced
Marcel Golay describes the binary Golay codes, which are perfect error-correcting codes widely used in deep-space communications. #errorcorrection #codingtheory
Golay Codes Introduced By Life of Riley - Own work based on: ✦ BinaryGolayCode.png: by Calcyman, Public domain, https://commons.wikimedia.org/w/index.php?curid=16562183
1949 CE
Shannon's Communication Theory of Secrecy Systems
Shannon publishes a paper applying information theory to cryptography, establishing the theoretical basis for secure communication. #cryptography #informationtheory
1950 CE
Hamming Codes Invented
Richard Hamming develops the first error-correcting codes, known as Hamming codes, at Bell Labs. These codes can detect and correct single-bit errors and are fundamental to coding theory. #errorcorrection #codingtheory
Hamming Codes Invented By User:Cburnett - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=1511411
1951 CE
Shannon's Noisy Channel Coding Theorem
Shannon proves that for any noisy channel, there exists a code that achieves reliable communication up to a maximum rate called the channel capacity. #informationtheory #mathematics
1955 CE
Convolutional Codes Introduced
Peter Elias introduces convolutional codes, a class of error-correcting codes that process data in a sliding window, enabling continuous stream error correction. #errorcorrection #codingtheory
1959 CE
BCH Codes Discovered
Bose, Ray-Chaudhuri, and Hocquenghem independently discover a class of cyclic error-correcting codes known as BCH codes, which allow for multiple error correction. #codingtheory #errorcorrection
1960 CE
Reed-Solomon Codes Published
Irving Reed and Gustave Solomon publish their seminal paper on Reed-Solomon codes, which are powerful non-binary cyclic codes used in storage and deep-space communication. #codingtheory #errorcorrection
1962 CE
Gallager Introduces LDPC Codes
Robert Gallager introduces low-density parity-check (LDPC) codes in his PhD dissertation, though they are largely forgotten for decades due to computational complexity. #codingtheory #LDPC
1967 CE
Viterbi Algorithm Developed
Andrew Viterbi publishes the Viterbi algorithm, a dynamic programming algorithm for maximum-likelihood decoding of convolutional codes, widely used in digital communications. #decoding #communications
1968 CE
Berlekamp-Massey Algorithm
James Massey publishes the Berlekamp-Massey algorithm, an efficient algorithm for decoding BCH and Reed-Solomon codes, enabling widespread use of algebraic codes. #decoding #errorcorrection
Berlekamp-Massey Algorithm By Aats1988 - Own work, Public domain, https://commons.wikimedia.org/w/index.php?curid=9030345
1971 CE
Viterbi Algorithm Paper Published
Andrew Viterbi formally publishes his algorithm in IEEE Transactions on Information Theory, later becoming essential for satellite and cellular communications. #communications #decoding
1973 CE
Forney Concatenated Codes
G. David Forney Jr. introduces concatenated codes, which combine inner and outer codes to achieve near-channel capacity performance, influencing deep-space standards. #codingtheory #capacity
1977 CE
Voyager Uses Reed-Solomon Codes
NASA's Voyager spacecraft use Reed-Solomon codes for error correction during its mission to Jupiter and Saturn, demonstrating the power of algebraic codes in deep space. #space #errorcorrection
1981 CE
CCSDS Standard for Telemetry
The Consultative Committee for Space Data Systems (CCSDS) adopts Reed-Solomon codes for space telemetry standardization, enabling reliable deep-space communications. #space #standards
1982 CE
Compact Disc Uses CIRC
The Compact Disc digital audio standard adopts the Cross-Interleaved Reed-Solomon Code (CIRC) for robust error correction against scratches and dust. #audio #consumertech
1984 CE
Ungerboeck Trellis Coded Modulation
Gottfried Ungerboeck introduces trellis coded modulation (TCM), combining coding and modulation to achieve coding gain without bandwidth expansion. #coding #modulation
1988 CE
V.42 Error Correction Standard
The ITU-T recommendation V.42bis includes error correction protocols for modems, incorporating Lempel-Ziv compression and adaptive error control. #standards #communications
1989 CE
CCSDS Reed-Solomon Standard Updated
CCSDS revises its recommendation for Reed-Solomon codes, enabling higher data rates for deep-space missions. #space #errorcorrection
1993 CE
Turbo Codes Invented
Claude Berrou, Alain Glavieux, and Punya Thitimajshima unveil turbo codes, achieving near-Shannon-limit performance with iterative decoding, revolutionizing channel coding. #codingtheory #turbocodes
1996 CE
LDPC Codes Rediscovered
David MacKay and others rediscover LDPC codes, showing that with iterative belief propagation decoding they also approach Shannon capacity, sparking new research. #codingtheory #LDPC
1997 CE
ADSL Uses Reed-Solomon
Asymmetric Digital Subscriber Line (ADSL) standards employ Reed-Solomon codes for robust data transmission over copper telephone lines. #broadband #telecom
1999 CE
3G Mobile Adopts Turbo Codes
The 3G cellular standard UTRA incorporates turbo codes for high-speed data channels, enabling reliable mobile internet. #mobile #telecom
2000 CE
Fountain Codes Introduced
Michael Luby introduces Luby Transform (LT) codes, the first practical fountain codes, enabling rateless erasure correction for multicast and file delivery. #erasurecoding #network
2001 CE
Raptor Codes Developed
Amin Shokrollahi introduces Raptor codes, an extension of fountain codes with linear-time encoding and decoding, later standardized for broadcast. #erasurecoding #broadcast
2002 CE
DVB-T2 Employs LDPC
The second-generation digital terrestrial television standard DVB-T2 uses LDPC codes for improved coverage and capacity. #broadcast #tv
2003 CE
DVB-S2 Adopts LDPC
The second-generation satellite broadcasting standard DVB-S2 selects LDPC codes with a BCH outer code, achieving high spectral efficiency. #broadcast #satellite
DVB-S2 Adopts LDPC By Andreas -horn- Hornig - Erstellt von mir, Photo nachverbessert durch Michael "Blacky" Ritter von www.hdtvtotal.com, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=839313
2005 CE
IEEE 802.11n Uses LDPC
The Wi-Fi standard 802.11n includes LDPC codes as an optional high-throughput mode for improved wireless LAN performance. #wifi #wireless
2007 CE
ITU G.hn Uses LDPC
The ITU G.hn standard for home networking adopts LDPC codes for powerline, phone line, and coaxial communication. #networking #home
2008 CE
Polar Codes Invented
Erdal Arıkan proves the existence of polar codes, a class of capacity-achieving codes with low encoding/decoding complexity based on channel polarization. #codingtheory #polarcodes )
2009 CE
WiMAX Adopts LDPC
The IEEE 802.16e (WiMAX) standard incorporates LDPC codes for efficient broadband wireless access. #wimax #wireless
WiMAX Adopts LDPC By WiMAX Forum - https://web.archive.org/web/20160315075519/http://resources.wimaxforum.org/sites/wimaxforum.org/files/document_library/DWP0610_3%20SECRETS%20TO%20CAPTURE%20THE%20MOBILE%20WIMAX%20MARKET.pdf, Public domain, https://commons.wikimedia.org/w/index.php?curid=48747545
2010 CE
LDPC for 802.11ac
The Wi-Fi standard 802.11ac adopts LDPC codes as a mandatory feature for very high throughput in the 5 GHz band. #wifi #highspeed
The 60 GHz Wi-Fi standard 802.11ad uses LDPC codes for multi-gigabit wireless communication. #wifi #millimeterwave
2013 CE
Polar Codes Adopted for 5G Control
The 3GPP decides to use polar codes for the control channel of 5G NR, ensuring ultra-reliable low-latency communication. #5G #telecom
2016 CE
5G NR Chooses LDPC for Data
3GPP selects LDPC codes for the 5G NR data channel, enabling high throughput and low latency. #5G #LDPC
2017 CE
5G NR Phase 1 Release
3GPP Release 15 specifies 5G NR with LDPC for data and polar codes for control, marking a major milestone in cellular evolution. #5G #standards
2018 CE
Quantum Error Correction Advances
Research on quantum error correction codes, such as surface codes and stabilizer codes, progresses, aiming to enable fault-tolerant quantum computing. #quantum #errorcorrection
2020 CE
LDPC in 5G NR Commercial Deployment
5G NR networks commercially deploy LDPC codes, achieving peak data rates beyond 10 Gbps. #5G #LDPC
2021 CE
Wi-Fi 7 Uses LDPC
IEEE 802.11be (Wi-Fi 7) includes LDPC codes for extreme throughput up to 46 Gbps. #wifi #nextgen
