Resumen de Aproximate and exact inference for channel coding.
Modern communications systems are one of the backbones of the present society, of our interconnected world. Day by day, the optimality com- munication systems is a must due to the huge growth of traffic in worldwide communications. Even though there exist many techniques to improve the efficiency of a transmission, channel coding is the one that allows to reach the maximum achievable limit, i.e. the channel capacity, that a communication system can transmit over a noisy channel. The essence of channel coding is quite simple: add redundant information to the transmission to protect it, to preserve this way the integrity of the transmitted information against the noise in the channel.
Channel coding is nowadays an essential part of any communication systems. Although in the beginning only wireline, satellite and wireless communica- tions implemented a channel encoder and decoder, nowadays even multi-Tb/s optical systems are starting to implement it. We have to come back to 1948, when Shannon proposed the concept of channel capacity in his groundbreak- ing paper ¿A mathematical theory on communications¿ (Shannon, 1948), to find the beginning of the field known as coding theory. Therein, Shannon postu- lates that there is a maximum achievable bound for a communication system, the channel capacity, and that the way to achieve it is through channel coding. Thereby, since then coding theorists have tried to find optimal coding and decoding techniques to achieve this limit.
We only had to wait fifteen years until Gallager proposed low-density parity- check (LDPC) codes in 1963 (Gallager, 1963), the first capacity-achieving tech- nique. However, this technique was forgotten due to its complexity require- ments could not fit in the technology of those ages. Then, it was promoted again by MacKay and Neal in the mid-nineties (MacKay & Neal, 1996), where they considered sum-product algorithm as the decoding technique (Pearl, 1988), also known as belief propagation (BP). During the last two decades, coding theorists have been focused on the paradigm of sparse linear block codes and iterative message passing decoders, loosely referred to modern coding theory (Richardson & Urbanke, 2008).
Assuming sum-product algorithm as the standard decoding algorithm, most of the effort has revolved around the optimal construction of the codes, looking for capacity achieving techniques under BP decoding. This search has led to optimized irregular LDPC constructions (Richardson et al., 2001) and LDPC convolutional (LDPCC) codes (Sridharan et al., 2004). Both techniques achieve capacity for a code length tending to infinity. But in the finite-length regime (Polyanskiy et al., 2010), they still suffer the negative effect of loops, which might remarkably degrade the final performance of the BP decoder (McGowan & Williamson, 2003). In addition, many communication systems rely on short and medium size code lengths, of hundreds and thousands, which points towards the importance of improving the performance of codes on this regime.
In this thesis, we propose to improve the decoding performance either through the decoding process or by means of improving the estimate of the input probabilities to the decoder. First, in Chapters 4 and 5, we rely on the proposal of Olmos et al., the TEP decoder (Olmos et al., 2012), to improve the finite-length performance of LDPC and LDPCC codes. The techniques used, the expectation propagation (EP) algorithm, and its extension the tree- structured EP (Minka, 2001; Minka & Qi, 2003), impose a tree-structure over the graph of the code, capturing this way the relations between variables.
We generalize first in Chapter 4 the TEP decoder over the binary erasure channel (BEC), proposing the GTEP algorithm. By graphically processing the graph of the code, the GTEP efficiently obtains the ML solution, and by using regular LDPC codes, it provides a capacity achieving performance.
Then, in Chapter 5 we extend the formulation of the TEP decoder for any binary memoryless channel (BMS). We focus on the decoding of LDPC and LDPCC codes over the binary symmetric channel and the additive white Gaus- sian noise (AWGN) channel. For these scenarios, we show how the pairwise terms introduced in the graph capture additional information that confronts the negative effect of loops. Thereby, the performance for the finite-length regime is remarkably improved.
Finally, in Chapter 6 we go one step forward and consider the existence of inter symbol interference in the AWGN channel. Despite in the analysis of the decoding performance it is always assumed a perfect knowledge of the channel, we come up with the possibility of uncertainties and errors in this estimation. By using a Bayesian approach to the problem of channel equalization, we propose a Bayesian equalizer that, without increasing the complexity of the generally used BCJR algorithm (Bahl et al., 1974), allows a better estimation of the probabilities provided to the LDPC decoder. By further studying the performance a¿er BP decoding, we conclude that the channel decoder benefits from this better estimation of the probabilities, reporting a remarkable increase in the performance of the communication system.
Summarizing, in this thesis we propose to focus on the decoding problem but from the perspective of the possible drawbacks that we might find in practical systems. For the natural flaw of codes in the finite-length regime, the cycles in the graph, we propose the GTEP and TEP decoders for the BEC and the BMS, respectively, techniques that demonstrate a remarkable performance over the BP for short and medium code lengths. In addition, we also highlight the great importance of the estimation of the probabilities fed to the decoder, and propose a Bayesian equalization technique to improve the performance of this process when there are uncertainties in the channel estimation.
References Bahl, L., Cocke, J., Jelinek, F., & Raviv, J. (1974). Optimal decoding of linear codes for minimizing symbol error rate (corresp.). Information Theory, IEEE Transactions on, 20(2), 284--287.
Gallager, R. G. (1963). Low density parity check codes. MIT Press.
MacKay, D., & Neal, R. (1996, Aug.). Near Shannon limit performance of low-density parity-check codes. Electronics Letters, 32(18), 1645. doi: 10 .1049/el:19961141 McGowan, J. A., & Williamson, R. C. (2003). Loop removal from LDPC codes.
In IEEE Information Theory Workshop (ITW) (pp. 230--233).
Minka, T. (2001). Expectation propagation for approximate Bayesian inference.
In Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence (UAI 01) (pp. 362--369).
Minka, T., & Qi, Y. (2003). Tree-structured approximations by expectation propagation. In Proceedings of the Neural Information and Processing Systems (NIPS).
Olmos, P. M., Murillo-Fuentes, J. J., & Pérez-Cruz, F. (2012). Tree-structure expectation propagation for LDPC tree-structure expectation propagation for LDPC decoding over the BEC. IEEE Transactions on Information Theory. Retrieved from http://arxiv.org/abs/1009.4287 Pearl, J. (1988). Probabilistic reasoning in intelligent systems: networks of plausible Inference. Morgan Kaufmann.
Polyanskiy, Y., Poor, H., & Verdu, S. (2010, May). Channel coding rate in the finite blocklength regime. IEEE Transactions on Information Theory, 56(5), 2307 -2359. doi: 10.1109/TIT.2010.2043769 Richardson, T. J., Shokrollahi, A., & Urbanke, R. (2001, Feb.). Design of capacity- approaching irregular low-density parity-check codes. IEEE Transactions on Information Theory, 47(2), 619 -637. doi: 10.1109/18.910578 Richardson, T. J., & Urbanke, R. (2008). Modern coding theory. Cambridge University Press.
Shannon, C. E. (1948). A mathematical theory of communication. Bell Systems Technical Journal, 27, 379-423,623-656.
Sridharan, A., Lentmaier, M., Costello, D. J., & Zigangirov, K. (2004, Oct.). Convergence analysis of a class of LDPC convolutional codes for the erasure channel. In Proc. 42nd Allerton Conf. on Communications, Control, and Computing (p. 953-962).
