{"title": "Neural Net Receivers in Multiple Access-Communications", "book": "Advances in Neural Information Processing Systems", "page_first": 272, "page_last": 280, "abstract": null, "full_text": "272 \n\nNEURAL NET RECEIVERS IN \n\nMULTIPLE-ACCESS COMMUNICATIONS \n\nBernd-Peter Paris, Geoffrey Orsak, Mahesh Varanasi, Behnaam Aazhang \n\nDepartment of Electrical and Computer Engineering \n\nRice University \n\nHouston, TX 77251-1892 \nABSTRACT \n\nThe application of neural networks to the demodulation of \nspread-spectrum signals in a multiple-access environment is \nconsidered. This study is motivated in large part by the fact \nthat, in a multiuser system, the conventional (matched fil(cid:173)\nter) receiver suffers severe performance degradation as the \nrelative powers of the interfering signals become large (the \n\"near-far\" problem). Furthermore, the optimum receiver, \nwhich alleviates the near-far problem, is too complex to be \nof practical use. Receivers based on multi-layer perceptrons \nare considered as a simple and robust alternative to the opti(cid:173)\nmum solution. The optimum receiver is used to benchmark \nthe performance of the neural net receiver; in particular, it is \nproven to be instrumental in identifying the decision regions \nof the neural networks. The back-propagation algorithm and \na modified version of it are used to train the neural net. An \nimportance sampling technique is introduced to reduce the \nnumber of simulations necessary to evaluate the performance \nof neural nets. In all examples considered the proposed neu(cid:173)\nral ~et receiver significantly outperforms the conventional \nrecelver. \n\nINTRODUCTION \n\nIn this paper we consider the problem of demodulating signals in a code-division \nmultiple-access (CDMA) Gaussian channel. Multiple accessing in code domain is \nachieved by spreading the spectrum of the transmitted signals using preassigned \ncode waveforms. The conventional method of demodulating a spread-spectrum sig(cid:173)\nnal in a multiuser environment employs one filter matched to the desired signal. \nSince the conventional receiver ignores the presence of interfering signals it is reli(cid:173)\nable only when there are few simultaneous transmissions. Furthermore, when the \nrelative received power of the interfering signals become large (the \"near-far\" prob(cid:173)\nlem), severe performance degradation of the system is observed even in situations \nwith relatively low bandwidth efficiencies (defined as the ratio of the number of \nchannel subscribers to the spread of the bandwidth) [Aazhang 87]. For this reason \nthere has been an interest in designing optimum receivers for multi-user communica(cid:173)\ntion systems [Verdu 86, Lupas 89, Poor 88]. The resulting optimum demodulators, \n\n\fNeural Net Receivers in Multiple-Access Communications \n\n273 \n\nhowever, have a variable decoding delay with computational and storage complexity \nthat depend exponentially on the number of active users. Unfortunately, this com(cid:173)\nputational intensity is unacceptable in many applications. There is hence a need \nfor near optimum receivers that are robust to near-far effects with a reasonable \ncomputational complexity to ensure their practical implementation. \n\nIn this study, we introduce a class of neural net receivers that are based on mul(cid:173)\n\ntilayer perceptrons trained via the back-propagation algorithm. Neural net receivers \nare very attractive alternatives to the optimum and conventional receivers due to \ntheir highly parallel structures. As we will observe, the performance of the neural \nnet receivers closely track that of the optimum receiver in all examples considered. \n\nSYSTEM DESCRIPTION \n\nIn the multiple-access network of interest, transmitters are assumed to share a radio \nband in a combination of the time and code domain. One way of multiple accessing \nin the code domain is spread spectrum, which is a signaling scheme that uses a much \nwider bandwidth than necessary for a given data rate. Let us assume that in a given \ntime interval there are K active transmitters in the network. In a simple setting, \nthe kth active user, in a symbol interval, transmits a signal from a binary signal \nset derived from the set of code waveforms assigned to the corresponding user. The \nsignal is time limited to the interval [a, T], where T is the symbol duration. \n\nIn this paper we will concentrate on symbol-synchronous CDMA systems. Syn(cid:173)\n\nchronous systems find applications in time slotted channels with the central (base) \nstation transmitting to remote (mobile) terminals and also in relays between cen(cid:173)\ntral stations. The synchronous problem will also be construed as providing us with \na manageable setting to better understand the issues in the more difficult asyn(cid:173)\nchronous situation. In a synchronous CDMA system, the users maintain time syn(cid:173)\nchronism so that the relative time delays associated with all users are assumed to be \nzero. To illustrate the potentials of the proposed multiuser detector, we present the \napplication to binary PSK direct-sequence signals in coherent systems. Therefore, \nthe signal at a given receiver is the superposition of the K transmitted signals in \nadditive channel noise (see [Aazhang 87, Lupas 89] and references within) \n\nP K \n\nret) = L L b~i) Akak(t - iT) cos(we[t - iT] + Ok) + nt, \n\nt E ~, \n\n(1) \n\ni=1 k=1 \n\nwhere P is the packet length, Ak is the signal amplitude, We is the carrier frequency, \nOk is the phase angle. The symbol b1i) E {-I, + I} denotes the bit that the kth user is \ntransmitting in the ith time interval. In this model, nt is the additive channel noise \nwhich is assumed to be a white Gaussian random process. The time-limited code \nwaveform, denoted.by ak(t), is derived from the spreading sequence assigned to the \nkth user. That is, ak(t) = Ef=-~/ a)k)p(t - jTe) where pet) is the unit rectangular \npulse of duration Te and N is the length of the spreading sequence. One code \nperiod !!(k) = [a~k),a~k), . . . ,a~~I] is used for spreading the signal per symbol so \n\n\f274 \n\nParis, Orsak, Varanasi and Aazhang \n\nthat T = NTc \u2022 In this system, spectrum efficiency is measured as the ratio of the \nnumber of channel users to the spread factor, K/ N. \n\nIn the next two sections, we first consider optimum synchronous demodulation of \nthe multiuser spread-spectrum signal. Then, we introduce the application of neural \nnetworks to the multiuser detection problem. \n\nOPTIMUM RECEIVER \n\nMultiuser detection is an active research area with the objective of developing strate(cid:173)\ngies for demodulation of information sent by several transmitters sharing a channel \n[Verdu 86, Poor 88, Varanasi 89, Lupas 89]. In these situations with two or more \nusers of a multiple-access Gaussian channel, one filter matched to the desired signal \nis no longer optimum since the decision statistics are effected by the other signals \n(e.g., the statistics are disturbed by cross-correlations with the interfering signals). \nEmploying conventional matched filters, because of its structural simplicity, may \nstill be justified if the system is operating at a low bandwidth efficiency. However, \nas the number of users in the system with fixed bandwidth grows or as the rel(cid:173)\native received powers of the interfering signals become large, severe performance \ndegradation of the conventional matched filter is observed [Aazhang 87]. For direct(cid:173)\nsequence spread-spectrum systems, optimum receivers obtained by Verdu and Poor \nrequire an extremely high degree of software complexity and storage, which may be \nunacceptable for most multiple-access systems [Verdu 86, Lupas 89]. Despite imple(cid:173)\nmentation problems, studies on optimum demodulation illustrate that the effects of \ninterfering signals in a CDMA system, in principle, can be neutralized. \n\nA complete study of the suboptimum neural net receiver requires a review \nof the maximum likelihood sequence detection formulation. Assuming that all \npossible information sequences are independent and equally likely, and defining \n!L{ i) = [b~i), b~i), ... , b}2]', it is easy to see that an optimum decision on fL{ i) is a \none-shot decision in that it requires the observation of the received signal only in \nthe ith time interval. Without loss of generality, we will therefore focus our attention \non i = 0 and drop the time superscript and consider the demodulation of the vector \nof bits !L with the observation of the received signal in the interval [0,11-\n\nIn a K -user Gaussian channel, the most likely information vector is chosen as \n\nthat which maximizes the log of the likelihood function (see [Lupas 89]) \n\nwhere Sk(t) = Akak(t) cos(wct + Ok) is the modulating signal of the kth user. The \noptimum decision can also be written as \n~pt = arg max \n\n{2y'IL - !L'HIL} , \n\n(3) \n\nte{ _l,+l}K \n\n-\n\nwhere H is the K x K matrix of signal cross-correlations such that the (k,l)th \nelement is hk,r =< Sk(t), Sr(t) >. The vector of sufficient statistics '[ consists of the \n\n\fNeural Net Receivers in Multiple-Access Communications \n\n275 \n\noutputs of a bank of J{ filters each matched to one of the signals \n\nYk = iT r(t)Sit;(t)dt, for k = 1,2, ... ,K. \n\n(4) \n\nThe maximization in (3) has been shown to be NP-complete [Lupas 89], i.e., no \nalgorithm is known that can solve the maximization problem in polynomial time in \nK. This computational intensity is unacceptable in many applications. In the next \nsection, we consider a suboptimum receiver that employs artificial neural networks \nfor finding a solution to a maximization problem similar to (3). \n\nNEURAL NETWORK \n\nUntil now the application of neural networ,ks to multiple-access communications has \nnot drawn much attention. In this study we employ neural networks for classifying \ndifferent signals in synchronous additive Gaussian channels. We assume that the \ninformation bits of the first of the K signals is of interest, therefore, the phase \nangle of the desired signal is assumed to be zero (i.e., (}1 = 0). Two configurations \nwith multi-layer perceptrons and sigmoid nonlinearity are considered for multiuser \ndetection of direct-sequence spread-spectrum signals. \n\nOne structure is depicted in Figure 1.b where a layered network of percep(cid:173)\n\ntrons processes the sufficient statistics (4) of the multi-user Gaussian channel. In \nthis structure the first layer of the net (referred to as the hidden layer) processes \n[Y1, Y2, ... , YK]. The output layer may only have one node since there is only one \nsignal that is being demodulated. This feed-forward structure is then trained using \nthe back-propagation algorithm [Rumelhart 86]. \n\nIn an alternate configuration, the continuous-time received signal is converted to \nan N-dimensional vector by sampling the output of the front-end filter at the chip \nrate Te- 1 as illustrated in Figure 1.a. The input vector to the net can be written so \nthat the demodulation of the first signal is viewed as a classification problem: \n\n(5) \n\nwhere \u00a31(1) is the spreading code vector of the first user, 1] is a length-N vector \nof filtered Gaussian noise samples and L = E[=2 bkA~ COS(8k)!!(k) is the multiple(cid:173)\naccess interference vector with Ak = AkTel2, Vk = 1,2, ... ,K. The layered neural \nnet is then trained to process the input vector for demodulation of the first user's \ninformation bits via the back-propagation algorithm. For this configuration we \nconsider two training methods, first the multi-layer receiver is trained, via the back(cid:173)\npropagation algorithm, to classify the parity of the desired signal (referred to as the \n\"trained\" example) [Lippmann 87]. In another attempt (referred to as the \"preset\" \nexample), the input layer of the net is preset as Gaussian classifiers and the other \nlayers are trained using the back propagation algorithm [Gschwendtner 88]. \n\nSince we are interested in understanding the internal representation of knowledge \nby the weights of the net, a signal space method is developed to illustrate decision \nregions. In a K -user system where the spreading sequences are not orthogonal, the \n\n\f276 \n\nParis, Orsak, Varanasi and Aazhang \n\nsignals can be represented by orthonormal bases using the Gram-Schmidt procedure. \nThe optimum decision regions in the signal space for the demodulation of 61 are \nknown [Poor 88] and can be directly compared to ones for the neural net. Figure 2 \nillustrates decision regions for the optimum receiver and for \"preset\" and \"trained\" \nneural net receivers. In this example, two users are sharing a channel with N = 3, \nsignal to noise ratio of user 1 (SN Rd equal to 8dB and relative energies of the \ntwo user, E2/ E1 = 6dB. As it is seen in this figure the decision region of the \n\"preset\" example is almost identical to the optimum boundary, however, the decision \nboundary for the \"trained\" example is quite conservative. Such comparisons are \ninstrumental not only in identifying the pattern by which decisions are made by \nthe neural networks but also in understanding the characteristics of the training \nalgorithms. \n\nPERFORMANCE ANALYSIS \n\nIn this paper, we motivate the application of neural nets to single-user detection \nin multiuser channels by comparing the performance of the receivers in Figure 1 to \nthat of the conventional and the optimum [Poor 88]. Since exact analysis of the bit \nerror probabilities for the neural net receivers are analytically intractable, we con(cid:173)\nsider Monte Carlo simulations. This method can produce very accurate estimates \nof bit-error probability if the number of simulations is sufficiently large to ensure \noccurrence of several erroneous decisions. The fact that these multiuser receivers \noperate with near optimum error rates puts a tremendous computational burden on \nthe computer system. The new variance reduction scheme, developed by Orsak and \nAazhang in [Orsak 89], first shifts the simulated channel noise to bias the simula(cid:173)\ntions and then scales the error rate to obtain an unbiased estimate with a reduced \nvariance. This importance sampling technique, which proved to be extremely effec(cid:173)\ntive in single-user detection [Orsak 89], is applied to the analysis of the multiuser \nsystems. \n\nAs discussed in [Orsak 89], the fundamental issue is to generate more errors by \nbiasing the simulations in cases where the error rate is very small. This strategy \nis better described by the two-user Gaussian example in Figure 2. In this example \nthe simulation is carried out by generating zero-mean Gaussian noise vectors 'I} , \nrandom phase (}2 and random values of the interfering bit 62 . Considering 61 = 1. \n(corresponding to signals +a1 + a2 or +a1 - a2 which are marked by \"+\" in Figure \n2) error occurs if the statistics fall on the left side of the decision boundary. It can \nbe shown that the most efficient biasing scheme corresponds to a shift of the mean \nof the Gaussian noise and the multiple-access interference such that the mean of the \nstatistics are placed on the decision boundary (the shifted signals are marked by \n\"0\" in Figure 2). Since this strategy generates much more errors than the standard \nMonte Carlo, errors are weighted to obtain an unbiased estimate of the error rate. \nThe importance sampling technique substantially reduces the number of simulation \ntrials compared to standard Monte Carlo for a given accuracy. In Figure 3 the gain \nwhich is defined as the ratio of the number of trials required for a fixed variance \nusing Monte Carlo to that using the importance sampling method, is plotted versus \n\n\fNeural Net Receivers in Multiple-Access Communications \n\n277 \n\nthe bit-error probability. In this example, the spreading sequence length, N is equal \n3 and relative energies of the two user, E2/ El = 6dB. The gain in this example \nof severe near-far problem is inversely proportional to the error rate. Furthermore, \nresults from extensive analysis indicated that the proposed importance sampling \ntechnique is well suited for problems in multi-user communications and less than \n100 trials is sufficient for an accurate error probability estimate. \n\nNUMERICAL RESULTS \n\nThe performance of the conventional, optimum [Poor 88] and the neural net re(cid:173)\nceivers are compared via Monte Carlo simulations employing the importance sam(cid:173)\npling method. Except for a difference in length of training periods, the two configu(cid:173)\nrations in Figure 1 result in similar average bit-error probabilities. Results presented \nhere correspond to the neural net receiver in Figure l.a. \nA two-user Gaussian channel is considered with severe near-far problem where \nE2/ El = 6dB and spreading sequence length N = 3. In Figure 4, the average \nbit-error probabilities of the four receivers (conventional, optimum, neural nets for \nthe \"trained\" and \"preset\" examples) are plotted versus the signal to noise ratio of \nthe first user (SN RI). It is clear from this figure that the two neural net receivers \noutperform the matched filter receiver over the range of SN R l . Figure 5 depicts \nthese average error probabilities versus the relative energies of the two users (i.e., \nE2/ El ) for a fixed SN Rl = 8dB and N = 3. As expected the conventional receiver \nbecomes multiple-access limited as E2 increases, however, the performance of the \nneural net receivers closely track that of the optimum receiver for all values of E 2 \u2022 \nWe also considered a three-user Gaussian example with a high bandwidth effi(cid:173)\nciency and severe near-far problem where spreading sequence length N = 3 and first \nand third users have equal energy and second user has four times more energy (Le., \nE2/ El = 6dB ). The average error probabilities of the four receivers versus SN Rl \nare depicted in Figure 6. The neural net receivers maintained their near optimum \nperformance even in this three user example with a spread fae tor of 3 corresponding \nto a bandwidth efficiency of 1. \n\nCONCLUSIONS \n\nIn this paper, we consider the problem of demodulating a signal in a multiple(cid:173)\naccess Gaussian channel. The error probability of different neural net receivers were \ncompared with the conventional and optimum receivers in a symbol-synchronous \nsystem. As expected the performance of the conventional receiver (matched filter) is \nvery sensitive to the strength of the interfering users. However, the error probability \nof the neural net receiver is independent of the strength of the other users and is \nat least one order of magnitude better than the conventional receiver. Except for a \ndifference in the length of training periods, the two configurations in Figure 1 result \nin similar average bit-error probabilities. However, the training strategies, \"preset\" \nand \"trained\", resulted in slightly different error rates and decision regions. \n\nThe multi-layer perceptron was very successful in the classification problem in the \npresence of interfering signals. In all the examples that were considered, two layers \n\n\f278 \n\nParis, Orsak, Varanasi and Aazhang \n\nof perceptrons proved to be sufficient to closely approximate the decision boundary \nof the optimum receiver. We anticipate that this application of neural networks \nwill shed more light on the potentials of neural nets in digital communications. The \nissues facing the project were quite general in nature and are reported in many neural \nnetwork studies. However, we were able to address these issues in multiple-access \ncommunications since the disturbances are structured and the optimum receiver \n(which is NP-hard) is well understood. \n\nReferences \n\n[Aazhang 87] \n\nB. Aazhang and H. V. Poor. Performance of DS/SSMA Com(cid:173)\nmunications in Impulsive Channels-Part I: Linear Correlation \nReceivers. \nIEEE Trans. Commun., COM-35(1l):1l79-1188, \nNovember 1987. \n\n[Gschwendtner 88] A. B. Gschwendtner. DARPA Neural Network Study. AFCEA \n\nInternational Press, 1988. \n\n[Lippmann 87] \n\nR. P. Lippmann and B. Gold. Neural-Net Classifiers Useful for \nSpeech Recognition. In IEEE First Conference on Neural Net(cid:173)\nworks, pages 417-425, San Diego, CA, June 21-24, 1987. \n\n[Lupas 89] \n\n[Orsak 89] \n\n[Poor 88] \n\n[Rumelhart 86] \n\nR. Lupas and S. Verdu. Linear Multiuser Detectors for Syn(cid:173)\nchronous Code-Division Multiple-Access Channels. IEEE Trans. \nInfo. Theory, IT-34, 1989. \n\nG. Orsak and B. Aazhang. On the Theory of Importance Sam(cid:173)\npling Applied to the Analysis of Detection Systems. \nTrans. Commun., COM-37, April, 1989. \n\nIEEE \n\nH. V. Poor and S. Verdu. Single-User Detectors for Multiuser \nChannels. IEEE Trans. Commun., COM-36(1):50-60, January, \n1988. \n\nD. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning \nInternal Representation by Error Propagation. In D. E. Rumel(cid:173)\nhart and J. L. McClelland, editors, Parallel Distributed Pro(cid:173)\ncessing: Explorations in the Microstructure of Cognition. Vol. I: \nFoundations, pages 318-362, MIT Press, 1986. \n\n[Varanasi 89] \n\nM. K. Varanasi and B. Aazhang. Multistage Detection in \nAsynchronous Code-Division Multiple-Access Communications. \nIEEE Trans. Commun., COM-37, 1989. \n\n[Verdu 86] \n\nS. Verdu. Optimum Multiuser Asymptotic Efficiency. \nTrans. Commun., COM-34(9):890-897, September, 1986. \n\nIEEE \n\n\fNeural Net Receivers in Multiple-Access Communications \n\n279 \n\nSampler \n(n+l)T c \n\nret) \n\n~1 \n\n(a) \nFigure 1. Two Neural Net Receiver Structures. \n4,---------------~.------rr------~ \n\n(b) \n\n\u2022 \n: \n\u2022 \nMatched Filter .... \n\no \n\nNeural Net (preset) \n\nl , \n\nr' \n~ \n\nl ,. \" \n\n2 \n\no \n\n-2 \n\nl \n~' Optimum Receiver \nA- Neural Net (trained) \n\nI \n\nf \n\n\" \nI \n\nI \n: \nI : \n\nI \n\n.... \n\n- 4~----~~~~--~----~----~--~ \n\n-3 \n\n-2 \n\n-1 \n\no \n\n2 \n\n3 \n\nFigure 2. Decision Boundaries of the Various Receivers. \n\n1012~ ________________________________ ~ \n\n10 10 \n\n10 8 \n\n.~ 10 6 \nC!' \n\n10 4 \n\n10 2 \n\nOpt. Receiver \nNeural Net (preset) \nNeural Net (trained) \n\nMatched Filter \n\n10\u00b0 ~~--~~~~~~--~~--~~--~~ \n10 -1 \n\n10-13 \n\n10 -11 \n\n10-9 \n\n10 -7 \n\n10-5 \n\n10-3 \n\nProb. of Error \n\nFigure 3. Importance Sampling Gain versus Error Rate for 2-user Example. \n\n\f280 \n\nParis, Orsak, Varanasi and Aazhang \n\n1 r--::::~~::::=----I \n\nMatched Filter \nNeural Net (trained) \nNeural Net (preset) \n\n10-\n10-2 \n10-3 \n15 10-4 \n.. 10-5 \n~ 10-6 \n'Q 10-7 \n~ 10-8 \nQ 10-9 \n\"= 10-10 \nto-11 \nto-I \nto -13+-_...,.._--. __ ,--_...,.._--. __ ,--_ .... \n16 \n\nOpt Receiver \n\n14 \n\n6 \n\n4 \n\n2 \n\n8SNR l~ dB 12 \n\nFigure 4. Prob. of Error as a Function of the SNR (E2/El = 4). \n10-1 ~------------------------------~ \n\nMatched Filter \nNeural Net (trained) \nNeural Net (preset) \nOpt. Receiver \n\n104+---~--~--~--~--~--,---~~ \n4 \n\no \n\n2 \n\n1 \n\n3 \n\nE2/El \n\nNeural Net (trained) \n\nFigure 5. Influence of MA-Interference (SNR = 8dB). \n10-1 '---~~-'--~F.:~~==~====---, \n10 -2 \n10 -3 \n10-4 \n10 -5 \n10 -6 \n10 -7 \n10 -8 \n10 -9 \n10 -10 \n10 ~1l \n10 -12 \n10 -13-+-'I'\"'\"\"\"II\"\"\"\"I\"' ........ ~.......,--r' ........ ~......., __ -r-..,--.......,.--.--r--t \n16 \n\nNeural Net (preset) \nOpt. Receiver \n\n14 \n\n12 \n\n10 \n\n4 \n\n6 \n\n8 \n\n2 \n\nSNR in dB \n\nFigure 6. Error Curves for the 3-User Example. \n\n\f", "award": [], "sourceid": 186, "authors": [{"given_name": "Bernd-Peter", "family_name": "Paris", "institution": null}, {"given_name": "Geoffrey", "family_name": "Orsak", "institution": null}, {"given_name": "Mahesh", "family_name": "Varanasi", "institution": null}, {"given_name": "Behnaam", "family_name": "Aazhang", "institution": null}]}