Advanced digital signal processing and noise reduction / Saeed V. Vaseghi.

Format:

Book

Author/Creator:

Vaseghi, Saeed V.

Contributor:

Sabin W. Colton, Jr., Memorial Fund.

Language:

English

Subjects (All):

Signal processing.

Electronic noise.

Digital filters (Mathematics).

Physical Description:

xxvi, 453 pages : illustrations ; 26 cm

Edition:

Third edition.

Place of Publication:

Chichester, England ; Hoboken, NJ : Wiley, [2006]

Summary:

Signal processing plays an increasingly central role in the development of modern telecommunication and information processing systems, with a wide range of applications in areas such as multimedia technology, audio-visual signal processing, cellular mobile communication, radar systems and financial data forecasting. The theory and application of signal processing deals with the identification, modelling and utilisation of patterns and structures in a signal process. The observation signals are often distorted, incomplete and noisy and hence, noise reduction and the removal of channel distortion is an important part of a signal processing system.

Advanced Digital Signal Processing and Noise Reduction, Third Edition, provides a fully updated and structured presentation of the theory and applications of statistical signal processing and noise reduction methods. Noise is the eternal bane of communications engineers, who are always striving to find new ways to improve the signal-to-noise ratio in communications systems and this resource will help them with this task.

Features two new chapters on Noise and Distortion and Noise in Wireless Communications. Topics discussed include: probability theory, Bayesian estimation and classification, hidden Markov models, adaptive filters, multi-band linear prediction, spectral estimation, and impulsive and transient noise removal. Explores practical solutions to interpolation of missing signals, echo cancellation, impulsive and transient noise removal, channel equalisation, HMM-based signal and noise decomposition. This is an invaluable text for senior undergraduates, postgraduates and researchers in the fields of digital signal processing, telecommunications and statistical data analysis. It will also appeal to engineers in telecommunications and audio and signal processing industries.

Contents:

1.1 Signals and Information 1

1.2 Signal Processing Methods 3

1.2.1 Transform-based Signal Processing 3

1.2.2 Model-based Signal Processing 4

1.2.3 Bayesian Signal Processing 4

1.2.4 Neural Networks 5

1.3 Applications of Digital Signal Processing 5

1.3.1 Adaptive Noise Cancellation 5

1.3.2 Adaptive Noise Reduction 6

1.3.3 Blind Channel Equalisation 7

1.3.4 Signal Classification and Pattern Recognition 8

1.3.5 Linear Prediction Modelling of Speech 9

1.3.6 Digital Coding of Audio Signals 10

1.3.7 Detection of Signals in Noise 12

1.3.8 Directional Reception of Waves: Beam-forming 13

1.3.9 Dolby Noise Reduction 15

1.3.10 Radar Signal Processing: Doppler Frequency Shift 15

1.4 Sampling and Analogue-to-digital Conversion 17

1.4.1 Sampling and Reconstruction of Analogue Signals 18

1.4.2 Quantisation 19

2 Noise and Distortion 23

2.2 White Noise 25

2.2.1 Band-limited White Noise 26

2.3 Coloured Noise 26

2.4 Impulsive Noise 27

2.5 Transient Noise Pulses 29

2.6 Thermal Noise 30

2.7 Shot Noise 31

2.8 Electromagnetic Noise 31

2.9 Channel Distortions 32

2.10 Echo and Multipath Reflections 33

2.11 Modelling Noise 33

2.11.1 Additive White Gaussian Noise Model 36

2.11.2 Hidden Markov Model for Noise 36

3 Probability and Information Models 39

3.2 Random Signals 41

3.2.1 Random and Stochastic Processes 43

3.2.2 The Space of a Random Process 43

3.3 Probability Models 44

3.3.1 Probability and Random Variables 45

3.3.2 Probability Mass Function 45

3.3.3 Probability Density Function 47

3.3.4 Probability Dgnsity Functions of Random Processes 48

3.4 Information Models 50

3.4.1 Entropy 51

3.4.2 Mutual Information 54

3.4.3 Entropy Coding 56

3.5 Stationary and Nonstationary Random Processes 59

3.5.1 Strict-sense Stationary Processes 61

3.5.2 Wide-sense Stationary Processes 61

3.5.3 Nonstationary Processes 62

3.6 Statistics (Expected Values) of a Random Process 62

3.6.1 The Mean Value 63

3.6.2 Autocorrelation 63

3.6.3 Autocovariance 66

3.6.4 Power Spectral Density 66

3.6.5 Joint Statistical Averages of Two Random Processes 68

3.6.6 Cross-correlation and Cross-covariance 68

3.6.7 Cross-power Spectral Density and Coherence 70

3.6.8 Ergodic Processes and Time-averaged Statistics 70

3.6.9 Mean-ergodic Processes 70

3.6.10 Correlation-ergodic Processes 72

3.7 Some Useful Classes of Random Processes 73

3.7.1 Gaussian (Normal) Process 73

3.7.2 Multivariate Gaussian Process 74

3.7:3 Mixture Gaussian Process 75

3.7.4 A Binary-state Gaussian Process 76

3.7.5 Poisson Process 77

3.7.6 Shot Noise 78

3.7.7 Poisson-Gaussian Model for Clutters and Impulsive Noise 79

3.7.8 Markov Processes 80

3.7.9 Markov Chain Processes 81

3.7.10 Gamma Probability Distribution 82

3.7.11 Rayleigh Probability Distribution 83

3.7.12 Laplacian Probability Distribution 83

3.8 Transformation of a Random Process 83

3.8.1 Monotonic Transformation of Random Processes 84

3.8.2 Many-to-one Mapping of Random Signals 86

4 Bayesian Inference 93

4.1 Bayesian Estimation Theory: Basic Definitions 94

4.1.1 Dynamic and Probability Models in Estimation 95

4.1.2 Parameter Space and Signal Space 96

4.1.3 Parameter Estimation and Signal Restoration 97

4.1.4 Performance Measures and Desirable Properties of Estimators 98

4.1.5 Prior and Posterior Spaces and Distributions 100

4.2 Bayesian Estimation 102

4.2.1 Maximum a Posteriori Estimation 103

4.2.2 Maximum-likelihood Estimation 104

4.2.3 Minimum Mean Square Error Estimation 107

4.2.4 Minimum Mean Absolute Value of Error Estimation 108

4.2.5 Equivalence of the MAP, ML, MMSE and MAVE for Gaussian Processes with Uniform Distributed Parameters 109

4.2.6 The Influence of the Prior on Estimation Bias and Variance 109

4.2.7 The Relative Importance of the Prior and the Observation 114

4.3 The Estimate-Maximise Method 116

4.3.1 Convergence of the EM Algorithm 117

4.4 Cramer-Rao Bound on the Minimum Estimator Variance 119

4.4.1 Cramer-Rao Bound for Random Parameters 120

4.4.2 Cramer-Rao Bound for a Vector Parameter 121

4.5 Design of Gaussian Mixture Models 121

4.5.1 EM Estimation of Gaussian Mixture Model 122

4.6 Bayesian Classification 124

4.6.1 Binary Classification 125

4.6.2 Classification Error 127

4.6.3 Bayesian Classification of Discrete-valued Parameters 128

4.6.4 Maximum a Posteriori Classification 128

4.6.5 Maximum-likelihood Classification 129

4.6.6 Minimum Mean Square Error Classification 129

4.6.7 Bayesian Classification of Finite State Processes 130

4.6.8 Bayesian Estimation of the Most Likely State Sequence 131

4.7 Modelling the Space of a Random Process 132

4.7.1 Vector Quantisation of a Random Process 132

4.7.2 Vector Quantisation using Gaussian Models 133

4.7.3 Design of a Vector Quantiser: K-means Clustering 133

5 Hidden Markov Models 137

5.1 Statistical Models for Nonstationary Processes 138

5.2 Hidden Markov Models 139

5.2.1 Comparison of Markov and Hidden Markov Models 139

5.2.2 A Physical Interpretation: HMMs of Speech 141

5.2.3 Hidden Markov Model as a Bayesian Model 142

5.2.4 Parameters of a Hidden Markov Model 143

5.2.5 State Observation Probability Models 143

5.2.6 State Transition Probabilities 144

5.2.7 State-Time Trellis Diagram 145

5.3 Training Hidden Markov Models 145

5.3.1 Forward-Backward Probability Computation 147

5.3.2 Baum-Welch Model Re-estimation 148

5.3.3 Training HMMs with Discrete Density Observation Models 149

5.3.4 HMMs with Continuous Density Observation Models 150

5.3.5 HMMs with Gaussian Mixture pdfs 151

5.4 Decoding of Signals using Hidden Markov Models 152

5.4.1 Viterbi Decoding Algorithm 154

5.5 HMMs in DNA and Protein Sequence Modelling 155

5.6 HMMs for Modelling Speech and Noise 156

5.6.1 Modelling Speech with HMMs 156

5.6.2 HMM-based Estimation of Signals in Noise 156

5.6.3 Signal and Noise Model Combination and Decomposition 158

5.6.4 Hidden Markov Model Combination 159

5.6.5 Decomposition of State Sequences of Signal and Noise 160

5.6.6 HMM-based Wiener Filters 160

5.6.7 Modelling Noise Characteristics 162

6 Least Square Error Filters 165

6.1 Least Square Error Estimation: Wiener Filters 166

6.2 Block-data Formulation of the Wiener Filter 170

6.2.1 QR Decomposition of the Least Square Error Equation 171

6.3 Interpretation of Wiener Filters as Projections in Vector Space 172

6.4 Analysis of the Least Mean Square Error Signal 174

6.5 Formulation of Wiener Filters in the Frequency Domain 175

6.6 Some Applications of Wiener Filters 177

6.6.1 Wiener Filters for Additive Noise Reduction 177

6.6.2 Wiener Filters and Separability of Signal and Noise 178

6.6.3 The Square-root Wiener Filter 179

6.6.4 Wiener Channel Equaliser 180

6.6.5 Time-alignment of Signals in Multichannel/Multisensor Systems 181

6.7 Implementation of Wiener Filters 182

6.7.1 The Choice of Wiener Filter Order 183

6.7.2 Improvements to Wiener Filters 184

7 Adaptive Filters 187

7.2 State-space Kalman Filters 188

7.2.1 Derivation of the Kalman Filter Algorithm 190

7.3 Sample-adaptive Filters 195

7.4 Recursive Least Square Adaptive Filters 196

7.4.1 The Matrix Inversion Lemma 198

7.4.2 Recursive Time-update of Filter Coefficients 199

7.5 The Steepest-descent Method 201

7.5.1 Convergence Rate 203

7.5.2 Vector-valued Adaptation Step Size 204

7.6 The LMS Filter 204

7.6.1 Leaky LMS Algorithm 205

7.6.2 Normalised LMS Algorithm 206

8 Linear Prediction Models 209

8.1 Linear Prediction Coding 210

8.1.1 Frequency Response of LP Models 213

8.1.2 Calculation of Predictor Coefficients 214

8.1.3 Effect of Estimation of Correlation Function on LP Model

Solution 216

8.1.4 The Inverse Filter: Spectral Whitening 216

8.1.5 The Prediction Error Signal 217

8.2 Forward, Backward and Lattice Predictors 219

8.2.1 Augmented Equations for Forward and Backward Predictors 220

8.2.2 Levinson-Durbin Recursive Solution 221

8.2.3 Lattice Predictors 223

8.2.4 Alternative Formulations of Least Square Error Prediction 224

8.2.5 Predictor Model Order Selection 225

8.3 Short- and Long-term Predictors 226

8.4 MAP Estimation of Predictor Coefficients 228

8.4.1 Probability Density Function of Predictor Output 229

8.4.2 Using the Prior pdf of the Predictor Coefficients 230

8.5 Formant-tracking LP Models 230

8.6 Sub-band Linear Prediction Model 232

8.7 Signal Restoration using Linear Prediction Models 233

8.7.1 Frequency-domain Signal Restoration using Prediction Models 235

8.7.2 Implementation of Sub-band Linear Prediction Wiener Filters 237

9 Power Spectrum and Correlation 241

9.1 Power Spectrum and Correlation 242

9.2 Fourier Series: Representation of Periodic Signals 243

9.3 Fourier Transform: Representation of Aperiodic Signals 245

9.3.1 Discrete Fourier Transform 246

9.3.2 Time/Frequency Resolutions, the Uncertainty Principle 247

9.3.3 Energy-spectral Density and Power-spectral Density 248

9.4 Nonparametric Power Spectrum Estimation 249

9.4.1 The Mean and Variance of Periodograms 250

9.4.2 Averaging Periodograms (Bartlett Method) 250

9.4.3 Welch Method: Averaging Periodograms from Overlapped and Windowed Segments 251

9.4.4 Blackman-Tukey Method 252

9.4.5 Power Spectrum Estimation from Autocorrelation of Overlapped Segments 253

9.5 Model-based Power Spectrum Estimation 254

9.5.1 Maximum-entropy Spectral Estimation 255

9.5.2 Autoregressive Power Spectrum Estimation 257

9.5.3 Moving-average Power Spectrum Estimation 257

9.5.4 Autoregressive Moving-average Power Spectrum Estimation 258

9.6 High-resolution Spectral Estimation Based on Subspace Eigenanalysis 259

9.6.1 Pisarenko Harmonic Decomposition 259

9.6.2 Multiple Signal Classification Spectral Estimation 261

9.6.3 Estimation of Signal Parameters via Rotational Invariance Techniques 264

10 Interpolation 267

10.1.1 Interpolation of a Sampled Signal 268

10.1.2 Digital Interpolation by a Factor of I 269

10.1.3 Interpolation of a Sequence of Lost Samples 271

10.1.4 The Factors that affect Interpolation Accuracy 273

10.2 Polynomial Interpolation 274

10.2.1 Lagrange Polynomial Interpolation 275

10.2.2 Newton Polynomial Interpolation 276

10.2.3 Hermite Polynomial Interpolation 278

10.2.4 Cubic Spline Interpolation 278

10.3 Model-based Interpolation 280

10.3.1 Maximum a Posteriori Interpolation 281

10.3.2 Least Square Error Autoregressive Interpolation 282

10.3.3 Interpolation based on a Short-term Prediction Model 283

10.3.4 Interpolation based on Long- and Short-term Correlations 286

10.3.5 LSAR Interpolation Error 289

10.3.6 Interpolation in Frequency-Time Domain 290

10.3.7 Interpolation using Adaptive Codebooks 293

10.3.8 Interpolation through Signal Substitution 294

11 Spectral Amplitude Estimation 297

11.1.1 Spectral Representation of Noisy Signals 299

11.1.2 Vector Representation of the Spectrum of Noisy Signals 299

11.2 Spectral Subtraction 300

11.2.1 Power Spectrum Subtraction 302

11.2.2 Magnitude Spectrum Subtraction 303

11.2.3 Spectral Subtraction Filter: Relation to Wiener Filters 303

11.2.4 Processing Distortions 304

11.2.5 Effect of Spectral Subtraction on Signal Distribution 305

11.2.6 Reducing the Noise Variance 306

11.2.7 Filtering Out the Processing Distortions 307

11.2.8 Nonlinear Spectra] Subtraction 308

11.2.9 Implementation of Spectral Subtraction 310

11.3 Bayesian MMSE Spectral Amplitude Estimation 312

11.4 Application to Speech Restoration and Recognition 315

12 Impulsive Noise 319

12.1 Impulsive Noise 320

12.1.1 Autocorrelation and Power Spectrum of Impulsive Noise 322

12.2 Statistical Models for Impulsive Noise 323

12.2.1 Bernoulli-Gaussian Model of Impulsive Noise 324

12.2.2 Poisson-Gaussian Model of Impulsive Noise 324

12.2.3 A Binary-state Model of Impulsive Noise 325

12.2.4 Signal-to-impulsive-noise Ratio 326

12.3 Median Filters 327

12.4 Impulsive Noise Removal using Linear Prediction Models 328

12.4.1 Impulsive Noise Detection 328

12.4.2 Analysis of Improvement in Noise Detectability 330

12.4.3 Two-sided Predictor for Impulsive Noise Detection 331

12.4.4 Interpolation of Discarded Samples 332

12.5 Robust Parameter Estimation 333

12.6 Restoration of Archived Gramophone Records 334

13 Transient Noise Pulses 337

13.1 Transient Noise Waveforms 337

13.2 Transient Noise Pulse Models 339

13.2.1 Noise Pulse Templates 340

13.2.2 Autoregressive Model of Transient Noise Pulses 341

13.2.3 Hidden Markov Model of a Noise Pulse Process 342

13.3 Detection of Noise Pulses 342

13.3.1 Matched Filter for Noise Pulse Detection 343

13.3.2 Noise Detection based on Inverse Filtering 344

13.3.3 Noise Detection based on HMM 344

13.4 Removal of Noise Pulse Distortions 345

13.4.1 Adaptive Subtraction of Noise Pulses 345

13.4.2 AR-based Restoration of Signals Distorted by Noise Pulses 347

14 Echo Cancellation 351

14.1 Introduction: Acoustic and Hybrid Echoes 352

14.2 Telephone Line Hybrid Echo 353

14.2.1 Echo: the Sources of Delay in Telephone Networks 354

14.2.2 Echo Return Loss 355

14.3 Hybrid Echo Suppression 355

14.4 Adaptive Echo Cancellation 356

14.4.1 Echo Canceller Adaptation Methods 357

14.4.2 Convergence of Line Echo Canceller 358

14.4.3 Echo Cancellation for Digital Data Transmission 359

14.5 Acoustic Echo 360

14.6 Sub-band Acoustic Echo Cancellation 363

14.7 Multiple-input Multiple-output Echo Cancellation 365

14.7.1 Stereophonic Echo Cancellation Systems 365

15 Channel Equalisation and Blind Deconvolution 371

15.1.1 The Ideal Inverse Channel Filter 373

15.1.2 Equalisation Error, Convolutional Noise 374

15.1.3 Blind Equalisation 374

15.1.4 Minimum- and Maximum-phase Channels 376

15.1.5 Wiener Equaliser 377

15.2 Blind Equalisation using the Channel Input Power Spectrum 379

15.2.1 Homomorphic Equalisation 380

15.2.2 Homomorphic Equalisation using a Bank of High-pass Filters 382

15.3 Equalisation based on Linear Prediction Models 382

15.3.1 Blind Equalisation through Model Factorisation 384

15.4 Bayesian Blind Deconvolution and Equalisation 385

15.4.1 Conditional Mean Channel Estimation 386

15.4.2 Maximum-likelihood Channel Estimation 386

15.4.3 Maximum a Posteriori Channel Estimation 386

15.4.4 Channel Equalisation based on Hidden Markov Models 387

15.4.5 MAP Channel Estimate based on HMMs 389

15.4.6 Implementations of HMM-based Deconvolution 390

15.5 Blind Equalisation for Digital Communications Channels 393

15.5.1 LMS Blind Equalisation 395

15.5.2 Equalisation of a Binary Digital Channel 397

15.6 Equalisation based on Higher-order Statistics 398

15.6.1 Higher-order Moments, Cumulants and Spectra 399

15.6.2 Higher-order Spectra of Linear Time-invariant Systems 401

15.6.3 Blind Equalisation based on Higher-order Cepstra 402

16 Speech Enhancement in Noise 409

16.2 Single-input Speech-enhancement Methods 411

16.2.1 An Overview of a Speech-enhancement System 411

16.2.2 Wiener Filter for De-noising Speech 414

16.2.3 Spectra] Subtraction of Noise 417

16.2.4 Bayesian MMSE Speech Enhancement 418

16.2.5 Kalman Filter 419

16.2.6 Speech Enhancement via LP Model Reconstruction 422

16.3 Multiple-input Speech-enhancement Methods 425

16.3.1 Beam-forming with Microphone Arrays 427

16.4 Speech Distortion Measurements 430

17 Noise in Wireless Communications 433

17.2 Noise, Capacity and Spectral Efficiency 436

17.3 Communications Signal Processing in Mobile Systems 438

17.4 Noise and Distortion in Mobile Communications Systems 439

17.4.1 Multipath Propagation of Electromagnetic Signals 440

17.4.2 Rake Receivers for Multipath Signals 441

17.4.3 Signal Fading in Mobile Communications Systems 442

17.4.4 Large-scale Signal Fading 443

17.4.5 Small-scale Fast Signal Fading 444

17.5 Smart Antennas 444

17.5.1 Switched and Adaptive Smart Antennas 446

17.5.2 Space-Time Signal Processing -

Diversity Schemes 446.

Notes:

Includes bibliographical references and index.

Local Notes:

Acquired for the Penn Libraries with assistance from the Sabin W. Colton, Jr., Memorial Fund.

ISBN:

047009494X

OCLC:

60791649

Publisher Number:

9780470094945 (hbk.)