ACS_DEF1213

This course will introduce fundamental concepts and methods on detection and estimation theory for signal processing and linear systems in the presence of stochastic disturbances. Students attending the course will be able to formulate and solve problems such as detection of event occurrences, extracting relevant information about the event, parameter estimation, system state estimation, sensor fusion, and dynamic smoothing. The analysis of the solutions obtained will be addressed based on concepts discussed along the course. A number of applications to several domains will be used to illustrate the main concepts.

Summary:

Motivation for detection, estimation and filtering in a stochastic setting; Random processes and linear systems; Estimation theory; Characteristics of estimators; Cramer-Rao lower bound; Linear systems in the presence of stochastic signals; Best linear unbiased estimators; Maximum likelihood estimation; Deterministic and stochastic least squares; Bayesian estimation; Wiener filtering; Kalman filtering; Detection theory: Receiver Operating Characteristics (ROC); Bayes risk; Minimum probability of error; Multiple hypothesis testing; Neyman-Pearson Theorem; Multiple Model Adaptive Estimation; Optimal Smoothing.

Pre-requisites:

It will be assumed that the students had some prior exposure to:

1. Random Variables and Stochastic Processes

2. Linear Algebra

3. Linear Systems Theory

4. The solutions to some problems must be implemented in MATLAB/SIMULINK.

Syllabus:

Motivation for estimation in stochastic signal processing - Presentation of the course; Motivating examples of signals and systems in detection and estimation problems;

Characteristics of estimators – Unbiased estimators; Minimum Variance Criterium; Extension to vector parameters; Efficiency of estimators;

Cramer-Rao lower bound – Estimator accuracy; Cramer-Rao lower bound (CRLB); CRLB for signals in white Gaussian noise;

Linear models in the presence of stochastic signals – Stationary and transient analysis; White Gaussian noise and linear systems;

Best linear unbiased estimators (BLUE) – definition of best linear unbiased transient analysis; White Gaussian noise and bandlimited systems; estimators (BLUE); Minimum variance unbiased estimation;

Maximum likelihood estimation – maximum likelihood estimator; Properties of the ML estimators; Solution for ML estimation; Monte-Carlo methods;

Deterministic and stochastic least squares – The least squares approach; Linear and nonlinear least squares; Geometric interpretation; Constrained least squares;

Bayesian estimation¹ – Philosophy and estimator design; Prior knowledge; Bayesian linear model; Bayesian estimation on the presence of Gaussian pdfs; Minimum Mean Square Estimators;

Kalman filtering¹ – Dynamic signal and systems models; Optimal estimation solution in the presence of white Gaussian noise – the Kalman filter; stability, convergence and robustness for linear time-varying and time-invariant Kalman filters; Kalman and Wiener filters;

Multiple Model Adaptive Estimation¹ – Joint system identification and parameter/state estimation using multiple models.

Optimal Smoothing¹ – Fixed-point, fixed interval, and fixed lag smoothers. Examples of application.

Advanced Topics - To be detailed later, e.g. Positioning and navigation systems; Failure detection and isolation; Multiple model adaptive estimation; Discretization; Missing data estimation; Outlier detection and removal; Feature based estimation; Principal component analysis; Nonlinear signal processing; Compressive sensing;

______________________

1 - notes handed to the students.

Grading Policy:

Five problem sets will be solved along the semester, covering the main topics of the course. A term paper in a topic jointly selected by the student and the faculty will be completed in the final 4 weeks. Each component will correspond to 50% of the final grade. Late problem sets will be strongly penalized.

First lecture:
Tuesday, March 12th, 16h00, Room C11

Problem Sets:

	Start date	Due Date
PS#1	March 18th	March 29th
PS#2	April 4th	April 18th
PS#3	May 1st	May 13th
PS#45 Data	June 11th	June 30th

Term Projects, July 26th , Room tbd

Bibliography

Main text books

• Steven M. Kay, Fundamentals of Statistical Signal Processing: Estimation Theory, Vol. I, Prentice Hall Signal Processing Series, 1993.

• A. Gelb, Applied Optimal Estimation, MIT Press, 1974.

Complementary references

• Harry L. Van Trees, Detection, Estimation, and Modulation Theory, Part I, John Wiley, 2001.

• Athanasios Papoulis and S. Unnikrishna Pillai, Probability, Random Variables and Stochastic Processes, McGraw Hill, 2001.

• Robert Brown and Patrick Hwang, Introduction to Random Signals and Applied Kalman Filtering, John Wiley, 1997.

• Gonzalo Arce, Nonlinear Signal Processing: A Statistical Approach, John Wiley, 2005.

Return to top