BASIC STATISTICS AND CONVOLUTION

BASIC STATISTICS

MEDIAN FILTER

CONVOLUTION

skewness measures the asymmetry about the mean.

Kurtosis is the fourth moment about the mean.  It measures the peakedness/flatness of a distribution relative to a normal distribution.

 

Median Filters

‘Smoothing’ a series of data values, often a function of time, allows us to look at a data set and find some information about the data that indicates the general trends. Usually this is performed by applying a low-pass or band-pass filter using the process of convolution which we will discuss later. However, this ‘filtered’ information can be contaminated by ‘noise spikes’ or outliers. Before smoothing the data, we can remove ‘noise spikes’ by applying a median filter.

The median is the middle value of a set of numbers. A median filter picks the median of a set of numbers from a group of numbers specified by the window length. This idea can be used to remove noise spikes from a data set. Let

1, 4, 5, 3, 6, 4 ,52, 1, 3, 5, 4, 23, 5, 5, 3, 6, –100, 9, 6

be the measured data. The large values, e.g., 52, 23 and –100, are probably noise. We can remove these ‘outliers’ with a 3 point median filter.

1st window: 1, 4, 5 sort = 1, 4, 5 median = 4

2nd window 4, 5, 3 sort = 3, 4, 5, median = 4

3rd window 5, 3, 6 sort = 3, 5, 6 median = 5

etc.

__4, 4, 5, 4, 6, 4, 3, 3, 4, 4, 5, 5, 5, 5, 6, 9, 6 __ is the result. Note that The result is missing the two endpoints. A graph of both series would show that the result has no spikes. We can now find the mean values in the same or different windows for the data set.

You can implement a median filter by using a sort subroutine and passing to the subroutine the data to be filtered beginning at the start of each window in succession.

Assume that the subroutine SORT is given a vector X of Length NSORT and returns the sorted vector Y. The following code will work except for end points:

PARAMETER (NX = 100, NSORT = 3)

REAL X(NX), Y(NSORT), F(NX)

:

:

ISORT = NSORT/2

DO J=1, NX – NSORT + 1

CALL SORT (X(J), NSORT, Y)

F(J + ISORT) = Y (ISORT + 1)

END DO

The median filtered result is now in F.

Why is the DO LOOP terminated at NX – NSORT + 1?

What are the first and final values for F?

What happens if NSORT = 5?
 
 
 

Convolution:

 

Convolution is described by the mathematical equation:

or

h (t) = f (t) * g (t)

Note that the symbol (*) does not mean multiplication, it means convolution.

This operation can also be done after a Fourier transform, by taking the Fourier transform of f(t) and g(t), to obtain F(w) and G(w) and then multiplying F(w) and G(w) to get H(w). Then take the inverse Fourier transform of H(w) to get h(t). Note:

where e-iwt and eiwt are the kernals of the Fourier transform. We will come back to this later.

There are 3 ways to perform the convolution integral:

1) Let

Construct the following table:

	f0	f1	f2
g0	g0f0	g0f1	g0f2
g1	g1f0	g1f1	g1f2
g2	g2f0	g2f1	g2f2

You can find h(t) by summing along the diagonals from the lower left to the upper right, i.e.

h₀ = g₀f₀

h₁ = g₁f₀ + g₀f₁

h₂ = g₂f₀ + g₁f₁ + g₀f₂

h₃ = g₂f₁ + g₁f₂

h₄ = g₂f₂
 

2) Delay, multiply, and add.

 Begin, by flipping (reversing) one of the series. We assume that all undefined positions are zero.
 
 

			g0	g1	g2
f2	f1	f0

 a) Delay by 1, multiply, and add to get h0.
 

		g0	g1	g2		h0 = g0 f0
f2	f1	f0

 b) Delay by 1, multiply, and add to get h1.
 

	g0	g1	g2			h1 = g0 f1 + g1 f0
f2	f1	f0

c) Delay again by 1, multiply, and add to get h2.
 

g0	g1	g2				h2 = g0 f2 + g1 f1 + g2 f0
f2	f1	f0

d) Delay again by 1, multiply, and add to get h3.
 

g0	g1	g2				h3 = g1 f2 + g2 f1
	f2	f1	f0

 e) Delay one last time,  multiply, and add to get h4.
 

g0	g1	g2				h4 = g2 f2
		f2	f1	f0

Use this algorithm to write your computer program.

3) Z-transforms (polynomial multiplication).

LetF(z) = f₀ + f₁z^-1+ f₂ z^-2

and

G(z) =g₀ + g₁z^-1+ g₂ z^-2

If we let z = eiw, then:

F (w) = f₀+ f₁ e^-iw+ f₂ e^-2iw

and

G (w) = g₀+ g₁ e^-iw+ g₂ e^-2iw
 
 

Convolution Theorem: f (t) * g (t) < --------> F (w) G (w)

f (t) * g (t) < --------> F (z) G (z)

therefore

F (z) G (z) = (f₀ + f₁z^-1+ f₂ z^-2)(g₀+ g₁ z^-1+ g₂ z^-2)
                = g₀f₀
                        + (g₁f₀ + g₀f₁)z^-1
                                    + (g₂f₀ + g₁f₁ + g₀f₂)z^-2
                                                + (g₂f₁ + g₁f₂)z^-3
                                                            + g₂f₂ z^-4