**Dimension Reduction Techniques : Introduction**

**Dimension reduction is a strategy with the help of which, data from high dimensional space can be converted to low dimensional space. This can be achieved using any one of the two dimension reduction techniques :****Linear Discriminant Analysis(LDA)****Principal Component Analysis(PCA)**

**1. **__Linear Discriminant Analysis(LDA)__

__Linear Discriminant Analysis(LDA)__

**Linear discriminant analysis i.e. LDA is one of the dimension reduction techniques which is capable of discriminatory information of the class.****The major advantage of using LDA strategy is, it tries to obtain directions along with classes which are best separated.****Scatter within class and Scatter between classes, both are considered when LDA is used.****Minimizing the variance within each class and maximizing the distance between the means are the main focus of LDA.**

__Algorithm for LDA__

__Algorithm for LDA__

**Let the number of classes be “c” and u**_{i }be the mean vector of class i, where i=1,2,3,.. .**Let N**_{i }be the number of samples within class i, where i=1,2,3…C.

**Total number of samples, N=∑ Ni.**

**Number of samples within Class Scatter Matrix.**

**Number of samples between Class Scatter Matrix.**

__Advantages Of : Linear Discriminant Analysis__

__Advantages Of : Linear Discriminant Analysis__

**Suitable for larger data set.****Calculations of scatter matrix in LDA is much easy as compared to****co-variance****matrix.**

__Disadvantages : Linear Discriminant Analysis__

__Disadvantages : Linear Discriminant Analysis__

**More redundancy in data.****Memory requirement is high.****More Noisy.**

__Applications : Linear Discriminant Analysis__

__Applications : Linear Discriminant Analysis__

**Face Recognition.****Earth Sciences.****Speech Classification.**

**2. **__Principal Component Analysis(PCA)__

__Principal Component Analysis(PCA)__

**Principal Component Analysis i.e. PCA is the other dimension reduction techniques which is capable of reducing the dimensionality of a given data set along with ability to retain maximum possible variation in the original data set.****PCA standouts with the advantage of mapping data from high dimensional space to low dimensional space.****Another advantage of PCA is, it is able to locate most accurate data representation in low dimensional space.****In PCA, Maximum variance is the direction in which data is projected.**

__Algorithm For PCA__

__Algorithm For PCA__

**Let d**_{1},d_{2}, d_{3},…,d_{d }be the whole data set consisting of d-dimensions.**Calculate the mean vector of these d-dimensions.****Calculate the covariance matrix of data set.****Calculate Eigen values(λ**_{1},λ_{2,}λ_{3,…,}λ_{d}) and their corresponding Eigen vectors (e_{1}, e_{2}, e_{3},….e_{d}).**Now, Sort the Eigen vectors in descending order and then choose “p” Eigen vectors having largest values in order to generate a matrix “A” with dimensions p*d.**

**i.e. A = d * p.**

**Using the matrix “A” (i.e. A = d * p) in order to transform samples into new subspace with the help of:**

**y = A ^{T} * x**

**Where, A ^{T} Transpose matrix of “A”**

__Advantages : Principal Component Analysis__

__Advantages : Principal Component Analysis__

**Less redundancy in data.****Lesser noise reduction.****Efficient for smaller**

__Disadvantages : Principal Component Analysis__

__Disadvantages : Principal Component Analysis__

**Calculation of exact****co-variance****matrix is very difficult.****Not suitable for larger data sets.**

__Applications : Principal Component Analysis__

__Applications : Principal Component Analysis__

**Nano-materials.****Neuroscience.****Biological Systems.**