Notes: • Present all responses in a clear, ordered and detailed manner, with a brief justification of each step. • Present all calculations. • Keep at least three decimal places in all calculations.
Problem 1 Consider the following multilayer perceptron shown in the figure below-left, where all the units have as activation function the logistic function = . Also consider the training set given on the table below-right.
x1 x2 d1 d2
-1 1 0.5 0.5
1 -2 0 -1
a) Compute the values obtained at the multilayer perceptron’s outputs for the second input pattern. Assume that all weight values are equal to 0.5. b) Draw the backpropagation network. Don’t forget to include the gains of all branches, as well as the input and output variables. If necessary, first draw the multilayer perceptron as a network with more detail, in order to be able to more easily specify some parameters of the backpropagation network. c) Assume that, initially, all the multilayer perceptron’s weights were equal to 0.4, and that, after a first update using backpropagation in real-time mode, all the weights were changed to 0.5. Compute the value of weight a after a second update, also in real-time mode. The cost function is the total squared error and the training is performed using non- adaptive step sizes with step size parameter η = 0.1, without momentum. d) Repeat 3) using the same step size parameter η = 0.1, with momentum with parameter α = 0.1.
Problem 2 Consider a classification problem in two dimensions, with two classes, in which the training set is given by
x1 x2 class x1 x2 class
0 √2 A 1 -3 B 1 √2 A √3 2 B
√2 0 A √6 1 B 0 1 A −√2 -√6 B
a) Graphically sketch the positions of these patterns. Show that the two classes are not linearly separable. Briefly describe a technique that can be used, in support vector machines, to obtain a linear separation when the input data are not linearly separable. b) Consider the following nonlinear mapping from input space to a two-dimensional feature space. Sketch the training patterns in feature space. = ; −2 c) Find the kernel function that corresponds to the nonlinear mapping given above. d) Find (by inspection) the widest-margin classifier in feature space. Indicate the support vectors and the equations of the classification boundary and of the margin boundaries. e) Sketch the classification regions of classes A and B in feature space. You may draw them on the sketch of b) above. f) Indicate, in input space, the support vectors and sketch the classification boundary and margins. You may draw them on the sketch of a) above. g) Sketch the classification regions of classes A and B in input space.
Problem 3 Consider the training set T1 = {–2, –1,1,4,5}. a) Perform one iteration of the k-means algorithm using three centers with initial positions -1, 0, and 3. b) Determine the value of the cost function for the position of the centers obtained in a). Is this a fixed point of the algorithm? Now consider a new training set T2 = {–2, –1,1}. c) Perform one iteration of the EM algorithm to estimate the parameters of a mixture of three Gaussians. Assume the following initial conditions: = −1 =1 = 0.6 =0 =1 = 0.2 =3 =1 = 0.2
Problem 4 Consider the training set and the tree given on the following table and figure, respectively. In the table, the represent attributes of the input patterns and represents the desired output for each pattern.
a) Find the number of errors that the tree yields in the training set. 0 0 1 b) Perform the pruning of the tree using the given training set.
Problem 5 Consider the set of patterns $%6,4() , %−2,0() , %3,0() , %1,4() * . The patterns are considered equiprobable. a) Find the first and second principal directions of the distribution of these patterns. Indicate those directions by means of vectors. b) Find the first principal component, the reconstruction with that principal component and the reconstruction error of the pattern %3,5() , based on the given distribution. If you didn’t solve item a) above, assume that the first principal direction was given by the vector %−3,1() . c) Consider two random variables , and - which are statistically independent from each other. Assume that they have the same principal directions, although possibly not in the same order (for example, the first principal direction of , may be the second principal direction of -). Show that, for any , . ∈ ℝ, the random variable 1 = , + .- also has the same principal directions (possibly in a different order from those of , or -). If you’re not able to give the proof as requested, you may make any additional assumptions that you find necessary (for example, that the principal directions of , and - are in the same order, or that = .). However, the value of this problem will be reduced, depending on the additional assumptions that you make.
