Quid 13. Djéeuss supervised and unsupervised learning. Answer Supervised learning : 1. Input feature 10. Supervised learning is also known as associative learning, in which the network is trained by providing it with input and matching output patterns. ith Supervised training requires the pairing of each input vector wit! a target vector representing the desired output. . The input vector together with the corresponding target vector is called training pair. Target feature Matching Neural network Weight/threshold adjustment Error vector Supervised learning algorithm Fig. 1.3.1. During the training session an input vector is applied to the network and it results in an output vector. ” This response is compared with the target response, Ifthe actual response differs from the target will generate an error signal. onset response, the network This error signal is then used to calcul should be made in the synaptic sete . e adjustment that matches the target output. t the actual output The error minimization in this ki .. or teacher. Skind oftraining requires a supervisor These input-output pains can be i by the system whi . Provided by = ns tem which contains the neat ne enexternal teacher, or mapping in So i ethods are use, to “ pelt-supervised) n classifies: . networks ang multilayer cation networks fas non-linear networks associationu Unsupervised learning 1 ov 10. 11. 12. 13. Supervised learning generates a global model that maps input objeets to desired outputs 8 the map is implemented od reasoning or the near In some et of local models such t neighbour algorithm, In order to solve problem of supervised learning follow: ing steps considered steps are i, Determine the type of training examples. ii, Gathering a training set ui, Determine the input feature representation of the learned function _ w. Determine the structure of the learned function and corresponding learning algorithm. v. Complete the design. a learning in which an output unit is trained to respond to ters of pattern within the input. Unsupervised training is employed in self-organizing neural networks. This training does not require a teacher. In this method of training, the input vectors of similar types are grouped without the use of training data to specify how a typical member of each group looks or to which group a member belongs. During training the neural network receives input patterns and organizes these patterns into categories. When new input pattern is applied, the neural network provides an output response indicating the class to which the input pattern belongs. Tf a clas: generated. Though unsupervised training does not require a teacher, it requires certain guidelines to form groups. Grouping can be done based on color, shape and any other propert! annot be found for the input pattern, a new cla: of the object. mt It is a method of machine learning where a model is observations. sethereit It is distinguished from supervised learning by the fact ths no priori output. In this, a data set of input objects is &: It treats input objects as a set of random v: conjunction with Bayesian inference t probabilitie: athered. ariables. Ite produce © an be used yndition®1-6 L (CS/IT-Sem-5) Introduction 14. Unsupervised learning is useful for data compression and clustering. Vector describing state of the environment Learning system Environment -————-*} Fig. 1.3.2. Block diagram of unsupervised learning. 15. In unsupervised learning, system is supposed to discover statistically salient features of the input population. Unlike the supervised learning paradigm, there is not a priori set of categories into which the patterns are to be classified; rather the system must develop its own representation of the input stimuli. 16. ESEce Explain briefly the term machine learning. Y Machine learning is an application of Artificial Intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. Machine learning focuses on the development of computer programs that can access data. The primary aim is to allow the computers to learn automatically without human intervention or assistance and adjust actions accordingly. Machine learning enables analysis of massive quantities of data It generally delivers faster and more accurate results in order to identify profitable opportunities or dangerous risks. Combining machine learning with AI and cognitive technologies can make it even more effective in processing large volumes of information.Data sci eee Machine learning 1. Data sci a eS eae concept used | Machine learning is defined as ao ee ataand includes the practice of using algorithms | ae sing, preparation, | to use data, learn from it and | ysis. then forecast future trends for that topic. 2. It includes various data | It includes subset of Artificial operations. Intelligence. 4 3. Data science works by | Machine learning uses efficient | rams that can use data | sourcing, cleaning, and | prog processing data to extract without being ex! | meaning out of it for do so. | analytical purposes. | analvtice a | Amazon Lex, IBM = SAS. Tableau, Apache. Spark, Amazon Lex, IBM Watson MATLAB are the tools used Studio, Microsoft Azure ML | in data science. Studio are the tools used i ML - _ aan pith 4 —_—————_—— jence deals wit plicitly told to Machine learning uses stati stical | Data sc structured and unstructured models. | data. _ _ po _ ee 7 _ | Fraud detection and | Recommendation systems such | healthcare analysis are as Spotify and Facial Recognition | | examples of data science. are examples of machine learning. ee ee SSS ae a —_ ©00©et |g tee OO Differentiate between linear regression and logisti ics S.No. | Linear regression Logistics regression Linear regression is supervised regression model. In Linear regression, we predict the value by an integer number. al Logistic regression is a supery classification model. In Logistic regression, We the value by 1 or 0. sed predict No activation function is used. Activ: ation function is used to convert a linear regression equation to the logistic regression equation. ‘A threshold value is added. No threshold value is needed. It is based on the least square estimation. The dependent variable consists of only two categories. Linear regression is used to estimate the dependent variable in case of a change in independent variables. event. Logistic regression is used to calculate the probability of an Linear regression assumes the normal or gaussian distribution of the dependent variable. nee eos assumes the omial distributio n dependent variable. of theQuefl2 Jain Naive Bayes classifier.Answer 1. Naive Bayes model is the most common Bayesian network model used in machine learning. 2. Here, the class variable C is the root which is to be predicted and the attribute variables X, are the leaves. 3. The model is Naive because it assumes that the attributes are conditionally independent of each other, given the class. 1 0.9 0.8 0.7 0.6 Decision tree Naive Bayes Proportion correct on test set 0.5 0.4 0 20 40 60 80 100 Training set size Fig. 2.12.1. The learning curve for Naive Bayes learning. 4. Assuming Boolean variables, the parameters are : 0 = P(C = true), 0,, = P(X, = true|C = true), 0,. = P(X, = true | C = False) 5. Naive Bayes models can be viewed as Bayesian networks in which each X has C as the sole parent and C has no parents. 1 6. ANaive Bayes model with gaussian P(X, |C) is equivalent to a mixture of gaussians with diagonal covariance matrices. 7. While mixtures of gaussians are used for density estimation. in continuous domains, Naive Bayes models used in discrete and mixed domains. 8. Naive Bayes models allow for very efficient inference of marginal and conditional distributions. 9. Naive Bayes learning has no difficulty with noisy data and can re more appropriate probabilistic predictions. ee1 np a af. | Wain EM algorithm with steps. aa The Expectation-Maximization (EM) algorithm is an iterative way to find maximum-likelihood estimates for model parameters when the data isfhcomplete or has missing data point or has some hidden variables. EM chooses random values for the w new set of data. ing data points and estimates a These new value: data, by filling up mii e then recursively used to estimate a better first ing points, until the values get fixed. These are the two basic steps of the EM algorithm : a. Estimation Step: i. Initialize «y,,¥, and 1, by random values, or by K means clustering results or by hierarchical clustering results. ii. Then for those given parameter values, estimate the value of the latent variables (i.e., 7,). b. Maximization Step : Update the value of the parameters (i.e., Hy, =, and 7, ) calculated using ML method : i. Initialize the mean p,, the covariance matrix Z, and the mixing coefficients x, by random values, (or other values). ii, Compute the 7, values for all k. iii, Again estimate all the parameters using the current iv. Compute log-likelihood function, % Values, v. Put some convergence criterion, vi. If a log-likelihood value converges to (or if all the parameters conver, some value else return to Step 2. Be to some values) then sto P,Qualg.19. rite short note on support vector machine.‘Answer | 1. A Support Vector Machine (SVM) is machine learning algorithm that analyzes data for classification and regression analysis. 2. SVM isa supervised learning method that looks at data and sorts it into one of two categories. 3. An SVM outputs a map of the sorted data with the margins between the two as far apart as possible. 4. Applications of SVM: i. Text and hypertext classification Image classification Recognizing handwritten characters iv. Biological sciences, including protein classificationQue 2.20. | What are the types of support vector machine ? Answer Following are the types of support vector machine : 1 Linear SVM : Linear SVM is used for linearly separable data, which means if a dataset can be classified into two classes by using a single straight line, then such data is termed as linearly separable data, and classifier is used called as Linear SVM classifier. Non-linear SVM : Non-Linear SVM is used for non-linearly separated data, which means if a dataset cannot be classified by using a straight line, then such data is termed as non-linear data and classifier used is called as Non-linear SVM classifier.Que 3.4. | Explain various decision tree learning algorithms. iii. iv. 3. ‘o construct a decision tree, ID3 uses a top-down, greedy search through the given sets, where each attribute at every tree node is tested to select the attribute that is best for classification of a given set. Therefore, the attribute with the highest information gain can be selected as the test attribute of the current node. Inthis algorithm, small decision trees are preferred over the larger ones. It is a heuristic algorithm because it does not construct the smallest tree. For building a decision tree model, ID3 only accepts categorical attributes. Accurate results are not given by ID3 when there is noise and when it is serially implemented. Therefore data is preprocessed before constructing a decision tree. For constructing a decision tree information gain is calculated for each and every attribute and attribute with the highest information gain becomes the root node. The rest possible values are denoted by arcs. . All the outcome instances that are possible are examined whether they belong to the same class or not. For the inst; class, a single name is used to denote the claes an of the same instances are classified on the basis of splitting ateinerwse theC4.5: iii, 4.5 is an algorithm used to generate a decision tree. It isan extension of ID3 algorithm. C4.5 generates decision trees which can be used for classification and therefore C4.5 is referred to as statistical classifier. It is better than the ID3 algorithm because it deals with both continuous and discrete attributes and also with the missing values and pruning trees after construction. C5.0 is the commercial successor of C4.5 because it is faster, memory efficient and used for building smaller decision trees. C4.5 performs by default a tree pruning process. This leads to the formation of smaller trees, simple rules and produces more intuitive interpretations. CART (Classification And Regression Trees) : vi. CART algorithm builds both classification and regression trees. The classification tree is constructed by CART through binary splitting of the attribute. . Gini Index is used for selecting the splitting attribute The CART is also used for regressi : . regression tree. on analysis with the help of The regression feature of CART can be used in forecasting * dependent variable given a set : period of time. of predictor variable over a give? CART has an average speed of continuous and nominal attribute Processing and supports both On data.scribe K-Nearest Neighbour algorithm with steps. The KNN classification algorithm is used to decide the new instance should belong to which class. When K = 1, we have the nearest neighbour algorithm. KNN classification is incremental. KNN classification does not have a training phase, all instances are stored. Training uses indexing to find neighbours quickly. During testing, KNN classification algorithm has to find K-nearest neighbours of a new instance. This is time consuming if we do exhaustive comparison. K-nearest neighbours use the local neighborhood to obtain a prediction. Algorithm : Let m be the number of training data samples. Let p be an unknown point. Store the training samples in an array of data points array. This means each element of this array represents a tuple (x, y). Fori=6tom: Calculate Euclidean distance d(arr[z], p). Make set S of K smallest distances obtained. Each of . corresponds to an already classified data point. of these distances Return the majority label among S. ~~Artificial Neural Network : Refer Q. 1.13, Page 1-14L, Unit-1. A neural network contains the following three layers : a b. Input layer : The activity of the input units represents the rav information that can feed into the network. Hidden layer: i. Hidden layer is used to determine the activity of each hidder unit. ii The activities of the input units and the weights depend on the connections between the input and the hidden units. iii, There may be one or more hidden layers. Output layer : The behaviour of the output units depends on th activity of the hidden units and the weights between the hidde" and output units.Que 1.13. | Write short note on Artificial Neural Network (ANN). Answer 1. Artificial Neural Networks (ANN) or neural networks are computational algorithms that intended to simulate the behaviour of biological systems composed of neurons.10. ANNs are computational models inspired by an animal's central nervous systems. It is capable of machine learning as well as pattern recognition. A neural network is an oriented graph. It consists of nodes which in the biological analogy represent neurons, connected by arcs. It corresponds to dendrites and synapses. Each arc associated with a weight at each node. A neural network is a machine learning algorithm based on the model of a human neuron. The human brain consists of millions of neurons. It sends and process signals in the form of electrical and chemical signals. ‘These neurons are connected with a special structure known as synapses. Synapses allow neurons to pass signals. An Artificial Neural Network is an information processing technique. It works like the way human brain processes information. ANN includes a large number of connected processing units that work together to process information. They also generate meaningful results from it.Jue 45. | Explain different types of neuron connection with architecture. 7 pifferent types of neuron connection are : Single-layer feed forward network : 1 a. In this type of network, we have only two layers i.e., input layer and output layer but input laver does not count because no computation is performed in this layer. b. Output layer is formed when different weights are applied on input nodes and the cumulative effect per node is taken. c. After this the neurons collectively give the output layer to compute the output signals. Input layer Output layer ltilayer feed forward network : This La, . Aaa yer has hidden layer which is internal to the net e we b . no direct contact with the external layer. ork and Existence of one or more hidden layers enaj ™Putationally stronger. sre are no feedback connections ii ; in whi “te fed back into itself. ch outputs of the mode] bles the network to be3. Single node with its own feedback: os When outputs can be directed back aoe e same ae 7 preceding layer nodes, then it results in feedbac! ae works. Recurrent networks are feedback networks with closed loop Fig. 4.5.1 shows a single recurrent network having single neuro with feedback to itself. a. b. 4, Single-layer recurrent network : a. Input Output —> — Feedback Fig. 4.5.1. This network is single layer network with feedback connecti?™ which processing element’s output can be directed back to tse!” to other processing element or both. where con neat network is a class of artificial neural nett inection: ' ni ees $ between nodes form a directed graph 2! This allows it to exhil / / tio sequence. Unlike ais cr een ee cat their internal state (, a ae eural networks, RNNs °# ory) to process sequences of inp"Multilayer recurrent network : a In this type of network, processing element output can be directed to the processing element in the same layer and in the preceding layer forming a multilayer recurrent network. They perform the same task for every element of a sequence, with the output being depended on the previous computations. Inputs are not needed at each time step. The main feature of a multilayer recurrent neural network is its hidden state, which captures information about a sequence.¥xplain perceptron with single flow graph. 1. The perceptron is the simplest form of a neural network used for classification of patterns said to be linearly separable. 2. It consists of a single neuron with adjustable synaptic weights and bias. 3. The perceptron build around a single neuron is limited for performing pattern classification with only two classes. 4. By expanding the output layer of perceptron to include more than one neuron, more than two classes can be classified. 5. Suppose, a perceptron have synaptic weights denoted by w,,W,, Ws, Wine 6. The input applied to the perceptron are denoted DY Ky yy eee Bye 7. The externally applied bias is denoted by b. xy Hand limiter Fig. 4.13.1. Signal flow graph of the perceptron. From the model, we find that the hard limiter input or induced local field of the neuron as 8. V= Swe, +b a10. i. The goal of the perceptron is to correctly classify goa" the set of externally applied input x,,x,,...... x, into one of two classes G, and G,. The decision rule for classification is that if output y is +1 then assign the point represented by input x,, x», ....... x,, to class G, else y is—1 then assign to class G,, ” In Fig. 4.13.2, if a point (x,, x,) lies below the boundary lines is assigned to class G, and above the line is assigned to class G,. Decision boundary is calculated as : WX, + Wor, + b= 0 Decision boundary to W)X1 + WX, +b=0 | Glass < Glass Gy 0 4 Fig. 4.13.2. There are two decision regions separated by a hyperplane defined as : Ywx,+b =0 The synaptic weights w,, w», .......W,, of the perceptron can be adapted on an iteration by iteration ba: For the adaption, an error-correction rule known as perceptron convergence algorithm is used. . For a perceptron to function properly, the two classes G, and G, must be linearly separable. Linearly separable means, the pattern or set of inputs to be classified must be separated by a straight line Generalizing, a set of points in n-dimensi if there is a hyperplane of (n ~ 1) dime: nai space are linearly separable ‘ons that separates the sets ‘@) A pair of linearly separable patterns (bys A Pair of non-linear se] Fig. 4.13.3, Parable Patterns4-13 L(CSAT-Sem-5) Machine Learning Techniques nally The goal of the perceptron is to correctly classify the set of exte applied input x,,x,,........x,, into one of two classes G, and Gy. The decision rule for classification is that if output y is +1 then assi point represented by input x,,.x,,......%, to class G, else y is ~1 then assign to class G,. 11. In Fig. 4.13.2, ifa point (x,, x, to class G, and above the line is assigned to class G,. Dec’ is calculated as : wx, + Wx, +b =0 9. a ign the ) lies below the boundary lines is assigned ision boundary Decision boundary fp WX, + ox +b =0 | Glass K Glass G, x a 1 Fig. 4.13.2. 12. There are two decision regions separated by a hyperplane defined as : Dwsx+b =0 a The synaptic weights w,, W,,. ,, of the perceptron can be adapted on an iteration by iteration basi 13, For the adaption, an error-correction rule known as perceptron convergence algorithm is used. 14. For a perceptron to function properly, the two classes G, and G, must be linearly separable. 2 1. Li y 8 ans, the i . inearly separable means, the pattern or set of inputs to be classified must be separated by a straight line. 16. Generalizing, a set of points in n-dimensional sj 7 +. z al space are linea: if there is a hyperplane of (n — 1) dimensions that separate: thee ple sets. $ ¥ or a) A pair of linearly ()A pair of n separable patterns ‘on-linearly Fig. 4.13.3, “parable patternsSelection of various parameters in BPN: 1, Number of hidden nodes : a. The guiding criterion is to select the minimum nodes in the first and third layer, so that the memory demand for storing the weights can be kept minimum. b. The number of separable regions in the input space M, is a function of the number of hidden nodes H in BPN andH=M-1. c. When the number of hidden nodes is equal to the number of training patterns, the learning could be fastest. d._ Insuch cases, BPN simply remembers training patterns losing all generalization capabilities. ec. Hence, as far as generalization is concerned, the number of hidden nodes should be small compared to the number of training patterns with help of Vapnik Chervonenkis dimension (VCdim) of probability theory. f. We can estimate the selection of number of hidden nodes for 4 given number of training patterns as number of weights which is equal to I, * I, + 1, * J, where J, and J, denote input and output nodes and I, denote hidden nodes. g. Assume the training samples T to be greater than VCdim. Now if we accept the ratio 10:1 ioe r= (1, +1,) ae ,+1,) Which yields the value for /,,. 2. Momentum coefficient a: a. Toreduce the training time we use the momentum factor becaus? it enhances the training process. b. The influences of momentum on weight change isOE awit! = — wr N35 +alAW] The momentum also overcomes the effect of local minima. ht change process c. d. The use of momentum term will carry a weig! aa through one or local minima and get it into global minima. (Weight change without momentum) [aw]? afAWl? [aw)+! (Momentum term) Fig. 4.17.1. Influence of momentum term on weight change. Sigmoidal gain 1: a. When the weights become large and force the neuron to operate in a region where sigmoidal function is very flat, a better method of coping with network paralysis is to adjust the sigmoidal gain. b. By decreasing this scaling factor, we effectively spread out sigmoidal function on wide range so that training proceeds faster. Local minima : a. One of the most practical solutions involves the introduction of a shock which changes all weights by specific or random amounts b. If this fails, then the most practical solution is to r ” ees erandomi weights and start the training all over. omize the} Q 21, at is Self-Organizing Map (SOM) ? 4 Ans : 1. Self-Organizing Map (SOM) provides a data visualization technique which helps to understand high dimensional data by reducing the dimensions of data to a map. 2. SOM also represents clustering concept by grouping similar data together.ee Tt 3, Aself-Orga nizing Map (SOM) or Self-Organizing Feature Map (SOFM) is a type of Artificial Neural Network (ANN) that is trained using unsupervised learning to produce a low-dimensional (typically two- dimensional), discretized representation of the input space of the training samples, called a map, and is therefore a method to do dimensionality reduction. Self-organizing maps differ from other artificial neural networks as they apply competitive learning as opposed to error-correction learning (such as backpropagation with gradient descent), and in the sense that they use a neighborhood function to preserve the topological properties of the input space.e convolutional networks.wae | Convolutional networks also known as Convolutional Neural Networks (CNNs) are a specialized kind of neural network for proce: that has a known, grid-like topology. Convolutional neural network indicates that the network employs a mathematical operation called convolution. Convolution is a specialized kind of linear operation. Convolutional networks are simply neural networks that use convolution in place of general matrix multiplication in at least one of their layers. ing data CNNs, (ConvNets), are quite similar to regular neural networks. They are still made up of neurons with weights that can be learned from data. Each neuron receives some inputs and performs a dot product. They still have a loss function on the last fully connected layer. They can still use a non-linearity function a regular neural network receives input data as a single vector and passes through a series of hidden layers. tN LAN | output layer input layer hidden layer1 _ hidden layer 2 Fig. 4.28.1. A regular three-layer neural network. Every hidden layer consists of neurons, wherein every neuron is fully connected to all the other neurons in the previous layer. Within a single layer, each neuron is completely independent and they do not share any connections. The fully connected layer, (the output layer), contains class scores in the case of an image classification problem. There are three m, " ain layers ij simple ConvNet. ¥ nQue 4.24, | What do you understand by deep learning ? za 1. Deep learning is the subfield of artificial intelligence that focuses on creating large neural network models that are capable of making accurate data-driven decisions.J Deep learning is used where the data is complex and has large datasets. 3, Facebook uses deep learning to analyze text in online conversations. Google and Microsoft all use deep learning for image search and machine translation. 4, Allmodern smart phones have deep learning systems running on them. For example, deep learning is the standard technology for speech recognition, and also for face detection on digital cameras. n In the healthcare sector, deep learning is used to process medical images (X-rays, CT, and MRI scans) and diagnose health conditions. 6, Deep learning is also at the core of self-driving cars, where it is used for localization and mapping, motion planning and steering, and environment perception, as well as tracking driver state. Que 4.25. | Describe different architecture of deep learning. Answer | Different architecture of deep learning are : 1. Deep Neural Network : It is a neural network with a certain level of complexity (having multiple hidden layers in between input and output layers). They are capable of modeling and processing non-linear relationships. 2 Deep Belief Network (DBN) : It is aclass of Deep Neural Network. It is multi-layer belief networks. Steps for performing DBN are: a. Learn a layer of features from visible units using Contrastive Divergence algorithm. b. Treat activations of previously trained features as visible units and then learn features of features. c. Finally, the whole DBN is trained when the learning for the final hidden layer is achieved. Recurrent (perform same task for every element of a sequence) Neural Network : Allows for parallel and sequential computation. Similar to the human brain (large feedback network of connected neurons). They are able to remember important things about the input they received and hence enable them to be more precise, 4.26. is Que What are the advantages, disadvantages and limitati Of dear 1 it .— —__“Tng Answer Advantages of deep learning : 1. Best in-class performance on problems. 2. Reduces need for feature engineering. 3. Eliminates unnecessary costs. 4. Identifies defects easily that are difficult to detect. Disadvantages of deep learning : 1. Large amount of data required. 2. Computationally expensive to train. 3. No strong theoretical foundation. Limitations of deep learning : 1. Learning through observations only. 2. The issue of biases. Que 4.27. | What are the various applications of deep learning? Answer Following are the application of deep learning : 1, Automatic text generation : Corpus of text is learned and from this model new text is generated, word-by-word or character- by-character. Then this model is capable of learning how to spell, punctuate, form sentences, or it may even capture the style. 2. Healthcare : Helps in diagnosing various diseases and treating it. 3. Automatic machine translation ; phrases in one language is transformed i; Learning is achieving top results in oa archer language (Dee? ie the areas of text, images). ‘ re content. i i ; being used in Gaming, Retail, Tourism. aa Pant Ths aiea i sitost 5. Pregiating earthquakes : Teaches a computer to perform visc b :Te iscoelastiC putatigns which are used in Predicting earthquakes. IN be Certain words, sentences oFwe] Describe reinforcement learning. fowver | Reinforcement learning is the study of how animals and artificial systems can learn to optimize their behaviour in the face of rewards and punishments. Reinforcement learning algorithms related to methods of dynamic programming which is a general approach to optimal control. = a penera’ approach to op Reinforcement learning phenomena have been observed in psychological studies of animal behaviour, and in neurobiological investigations of neuromodulation and addiction. The task of reinforcement learning is to use observed rewards to learn an optimal policy for the environment. An optimal policy is a policy that maximizes the expected total reward. Without some feedback about what is good and what is bad, the agent will have no grounds for deciding which move to make. The agents needs to know that something good has happened when it Wins and that something bad has happened when it loses. This kind of feedback is called a reward or reinforcement, Reinforcement learning is valuable in the field of robot: ‘asks to be performed are frequently complex e & programs and no training data is available, ics, where the nough to defy encoding In many i i . 'y complex domains, reinforcement learning ig “ay to train a program to perform at high levels 81s the only feasibleMachine Learning Techniques 5-3 L (CS/IT-Sem-5) Primary reinforcement signal Critic Heuristic reinforcement signal earning system 5.1.1. Block diagram of reinforcement learning. Fig. poque 5.14. Explain the term Q-learning ‘ower | 1 2 Qlearning 15 a model-free reinforcement learning algorithm. m. Value based algorithms learning is a values-based learning algorith arly Bellman updates the value function based on an equation (particul equation). Whereas the other type, policy-based estimates agreedy policy obtained from the last policy improvement. policy learner i.e., it learns the value of t t’s actions. the value function with Qlearning is an off- he optimal policy independently of the agen On the other hand, an on-policy learner learns the value of the policy being carried out by the agent, including the exploration steps and it will find a policy that is optimal, taking into account the exploration inherent scribe Q-learning algorithm process.Answer Step 1: Initialize the Q-table : First the Q-table has to be built. There are n columns, where n = number of actions. There are m rows, where m = number of states. In our example n = Go left, Go right, Go up Correct path, Wrong path and End. First, le Step 2: Choose an action. ; Step 3 : Perform an action : The combination of steps 2 and 3 is performed for an undefined amount of time. These steps run until the time training is stopped, or when the training loop stopped as defined in the code. a. First, an action (a) in the state (s) is chosen based on the Q-table. Note that, when the episode initially starts, every Q-value should be 0. b. Then, update the Q-values for being at the start and moving right using the Bellman equation. Step 4 : Measure reward : Now we have taken an action and observed an outcome and reward. Step 5 : Evaluate : We need to update the function Q(s, a) This process is repeated again and again until the learning is stopped. In this way the Q-table is been updated and the value function Q is maximized. Here the Q returns the expected future reward of that action at that state. and Go down and m = Start, Idle, ts initialize the value at 0. PART-4 Introduction to Deep Q Learning. [ Questions-Answers , Long Answer Type and Medium Answer Type Questions Qublb.26. | Dyscribe deep Q-learning. Answer 1. In deep Q-learning, we use an value function. eural network to approximate the Q- 2. The state is given as the input generated as the output, "4 the @-value of all possible actions isQ learning State tr Q-value action 1 Q-value action 2 | —>[Q-value action N] Deep Q learning Fig. 5.16.1. On a higher level, Deep Q learning works as such : i. Gather and store samples in a replay buffer with current policy. ii, Random sample batches of experiences from the replay buffer. iii, Use the sampled experiences to update the Q network. iv. Repeat 1-3.Genetic algorithms are computerized search and optimization algorithm based on mechanics of natural genetics and natural selection. These algorithms mimic the principle of natural genetics and natural selection to construct search and optimization procedure. Genetic algorithms convert the design space into genetic space. Design space is a set of feasible solutions. Genetic algorithms work with a coding of variables. The advantage of working with a coding of variables space is that coding discretizes the search space even though the function may be continuous. Search space is the space for all possible feasible solutions of particular problem. Following are the benefits of Genetic algorithm : a. They are robust. b. They provide optimization over large space state. c. They do not break on slight change in input or presence of noise. Following are the application of Genetic algorithm : a. Recurrent neural networkb. Mutation testing c. Code breaking ad Filtering and signal processing e. Learning fuzzy rule base Que 5.20. | Write procedure of Genetic algorithm with advantages and disadvantages. Answer Procedure of Genetic algorithm : Generate a set of individuals as the initial population. Use genetic operators such as selection or cross over. Apply mutation or digital reverse if necessary. Evaluate the fitness function of the new population. of eM Use the fitness function for determining the best individuals and replace predefined members from the original population. Iterate steps 2-5 and terminate when some predefined population threshold is met. tivantages of genetic algorithm : » Genetic algorithms can be executed in parallel. Hence, genetic algorithms are faster. : Itis useful for solving optimization problems. si advantages of Genetic algorithm : Ident i i probication of the fitness function is difficult as it depends on the Totlation of suitable genetic operators is difficult. Mero,yu e 5.29. | Draw genetics cycle for genetic algorithm, qe20.] Answer Generational cycle of GA: 1 Population (Chromosomes) Decoded Offsprings string New generation Genetic P. t Evaluation operator arents (Fitness) Manipulation Mate Reproduction Selection E (Mating pool) Fig. 5.29.1. The GA cycle, Components of Senerational cycle in GA: Population (Chromosomes) ; A Population is collection of individual A population consists of a number of individuals being tested, inn Phenotype parameters defining the individuals and some inform#t about search space, ; . ve Evaluation fitness) ; 4 fitness function is a particular type often function that quantifies the optimality of a solution (ie.,a chrom0sOEE, APAEL, BEBE Se NACE CRABS UE UELERL in a genetic algorithm so that particular chromosome may be ranked against all the other chromosomes. 3 Selection : During each successive generation, a proportion of the existing population is selected to breed a new generation. Individual solutions are selected through a fitness-based process. 4 Generic operator : A genetic operator is an operator used in genetic algorithm to guide the algorithm towards a solution to a given problem.