We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17
Intro to EECE 490/690
Machine Learning – Lec 2
Mariette Awad
Slide sources: presentations and Stanford Intro to ML course
Lecture Outline • What is ML? • What are types of ML? • What are the elements of ML cycle? Lectures Outcomes • Define ML • Understand the differences between the types of ML • Identify the different elements of the ML cycle and understand the impact of the data split on the model performance Machine Learning • Top 1 of 14 skills in IT (https://learntocodewith.me/posts/tech-skills-in-demand/) • Grew out of AI to build intelligent machines: have machine learn on their own • Some definitions • Arthur Samuel – 1959: teaching computers to learn on their own without being programmed. He developed a checker’s game and the computer played 10s of thousands of games against itself and learned what are bad/good board positions and became better player than Arthur himself. • Tom Mitchell – 1998: Well-posed learning: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E. For the checker’s game, E: playing checkers games; T: Playing Checkers; P: probability of winning. What is Machine Learning? • Machine learning aims at developing algorithms and models for machines to perform predictions or learn to perform human tasks.
Data (X) Model (𝜃𝐿 )
Machine Learning
• Once the model is learned, it can be used for desired prediction
of new unknown data. What is Machine Learning (cont-d)? • Mathematically Machine learning tries to derive a mapping/function/model, where the input is typically training data, and the output is the desired model.
• Mathematically, assume models can be represented by a
set of parameters 𝜃.
• The desired model parameters 𝜃𝐿 are found as:
𝜃𝐿 = argmin[LossFunction(𝑋, 𝜃)] 𝜃 X represents the input data - with or without labels; depending whether we have supervised or unsupervised learning. Traditional Programming and Machine Learning
Write a Computer program with explicit rules to follow:
If email contains … then mark as spam If email contains … then mark as spam
Write a Computer program to learn from labeled data:
Try to classify this data and change model parameters to reduce errors Artificial Intelligence Big Data The science of processing Make machine performance cognitive functions structured and unstructured data so that meaningful insights can be derived from it
Data Science Machine Learning
The science of deriving predictions from the insights Uses algorithms to make the AI learn in order to benefit the without programming it explicitly business Data Analytics
Curating meaningful insights Deep Learning
from the data Cascade of Neural Networks. It is a part Machine Learning Types of ML Algorithms Types of ML Algorithms • Supervised Learning: Teach the computer how to learn • Unsupervised Learning: Teach computer to learn by itself about the data/environment • Reinforcement learning: Teach computer to learn by itself how to take actions to maximize some notion of total reward Design Cycle for: Model Development and Evaluation The Design Cycle • Data collection • Feature Choice • Model Choice • Training • Evaluation • Computational Complexity ML Design Cycle- Model Development Steps 1. Decide on initial predictors/features/attributes for the model in non deep models 2. **Collect (Training) Data. Also called Ground Truth data. 3. Pre-process the predictor data. 4. Finalize choice of predictors (e.g. feature engineering/reduction) 5. Split Data into Training data versus Test data. 6. Develop and Tune Model a. Model Tuning & Resampling: Select Model Parameters (also called hyperparameters) – Avoid overfitting! Another round of data splitting for evaluation b. Develop model based on chosen parameters. Use all Training data. 7. Evaluate Model with test data 8. Compare models based on resampled training data and test data. Use boxplot comparison or t-test. 9. Refine Model Data Split for Training versus Test • test data ≠ training data to provide an unbiased evaluation. Works well with large data. • Rule of thumb: 67%-80% versus 20-33%. • Possible Approaches for splits: • Random Sampling. Works poorly with imbalanced data. • Stratified sampling for the classes, with random sampling within group. • Sample based on predictors using bottom- up approach with maximum dissimilarity. Start test data with one point. FYI: AI Technologies Libraries • Sci-Kit Learn supports designing algorithms and models for classifications, clustering, pre-processing, regression, Dimensional reduction, and Model selection. Use it for feature-based ML • Keras Mainly built on TensorFlow, the framework successfully manages to be easy in terms of use. Use it for Neural Network and Back-propagation For Deep Learning • PyTorch Machine Learning Framework, which is based on Torch and Caffe2, suited to design neural networks and deep learning • TensorFlow — The open-source machine learning framework maintained by Google. Tools: Python and R dominating