EdYoda

Data Scientist Program

Program Curriculum
 Learning outcomes:
• Learn to implement Machine Learning techniques using Python
• Learn data visualization techniques
• Learn to analyze raw data
• Learn Big Data and Spark

Python

1. Introduction to Python

• Useful Python Resources

• Python Tools and Utilities
• Python Features

2. Python Environment

• Local Environment Setup

• Downloads and Installations
• Setting up Environment Path

3. Executing Python

• Interactive Mode
• Scripting Mode
• Integrated Development Environment

4. Python Basic Syntax

• Python Identifiers
• Reserved Words
• Lines and Indentation

www.edyoda.com [email protected]
5. Python Variable Types

• Assigning Values to Variables

• Multiple Assignment
• Standard Data Types
• Data Type Conversion

6. Python Basic Operators

• Arithmetic Operators
• Comparison Operators
• Assignment Operators
• Bitwise Operators
• Logical Operators
• Membership Operators
• Identity Operators
• Operators Precedence

7. Python Decision Making

• IF statements
• IF...ELIF...ELSE Statements
• Nested IF statements

8. Python Loops

• While loop
• For loop
• Nested loop
• Break control statement
• Continue statement
• Pass statement

9. Python Numbers

• Number type conversion

• Mathematical function
• Random number function
• Trigonometric function

www.edyoda.com [email protected]
10. Python Strings

• String special operators

• String formatting operator
• Built-in string methods

11. Python Lists

• Basic list operations

• Indexing and slicing
• Built-in functions and methods

12. Python Tuples

• Basic tuple operations

• Indexing and slicing
• Built-in functions

13. Python Dictionary

• Basic Dictionary operations

• Built-in Functions and Methods
• Use cases

14. Python Functions

• Pass by reference and value

• Function Arguments
• Scope of variables
• Default Argument Values
• Keyword Arguments
• Arbitrary Argument Lists
• Unpacking Argument Lists
• Lambda Expressions
• Documentation Strings

www.edyoda.com [email protected]
15. Python Modules

• Importing Modules
• Namespaces and scoping
• Packages

16. Python Files I/O

• Writing and Parsing Text Files

• Parsing Text Using Regular Expressions
• Writing and Parsing XML Files
• Writing and Parsing JSON Files
• Writing and Parsing CSV Files

17. Python Exceptions

• The except clause with multiple exceptions

• The try-finally clause
• Argument of an Exception
• Raising an exception
• User-Defined Exceptions

18. Python Classes and Objects

• Creating Classes
• Creating instance objects
• Destroying Objects (Garbage Collection)
• Custom Classes
• Attributes and Methods
• Inheritance and Polymorphism
• Using Properties to Control Attribute Access

19. Functional Programming

• Lambda
• Filter
• Map
• Functools

www.edyoda.com [email protected]
20. Iterators and Generators

• Itertools
• Generators
• Decorators

21. Collections

• Deque
• Counter
• OrderedDict
• ChainMap

23. Debugging, Testing

• Pdb
• Breakpoints

24. Regular Expressions

• Characters and Character Classes

• Quantifiers
• Grouping and Capturing
• Assertions and Flags
• The Regular Expression Module

25. Deploying Python Applications

• Pip
• Virtualenv
• The init.py files
• The setup.py file
• Installing the package
• Software deployment in Python

www.edyoda.com [email protected]
 Data Wrangling

1. Black Box Introduction to Machine Learning

• What is not Machine Learning

• What is Machine Learning
• Types of ML - Supervised, Unsupervised
• Supervised - Classification, Regression
• Unsupervised - Clustering, Association
• Machine Learning Pipeline

2. Essential NumPy

• Introduction to NumPy
• Creation
• Access
• Stacking and Splitting
• Methods
• Broadcasting

3. Pandas for Machine Learning

• Introduction to Pandas
• Understanding Series & DataFrames
• Loading CSV,JSON
• Connecting databases
• Descriptive Statistics
• Accessing subsets of data - Rows, Columns, Filters
• Handling Missing Data
• Dropping rows & columns
• Handling Duplicates
• Function Application - map, apply, groupby, rolling, str
• Merge, Join & Concatenate
• Stacking, Unstacking & Melting
• Pivot-tables
• Normalizing JSON
• Application - EDA on Employee data, sales data

www.edyoda.com [email protected]
4. Understanding Visualization:

• Introduction to matplotlib & seaborn

• Basic Plotting
• Title, Labels, Legends, Grid, colormap, xticks, yticks
• Color, linewidth
• Sub Plotting
• Scatter plot
• Histogram
• Bar Graphs
• Plotting distributions
• Plotting 3D data
• Fundamentals of Tableau

Mathematics Fundamentals

1. Essential Maths & Statistics

• Essential Linear Algebra

• Matrix Operations
• Understanding distributions
• Probability Concepts
• Calculus
• Understanding distributions
• Mean, Median, Mode, Quantile
• Other statistics Concepts
• Sampling Techniques

Machine Learning

1. Linear Models for Classification & Regression

• Simple Linear Regression using Ordinary Least Squares

• Gradient Descent Algorithm
• Regularized Regression Methods - Ridge, Lasso, Elastic Net
• Logistic Regression for Classification
• OnLine Learning Methods - Stochastic Gradient Descent & Passive Aggressive
• Robust Regression - Dealing with outliers & Model errors
• Polynomial Regression
• Bias-Variance Tradeoff
• Application - House Price, Cancer Prediction, Insurance Prediction

www.edyoda.com [email protected]
2. Preprocessing for Machine Learning

• Introduction to Preprocessing
• StandardScaler
• MinMaxScaler
• RobustScaler
• Normalization
• Binarization
• Encoding Categorical (Ordinal & Nominal) Features
• Imputation
• Polynomial Features
• Custom Transformer
• Text Processing
• CountVectorizer
• TfIdf
• HashingVectorizer
• Image using skimage

3. Decision Trees

• Introduction to Decision Trees

• The Decision Tree Algorithms
• Decision Tree for Classification
• Decision Tree for Regression
• Advantages & Limitations of Decision Trees
• Application - Cloth Prediction

4. Naive Bayes

• Introduction Bayes' Theorem

• Naive Bayes Classifier
• Gaussian Naive Bayes
• Multinomial Naive Bayes
• Bernoulli’s Naive Bayes
• Naive Bayes for out-of-core
• Application - Text Classification, Sentiment Analysis and Spam & Non-spam
classification

www.edyoda.com [email protected]
5. Composite Estimators using Pipelines & FeatureUnions

• Introduction to Composite Estimators

• Pipelines
• Transformed Target Regressor
• FeatureUnions
• ColumnTransformer
• GridSearch on pipeline
• Application - Author classification

6. Model Selection & Evaluation

• Cross Validation
• Hyperparameter Tuning
• Model Evaluation
• Model Persistence
• Validation Curves
• Learning Curves

7. Feature Selection & Dimensionality Reduction

• Introduction to Feature Selection

• Variance Threshold
• Chi-squared stats
• ANOVA using f_classif
• Univariate Linear Regression Tests using f_regression
• F-score vs Mutual Information
• Mutual Information for discrete value
• Mutual Information for continues value
• SelectKBest
• SelectPercentile
• SelectFromModel
• Recursive Feature Elimination
• PCA
• SVD
• Application - Credit Risk Prediction

8. Nearest Neighbors

• Fundamentals of Nearest Neighbor Algorithm

• Unsupervised Nearest Neighbors
• Nearest Neighbors for Classification

www.edyoda.com [email protected]
 • Nearest Neighbors for Regression
• Nearest Centroid Classifier
• Application - Nearest neighbour for face inpainting

9. Clustering Techniques

• Introduction to Unsupervised Learning

• Clustering
• Similarity or Distance Calculation
• Clustering as an Optimization Function
• Types of Clustering Methods
• Partitioning Clustering - KMeans & Meanshift
• Hierarchical Clustering - Agglomerative
• Density Based Clustering - DBSCAN
• Measuring Performance of Clusters
• Comparing all clustering methods
• Application - Grouping similar customers

10. Anomaly Detection

• What are Outliers ?

• Statistical Methods for Univariate Data
• Using Gaussian Mixture Models
• Fitting an elliptic envelope
• Isolation Forest
• Local Outlier Factor
• Using clustering method like DBSCAN
• Application - Anomaly detection for credit risk prediction

11. Support Vector Machines

• Introduction to Support Vector Machines

• Maximal Margin Classifier
• Soft Margin Classifier
• SVM Algorithm for Classification
• SVM for Regression
• Hyper-parameters in SVM
• Application - Face recognition and breast cancer classification

www.edyoda.com [email protected]
12. Dealing with Imbalanced Classes

• What are imbalanced classes & their impact?

• OverSampling
• UnderSampling
• Connecting Sampler to pipelines
• Making classification algorithm aware of Imbalance
• Anomaly Detection
• Application - Fraud detection

13. Ensemble Methods

• Introduction to Ensemble Methods

• RandomForest
• AdaBoost
• Gradient Boosting Tree
• VotingClassifier
• XGBoost
• Application - Malicious data detection

14. Recommendation Engine

• Understanding distance vector calculation - cosine, euclidean, manhattan

• Types of Recommendation Engines
• Recommendation based on similarity
• Application - Grouping videos based on description, user rating prediction

15. Time Series Modeling

• Simple Average & Moving Average

• Single Exponential Smoothing
• Holt’s linear trend method
• Holt’s winter seasonal method
• ARIMA

16. Packaging & Deployment

• Creating Python Package

• Deploy trained model behind REST interface
• Deploy model behind API call
• Deploy on AWS cloud (optional)

www.edyoda.com [email protected]
 Big Data Ecosystem

1. Introduction to Big Data

• Big Data
• Understanding distributed computing
• Introduction to Hadoop
• HDFS, YARN, MapReduce
• Limitations of Hadoop
• Introduction to Spark
• Introduction to Kafka
• Hive
• Cassandra

2. Internal Details of Spark

• Driver
• Executors
• Partitions
• Jobs
• Stages
• Tasks
• Resilient Distributed Datastructure
• DataFrames as a High Level Datastructure

3. Foundations of Spark using RDD

• Basics of Distributed Computing

• Resilient Distributed Dataset
• Simple Transformers - map,filter,groupby
• Actions - Collect, count, foreach
• Complex api - combinebykey
• Caching, Debugging
• Important Configuration

4. Data Wrangling using DataFrames

• Creating DataFrames from collections

• Creating a DataFrame from csv,json etc.
• DataFrame Row

www.edyoda.com [email protected]
 • DataFrame Column
• Creating tables from dataframe
• SQL query
• DataFrame Grouping
• DataFrame Functions
• User Defined Functions (UDF)

5. Packaging & Deployment of Spark Applications

• The spark-submit command

• Command line parameters
• Deploying the app programmatically
• Configuring your SparkSession
• Modularizing code
• Structure of the module
• Building an egg
• User defined functions in Spark
• Submitting a job
• Monitoring execution

www.edyoda.com [email protected]