1,Write the difference b/n cost function and lost function

and describe it’s purpose in LR?

cost function
 is used for group of data so it is average error for a
group of data.
 It provides a single scalar value that summarizes
the model performance on all data points.
lost function
 is to capture the difference between the actual and
the predicted values for a single record.
 Lost function is only for a single data point.
Their Purpose
 The
NAME SOLOMON cost
ABRHA function
MACHINE for logistic
LEARNING regression
ASSIGNMENT is the
-3 ID 122/12 [Year]
average of log loss over all training examples. It is
often referred to as the cross entropy cost function
and is designed to optimize the parameters to
minimize the prediction error for binary classification
tasks.
 The loss function used in logistic regression is log
loss (logistic loss or cross-entropy loss). It measures
the performance of a classification model whose
output is a probability value between 0 and 1,
penalizing incorrect classifications more heavily.
 The loss function measures the error for a single
training example, while the cost function is the
average of the loss functions over all training
examples. In logistic regression, the cost function is the
 mean of the loss calculated by the logistic loss (log loss
or cross-entropy loss) across the entire dataset.
2,Explain learning rate and how we used in ML?
 The learning rate in machine learning (ML) is a
configuration parameter that determines how
much an algorithm changes its model in
response to the estimated error each time the
model weights are updated . it also defines the
speed at which machine learning model learns.
 Learning rate controls the size of the steps taken towards
minimizing the cost function during the optimization process
(e.g., gradient descent).
 Learning rate in gradient descent algorithms, which aim
to minimize a given cost function, the learning rate
NAME SOLOMON ABRHA MACHINE LEARNING ASSIGNMENT -3 ID 122/12
controls how big the step would be on each iteration. [Year]
When the learning rate is low, the model makes small
updates to the weights and requires more iterations to
converge to the solution, leading to a slow training
process. On the other hand, when the learning rate is
high, the model makes larger updates to the weights,
which might lead the model to overshoot the optimal
solution and may even fail to converge.
?3, What is stochastic gradient descent and
Regularization? Explain with help of mathematics.
 stochastic Gradient Descent (SGD) is an
optimization technique used in machine learning
to minimize errors in predictive models. Unlike
regular gradient descent, which uses the entire
 dataset to calculate the gradient and update
model parameters, SGD updates the parameters
using only one data point at a time.

Regularization
NAME SOLOMON ABRHA is a technique
MACHINE LEARNING used to
ASSIGNMENT -3 prevent
ID 122/12overfitting by
[Year]
adding a penalty term to the cost function. This encourages
the model to keep the weights small, which reduces the
model's complexity and helps it generalize better to unseen
data.
 4,Explain linear and non-leaner model.

Linear models
The output
o ABRHA
NAME SOLOMON changes
MACHINE by the
LEARNING same amount
ASSIGNMENT -3 IDfor any given change
122/12 [Year]
in the input. For example, the equation and also if the
equation gives straight line it is linear model , 𝑦=2𝑥 +5 is
linear, where y is out put and x is an input.
o A linear model assumes a linear relationship between the
input features (X) and the output (y).

Nonlinear models

 A non-linear model captures more complex relationships

between the input features and the output.
 The output changes by different amounts for different
changes in the input. Nonlinear models can capture more
complex relationships than linear models. And also if the
equation give like curve ,circle,parabola or other conics it is a
non linear model.
 5,Write the difference between linear and logistic
regression.

Linear regression

Used for predicting continuous values (regression tasks).

 The output is a continuous value ranging from −∞ to +∞.
 Y=a0 +a1x1+a2x2+………anxn, is its equation
 Does not have a decision boundary; predicts continuous values.
 assumes a linear relationship between input features and output.
 It has Regression problem (continuous data) tasks .
 he goal of the Linear regression is to find the best fit line that
can accurately predict the output for the continuous
dependent variable.
 The output for Linear regression should only be the
continuous values such as price, age, salary, etc. The
relationship between the dependent variable and independent
variable
NAME SOLOMON ABRHAcan be shown
MACHINE in below
LEARNING image.-3 ID 122/12
ASSIGNMENT [Year]
 Logistic regression

 Used for classification tasks (binary or multiclass).

 The output is a probability between 0 and 1.
 Applies a sigmoid function to map inputs to probabilities.
 Predicting spam vs. non-spam, tumor malignancy, etc.
 The predicted value represents the probability of a class.
 Focuses on Classification problem (discrete data) tasks.
 In logistic regression, we pass the weighted sum of inputs
through an activation function that can map values in between
0 and 1. Such activation function is known as sigmoid
function and the curve obtained is called as sigmoid curve or
S-curve. Consider the below image:

NAME SOLOMON ABRHA MACHINE LEARNING ASSIGNMENT -3 ID 122/12 [Year]

6, Explain prediction and classification machine learning

tasks.
 Prediction tasks involve estimating a continuous numerical value
based on input data. These tasks are typically addressed using
regression algorithms.

Key Characteristics:

 The output (target variable) is continuous.

 The goal is to predict values within a certain range (e.g.,
temperature, stock prices).
 The relationship between input features and the target variable can
be linear or non-linear.

Algorithms Used:

Linear Regression
 Polynomial Regression
 Decision Trees (for regression)
 Support Vector Regression (SVR)
 Neural
NAME SOLOMON Networks
ABRHA MACHINE (for regression)
LEARNING ASSIGNMENT -3 ID 122/12 [Year]
2. Classification Tasks

Definition:
Classification tasks involve predicting a categorical label (class) for a
given input. These tasks are commonly addressed using classification
algorithms.

Key Characteristics:

 The output (target variable) is discrete (e.g., binary or multiclass

labels).
 The goal is to assign input data to one or more predefined
categories.
 It often involves learning decision boundaries between classes.

Algorithms Used:
  Logistic Regression
 Decision Trees (for classification)
 Support Vector Machines (SVM)
 k-Nearest Neighbors (k-NN)
 Neural Networks (for classification)

7. Write the difference b/n bias and variance.

Bias is simply defined as the inability of the model because

of that there is some difference or error occurring between
the model’s predicted value and the actual value. These
differences between actual or expected values and the
predicted values are known as error or bias error or error due
to bias. Bias is a systematic error that occurs due to wrong
assumptions in the machine learning process.
 Bias refers to the error due to overly simplistic assumptions in the
model.
 Results from underfitting, where the model fails to capture the
NAME SOLOMON ABRHA MACHINE LEARNING ASSIGNMENT -3 ID 122/12 [Year]
underlying patterns in the data.
 Leads to a model that is too simple and performs poorly on both
training and test data
 High training error and high test error.
 High bias reduces generalization by failing to capture enough
complexity.
 Variance is the measure of spread in data from
its mean position. In machine learning variance is the amount
by which the performance of a predictive model changes
when it is trained on different subsets of the training data.
More specifically, variance is the variability of the model that
how much it is sensitive to another subset of the training
dataset. i.e. how much it can adjust on the new subset of the
training dataset.
 Variance refers to the error caused by the model’s sensitivity to
small fluctuations in the training data.
  Results from overfitting, where the model captures noise or
random fluctuations in the data.
 Leads to a model that performs well on training data but poorly on
test data (low generalization).
 Too flexible; models noise in the training data as if it were a true
pattern
 High variance reduces generalization by focusing too much on
training data.

8, Explain and drive the math’s of the cost function and

gradient decent of logistic regression.

 Logistic regression is used for binary classification problems,

where the goal is to classify data points into one of two classes
(e.g., 0 or 1). It uses a sigmoid function to output probabilities,
which are then thresholded to make predictions.
 Logistic regression predicts the probability of the output y being 1
NAME SOLOMON ABRHA MACHINE LEARNING ASSIGNMENT -3 ID 122/12
given input x, using a sigmoid (logistic) function (mathematical[Year]
expression look at the next image).

 9. How is Machine Learning Different from Statistical
modeling?

Machine learning and statistical modeling are both used to learn

from data, but they differ in their goals, how they use data, and
how interpretable their models are:
Statistical modeling is used to make inferences about a
population based on a sample, while machine learning is used to
make predictions by finding patterns in data.
Machine learning requires large amounts of data to make
accurate predictions, while statistical models don't involve
multiple subsets of data.
Statistical models are typically easier to understand than
machine learning models because they are based on fewer
variables.
NAME SOLOMON ABRHA MACHINE LEARNING ASSIGNMENT -3 ID 122/12 [Year]
Statistical models define mathematical relationships between
variables, while machine learning models enable computers to
learn from data without explicit programming.
Machine learning models have a higher tolerance for uncertainty
than statistical models.
Statistical models struggle with large datasets and become less
reliable as they reach a certain threshold
Machine learning and statistical modeling are both used to learn
from data, but they differ in their goals, how they use data, and
how interpretable their models are:

 Machine Learning: The primary goal is to create algorithms that

allow a system to automatically learn patterns from data and make
predictions or decisions. It's often used for tasks like classification,
regression, clustering, and anomaly detection.
 Machine Learning: It uses algorithms that can automatically improve
their performance with more data, often focusing on prediction
accuracy
 Machine Learning: ML algorithms often make fewer assumptions
about the underlying data distribution. They are flexible and can
work with complex, high-dimensional data without needing explicit
assumptions about data relationships.
 ML generally requires large amounts of data to train models
effectively, as more data helps improve performance.
Statistical Modeling: The goal is to understand the relationship
between variables, often to make inferences or test hypotheses. It
focuses on explaining the data and understanding underlying processes.

Statistical Modeling: It tends to focus more on explaining

relationships between variables (e.g., regression models).
 Statistical models typically require assumptions about the data
(e.g., normality, linearity, independence of errors) and aim to
NAME SOLOMON ABRHA MACHINE LEARNING ASSIGNMENT -3 ID 122/12
estimate parameters that explain the data.
[Year]