KONGU ENGINEERING COLLEGE, PERUNDURAI 638 060

CONTINUOUS ASSESSMENT TEST II

Answerkey
20ADH02 Internet of Things and Machine Learning

PART - A

1. Stochastic gradient optimizer [CO4] [K1]

Adam optimizer
Adagrad optimizer
RMSProp optimizer
2. Crossover [CO4] [K2]
Select a random position on the chromosome of two parents, then the genetic information is
swapped between them about this point, with a probability Px. This results in two new offspring.
Mutation:
Mutation operator helps in maintaining and introducing diversity in the population. The mutation
operator is applied to certain genes (bits) of the child chromosomes with a probability, Pm.

3. AlphaGo Zero [CO4] [K1]

AI-controlled sailplanes
Locomotion behavior
gameplaying
and robotics.
4. Actions are the set of all possible things that the agent can do in a particular state. [CO4] [K1]

5. The simple Q-learning algorithm involves maintaining a table of the size m×n, where m is the total [CO4] [K2]
number of states and n the total number of possible actions. An alternative is to replace the table
with a neural network acting as a function approximator, approximating the Q-function for each
possible action. When the neural network that use to approximate the Q-function is a deep neural
network, called Deep Q-Network (DQN)

6. Generative models are an exciting new branch of deep learning models that learn through [CO5] [K1]
unsupervised learning. The main idea is to generate new samples having the same distribution as the
given training data.

7. Classification: Logistic regression, linear support vector machine, Naive Bayes [CO5] [K2]
Regression: Generalized linear regression

8. Vibration sensors, Current/voltage sensors, Ultrasound analysis, Infrared thermography, Sensors to detect [CO5] [K1]
liquid quality

9. Capturing data at regular intervals where the time-stamp is also part of data. The methods that allow [CO5] [K1]
us to capture inherent patterns in time-dependent data is called time series modeling.

10. Amazon offers almost every feature under the cloud, from a cloud database, to cloud computing [CO5] [K2]
resources, to even cloud analytics. It even provides space to build a secure data lake. Its IoT core
allows users to connect devices to the cloud. It provides a single dashboard that can be used to
control the services you sign for.

Part – B

11. i) Value-based methods: In these methods, the algorithms take the action that maximizes (5) [CO4] [K1]
the value function. The agent here learns to predict how good a given state or action
would be. Hence, here, the aim is to find the optimal value. An example of the value-
 based method is Q-learning. Consider, for example, our RL agent in a maze: assuming
that the value of each state is the negative of the number of steps needed to reach from
that box to the goal, then, at each time step, the agent will choose the action that takes it
to a state with optimal value, as in the following diagram. So, starting from a value of -6,
it'll move to -5, -4, -3, -2, -1, and eventually reach the goal with the value 0:

Policy-based methods: In these methods, the algorithms predict the best policy which
maximizes the value function. The aim is to find the optimal policy. An example of the
policy-based method is policy gradients. Here, we approximate the policy function,
which allows us to map each state to the best corresponding action.
ii) (5) [CO4] [K1]

The convolutional is used to extract features but, in later stages, it's separated into two
separate networks, one providing the value and another providing the advantage. Later, the
two stages are recombined using an aggregating layer to estimate the Q-value. At the
aggregate layer, the value and advantage are combined such that it's possible to recover both
V and A uniquely from a given Q.

12. i) from gym import envs (6) [CO4] [K3]

print(envs.registry.all())
import gym
env = gym.make('Pong-v0')
obs = env.reset()
env.render()
frames = [] # array to store state space at each step
for _ in range(300):
frames.append(env.render(mode='rgb_array'))
obs,reward,done, _ = env.render(env.action_space.sample())
if done:
break

import matplotlib.animation as animation

from JSAnimation.Ipython_display import display_animation
from IPython.display import display
patch = plt.imshow(frames[0])

2
 plt.axis('off')
def animate(i)
patch.set_data(frames[i])
anim = animation.FuncAnimation(plt.gcf(), animate, \
frames=len(frames), interval=100)
display(display_animation(anim, default_mode='loop')

ii) In the policy gradient method, we introduced the baseline to reduce variance, but still, both (4) [CO4] [K1]
action and baseline (look closely: the variance is the expected sum of rewards, or in other
words, the goodness of the state or its value function) were changing simultaneously. It
consists of two neural networks, one approximating the policy, called the actor-network,
and the other approximating the value, called the critic-network. We alternate between a
policy evaluation and a policy improvement step, resulting in more stable learning. The critic
uses the state and action values to estimate a value function, which is then used to update the
actor's policy network parameters so that the overall performance improves.

13. i) A very interesting property of CNNs is that each convolutional layer learns to identify (10) [CO5] [K2]
different features from the image, which is they act as feature extractors. The lower
convolutional layers filter out basic shapes like lines and circles, while
higher layers filter more abstract shapes. This property can be used to employ a CNN trained
on one set of images to classify another set of similar domain images by just changing the top
fully connected layers. This technique is called transfer learning.
The SparkDL module provides an easy and convenient way to use pre-trained models with
the help of the class DeepImageFeaturizer.
InceptionV3
Xception
ResNet50
VGG16
VGG19
Any suitable example code can be given using transfer learning.
14. i) Capturing data at regular intervals where the time-stamp is also part of data. The (10) [CO5] [K2]
methods that allow us to capture inherent patterns in time-dependent data is called time

3
series modeling.
Plotting can be done for any suitable time series data.

Trend means to find whether, on average, the measurements tend to decrease (or increase)
over time. The most common way to find a trend is by plotting a moving average.
Seasonality is the presence of a regularly repeating pattern of highs and lows related to time
(for example, sine series). The easiest way is to find autocorrelation in the data. Once you
find the seasonality, you can remove it by differencing the data by a time lag corresponding
to the season length.
Trend and Seasonality can be visualized using any suitable plot for the data.

Whether the series is stationary, that is, the mean of the series is no longer a function of time.
Stationarity of data is essential for time series modeling. We achieve stationarity by removing
any trends or seasonality present within the data. Once the data is stationary, we can use
regression models to model it.
If a series has stationarity, then it means the data has same distribution across time, and
hence, can be estimated in time.