1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

 Navigation

Click to Take the FREE Deep Learning Crash-Course

Search... 

Your First Deep Learning Project in Python with

Keras Step-By-Step
by Jason Brownlee on July 24, 2019 in Deep Learning

Tweet Share Share

Last Updated on September 15, 2020

Keras is a powerful and easy-to-use free open source Python library for developing and evaluating
deep learning models.

It wraps the efficient numerical computation libraries Theano and TensorFlow and allows you to define
and train neural network models in just a few lines of code.

In this tutorial, you will discover how to create your first deep learning neural network model in Python
using Keras.

Kick-start your project with my new book Deep Learning With Python, including step-by-step tutorials
and the Python source code files for all examples.

Let’s get started.

Update Feb/2017: Updated prediction example so rounding works in Python 2 and 3.

Update Mar/2017: Updated example for the latest versions of Keras and TensorFlow.
Update Mar/2018: Added alternate link to download the dataset.
Update Jul/2019: Expanded and added more useful resources.
Update Sep/2019: Updated for Keras v2.2.5 API.
Update Oct/2019: Updated for Keras v2.3.0 API and TensorFlow v2.0.0.
Update Aug/2020: Updated for Keras v2.4.3 and TensorFlow v2.3.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 1/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Start Machine Learning ×

You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

Email Address

START MY EMAIL COURSE

Develop Your First Neural Network in Python With Keras Step-By-Step

Photo by Phil Whitehouse, some rights reserved.

Keras Tutorial Overview

There is not a lot of code required, but we are going to step over it slowly so that you will know how to
create your own models in the future.

The steps you are going to cover in this tutorial are as follows:

1. Load Data.
2. Define Keras Model.
3. Compile Keras Model.
4. Fit Keras Model. Start Machine Learning
5. Evaluate Keras Model.
6. Tie It All Together.
7. Make Predictions

This Keras tutorial has a few requirements:

1. You have Python 2 or 3 installed and configured.

2. You have SciPy (including NumPy) installed and configured.
3. You have Keras and a backend (Theano or TensorFlow) installed and configured.

If you need help with your environment, see the tutorial:

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 2/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

How to Setup a Python Environment for Deep Learning

Create a new file called keras_first_network.py and type or copy-and-paste the code into the file as
you go.

Need help with Deep Learning in Python?

Take my free 2-week email course and discover MLPs, CNNs and LSTMs (with code).

Click to sign-up now and also get a free PDF Ebook version of the course.

Start Your FREE Mini-Course Now!

Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
1. Load Data
The first step is to define the functions and classes we Email
intendAddress
to use in this tutorial.

We will use the NumPy library to load our dataset and we will use two classes from the Keras library to
START MY EMAIL COURSE
define our model.

The imports required are listed below.

1 # first neural network with keras tutorial

2 from numpy import loadtxt
3 from keras.models import Sequential
4 from keras.layers import Dense
5 ...

We can now load our dataset.

In this Keras tutorial, we are going to use the Pima Indians onset of diabetes dataset. This is a standard
machine learning dataset from the UCI Machine Learning repository. It describes patient medical record
data for Pima Indians and whether they had an onset of diabetes within five years.

As such, it is a binary classification problem (onset of Start

diabetes as 1 Learning
Machine or not as 0). All of the input
variables that describe each patient are numerical. This makes it easy to use directly with neural
networks that expect numerical input and output values, and ideal for our first neural network in Keras.

The dataset is available from here:

Dataset CSV File (pima-indians-diabetes.csv)

Dataset Details

Download the dataset and place it in your local working directory, the same location as your python file.

Save it with the filename:

1 pima-indians-diabetes.csv
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 3/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Take a look inside the file, you should see rows of data like the following:

1 6,148,72,35,0,33.6,0.627,50,1
2 1,85,66,29,0,26.6,0.351,31,0
3 8,183,64,0,0,23.3,0.672,32,1
4 1,89,66,23,94,28.1,0.167,21,0
5 0,137,40,35,168,43.1,2.288,33,1
6 ...

We can now load the file as a matrix of numbers using the NumPy function loadtxt().

There are eight input variables and one output variable (the last column). We will be learning a model to
map rows of input variables (X) to an output variable (y), which we often summarize as y = f(X).

The variables can be summarized as follows:

Input Variables (X):

Start Machine Learning ×
You can master applied Machine Learning
1. Number of times pregnant
without math or fancy degrees.
2. Plasma glucose concentration a 2 hours in an oral glucose
Find out howtolerance
in this freetest
and practical course.
3. Diastolic blood pressure (mm Hg)
4. Triceps skin fold thickness (mm)
Email Address
5. 2-Hour serum insulin (mu U/ml)
6. Body mass index (weight in kg/(height in m)^2)
7. Diabetes pedigree function START MY EMAIL COURSE

8. Age (years)

Output Variables (y):

1. Class variable (0 or 1)

Once the CSV file is loaded into memory, we can split the columns of data into input and output
variables.

The data will be stored in a 2D array where the first dimension is rows and the second dimension is
columns, e.g. [rows, columns].

We can split the array into two arrays by selecting subsets of columns using the standard NumPy slice
operator or “:” We can select the first 8 columns from index 0 to index 7 via the slice 0:8. We can then
select the output column (the 9th variable) via index 8.Start Machine Learning

1 ...
2 # load the dataset
3 dataset = loadtxt('pima-indians-diabetes.csv', delimiter=',')
4 # split into input (X) and output (y) variables
5 X = dataset[:,0:8]
6 y = dataset[:,8]
7 ...

We are now ready to define our neural network model.

Note, the dataset has 9 columns and the range 0:8 will select columns from 0 to 7, stopping before
index 8. If this is new to you, then you can learn more about array slicing and ranges in this post:

How to Index, Slice and Reshape NumPy Arrays for Machine Learning in Python
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 4/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

2. Define Keras Model

Models in Keras are defined as a sequence of layers.

We create a Sequential model and add layers one at a time until we are happy with our network
architecture.

The first thing to get right is to ensure the input layer has the right number of input features. This can be
specified when creating the first layer with the input_dim argument and setting it to 8 for the 8 input
variables.

How do we know the number of layers and their types?

This is a very hard question. There are heuristics thatStart

we can Machine
use and oftenLearning
the best network structure ×
is found through a process of trial and error experimentation (I explain more about this here). Generally,
You can
you need a network large enough to capture the structure master
of the applied Machine Learning
problem.
without math or fancy degrees.
In this example, we will use a fully-connected networkFind out howwith
structure in this free layers.
three and practical course.

Fully connected layers are defined using the Dense class. We

Email can specify the number of neurons or
Address
nodes in the layer as the first argument, and specify the activation function using the activation
argument.
START MY EMAIL COURSE

We will use the rectified linear unit activation function referred to as ReLU on the first two layers and the
Sigmoid function in the output layer.

It used to be the case that Sigmoid and Tanh activation functions were preferred for all layers. These
days, better performance is achieved using the ReLU activation function. We use a sigmoid on the
output layer to ensure our network output is between 0 and 1 and easy to map to either a probability of
class 1 or snap to a hard classification of either class with a default threshold of 0.5.

We can piece it all together by adding each layer:

The model expects rows of data with 8 variables (the input_dim=8 argument)
The first hidden layer has 12 nodes and uses the relu activation function.
The second hidden layer has 8 nodes and uses the relu activation function.
The output layer has one node and uses the sigmoid activation function.
Start Machine Learning
1 ...
2 # define the keras model
3 model = Sequential()
4 model.add(Dense(12, input_dim=8, activation='relu'))
5 model.add(Dense(8, activation='relu'))
6 model.add(Dense(1, activation='sigmoid'))
7 ...

Note, the most confusing thing here is that the shape of the input to the model is defined as an
argument on the first hidden layer. This means that the line of code that adds the first Dense layer is
doing 2 things, defining the input or visible layer and the first hidden layer.

3. Compile Keras Model

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 5/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Now that the model is defined, we can compile it.

Compiling the model uses the efficient numerical libraries under the covers (the so-called backend)
such as Theano or TensorFlow. The backend automatically chooses the best way to represent the
network for training and making predictions to run on your hardware, such as CPU or GPU or even
distributed.

When compiling, we must specify some additional properties required when training the network.
Remember training a network means finding the best set of weights to map inputs to outputs in our
dataset.

We must specify the loss function to use to evaluate a set of weights, the optimizer is used to search
through different weights for the network and any optional metrics we would like to collect and report
during training. Start Machine Learning ×
In this case, we will use cross entropy as the loss argument. This loss
You can master is for
applied a binary
Machine classification
Learning
without math or fancy degrees.
problems and is defined in Keras as “binary_crossentropy“. You can learn more about choosing loss
functions based on your problem here: Find out how in this free and practical course.

How to Choose Loss Functions When Training Deep Learning

Email AddressNeural Networks

We will define the optimizer as the efficient stochastic gradient descent algorithm “adam“. This is a
popular version of gradient descent because it automatically
STARTtunes itselfCOURSE
MY EMAIL and gives good results in a
wide range of problems. To learn more about the Adam version of stochastic gradient descent see the
post:

Gentle Introduction to the Adam Optimization Algorithm for Deep Learning

Finally, because it is a classification problem, we will collect and report the classification accuracy,
defined via the metrics argument.

1 ...
2 # compile the keras model
3 model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
4 ...

4. Fit Keras Model

We have defined our model and compiled it ready for Start
efficient computation.
Machine Learning

Now it is time to execute the model on some data.

We can train or fit our model on our loaded data by calling the fit() function on the model.

Training occurs over epochs and each epoch is split into batches.

Epoch: One pass through all of the rows in the training dataset.
Batch: One or more samples considered by the model within an epoch before weights are
updated.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 6/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

One epoch is comprised of one or more batches, based on the chosen batch size and the model is fit
for many epochs. For more on the difference between epochs and batches, see the post:

What is the Difference Between a Batch and an Epoch in a Neural Network?

The training process will run for a fixed number of iterations through the dataset called epochs, that we
must specify using the epochs argument. We must also set the number of dataset rows that are
considered before the model weights are updated within each epoch, called the batch size and set
using the batch_size argument.

For this problem, we will run for a small number of epochs (150) and use a relatively small batch size of
10.

These configurations can be chosen experimentally by trial and error. We want to train the model
Start Machine Learning ×
enough so that it learns a good (or good enough) mapping of rows of input data to the output
classification. The model will always have some error,You
butcan
themaster
amount of error
applied will level
Machine out after some
Learning
point for a given model configuration. This is called model convergence.
without math or fancy degrees.
Find out how in this free and practical course.
1 ...
2 # fit the keras model on the dataset
3 model.fit(X, y, epochs=150, batch_size=10) Email Address
4 ...

This is where the work happens on your CPU or GPU.

START MY EMAIL COURSE

No GPU is required for this example, but if you’re interested in how to run large models on GPU
hardware cheaply in the cloud, see this post:

How to Setup Amazon AWS EC2 GPUs to Train Keras Deep Learning Models

5. Evaluate Keras Model

We have trained our neural network on the entire dataset and we can evaluate the performance of the
network on the same dataset.

This will only give us an idea of how well we have modeled the dataset (e.g. train accuracy), but no idea
of how well the algorithm might perform on new data. We have done this for simplicity, but ideally, you
could separate your data into train and test datasets for training and evaluation of your model.
Start Machine Learning
You can evaluate your model on your training dataset using the evaluate() function on your model and
pass it the same input and output used to train the model.

This will generate a prediction for each input and output pair and collect scores, including the average
loss and any metrics you have configured, such as accuracy.

The evaluate() function will return a list with two values. The first will be the loss of the model on the
dataset and the second will be the accuracy of the model on the dataset. We are only interested in
reporting the accuracy, so we will ignore the loss value.

1 ...
2 # evaluate the keras model
3 _, accuracy = model.evaluate(X, y)

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 7/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

4 print('Accuracy: %.2f' % (accuracy*100))

6. Tie It All Together

You have just seen how you can easily create your first neural network model in Keras.

Let’s tie it all together into a complete code example.

1 # first neural network with keras tutorial

2 from numpy import loadtxt
3 from keras.models import Sequential
4 from keras.layers import Dense
5 # load the dataset
6 dataset = loadtxt('pima-indians-diabetes.csv', delimiter=',')
7 # split into input (X) and output (y) variables
8
9
X = dataset[:,0:8]
y = dataset[:,8] Start Machine Learning ×
10 # define the keras model
11 model = Sequential() You can master applied Machine Learning
12 model.add(Dense(12, input_dim=8, activation='relu'))
13 model.add(Dense(8, activation='relu')) without math or fancy degrees.
14 model.add(Dense(1, activation='sigmoid')) Find out how in this free and practical course.
15 # compile the keras model
16 model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
17 # fit the keras model on the dataset Email Address
18 model.fit(X, y, epochs=150, batch_size=10)
19 # evaluate the keras model
20 _, accuracy = model.evaluate(X, y)
21 print('Accuracy: %.2f' % (accuracy*100)) START MY EMAIL COURSE

You can copy all of the code into your Python file and save it as “keras_first_network.py” in the same
directory as your data file “pima-indians-diabetes.csv“. You can then run the Python file as a script
from your command line (command prompt) as follows:

1 python keras_first_network.py

Running this example, you should see a message for each of the 150 epochs printing the loss and
accuracy, followed by the final evaluation of the trained model on the training dataset.

It takes about 10 seconds to execute on my workstation running on the CPU.

Ideally, we would like the loss to go to zero and accuracy to go to 1.0 (e.g. 100%). This is not possible
for any but the most trivial machine learning problems. Instead, we will always have some error in our
model. The goal is to choose a model configuration and training configuration that achieve the lowest
loss and highest accuracy possible for a given dataset.Start Machine Learning

1 ...
2 768/768 [==============================] - 0s 63us/step - loss: 0.4817 - acc: 0.7708
3 Epoch 147/150
4 768/768 [==============================] - 0s 63us/step - loss: 0.4764 - acc: 0.7747
5 Epoch 148/150
6 768/768 [==============================] - 0s 63us/step - loss: 0.4737 - acc: 0.7682
7 Epoch 149/150
8 768/768 [==============================] - 0s 64us/step - loss: 0.4730 - acc: 0.7747
9 Epoch 150/150
10 768/768 [==============================] - 0s 63us/step - loss: 0.4754 - acc: 0.7799
11 768/768 [==============================] - 0s 38us/step
12 Accuracy: 76.56

Note, if you try running this example in an IPython or Jupyter notebook you may get an error.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 8/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

The reason is the output progress bars during training. You can easily turn these off by setting
verbose=0 in the call to the fit() and evaluate() functions, for example:

1 ...
2 # fit the keras model on the dataset without progress bars
3 model.fit(X, y, epochs=150, batch_size=10, verbose=0)
4 # evaluate the keras model
5 _, accuracy = model.evaluate(X, y, verbose=0)
6 ...

Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or
differences in numerical precision. Consider running the example a few times and compare the average
outcome.

What score did you get?

Post your results in the comments below. Start Machine Learning ×
Neural networks are a stochastic algorithm, meaning You thatcan
themaster
same applied Machine
algorithm Learning
on the same data can
without math or fancy degrees.
train a different model with different skill each time the code is run. This is a feature, not a bug. You can
Find out how in this free and practical course.
learn more about this in the post:

Embrace Randomness in Machine Learning Email Address

The variance in the performance of the model means that to get a reasonable approximation of how
START MY EMAIL COURSE
well your model is performing, you may need to fit it many times and calculate the average of the
accuracy scores. For more on this approach to evaluating neural networks, see the post:

How to Evaluate the Skill of Deep Learning Models

For example, below are the accuracy scores from re-running the example 5 times:

1 Accuracy: 75.00
2 Accuracy: 77.73
3 Accuracy: 77.60
4 Accuracy: 78.12
5 Accuracy: 76.17

We can see that all accuracy scores are around 77% and the average is 76.924%.

7. Make Predictions
Start Machine Learning
The number one question I get asked is:

After I train my model, how can I use it to make predictions on new data?

Great question.

We can adapt the above example and use it to generate predictions on the training dataset, pretending
it is a new dataset we have not seen before.

Making predictions is as easy as calling the predict() function on the model. We are using a sigmoid
activation function on the output layer, so the predictions will be a probability in the range between 0

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 9/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

and 1. We can easily convert them into a crisp binary prediction for this classification task by rounding
them.

For example:

1 ...
2 # make probability predictions with the model
3 predictions = model.predict(X)
4 # round predictions
5 rounded = [round(x[0]) for x in predictions]

Alternately, we can call the predict_classes() function on the model to predict crisp classes directly, for
example:

1 ...
2 # make class predictions with the model
3 predictions = model.predict_classes(X) Start Machine Learning ×
The complete example below makes predictions for each example
You can master in the dataset,
applied Machine then prints the input
Learning
data, predicted class and expected class for the first 5without
examples
mathinorthe dataset.
fancy degrees.
Find out how in this free and practical course.
1 # first neural network with keras make predictions
2 from numpy import loadtxt
3 from keras.models import Sequential Email Address
4 from keras.layers import Dense
5 # load the dataset
6 dataset = loadtxt('pima-indians-diabetes.csv', delimiter=',')
START MY EMAIL COURSE
7 # split into input (X) and output (y) variables
8 X = dataset[:,0:8]
9 y = dataset[:,8]
10 # define the keras model
11 model = Sequential()
12 model.add(Dense(12, input_dim=8, activation='relu'))
13 model.add(Dense(8, activation='relu'))
14 model.add(Dense(1, activation='sigmoid'))
15 # compile the keras model
16 model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
17 # fit the keras model on the dataset
18 model.fit(X, y, epochs=150, batch_size=10, verbose=0)
19 # make class predictions with the model
20 predictions = model.predict_classes(X)
21 # summarize the first 5 cases
22 for i in range(5):
23 print('%s => %d (expected %d)' % (X[i].tolist(), predictions[i], y[i]))

Running the example does not show the progress bar as before as we have set the verbose argument
to 0.
Start Machine Learning
After the model is fit, predictions are made for all examples in the dataset, and the input rows and
predicted class value for the first 5 examples is printed and compared to the expected class value.

We can see that most rows are correctly predicted. In fact, we would expect about 76.9% of the rows to
be correctly predicted based on our estimated performance of the model in the previous section.

1 [6.0, 148.0, 72.0, 35.0, 0.0, 33.6, 0.627, 50.0] => 0 (expected 1)
2 [1.0, 85.0, 66.0, 29.0, 0.0, 26.6, 0.351, 31.0] => 0 (expected 0)
3 [8.0, 183.0, 64.0, 0.0, 0.0, 23.3, 0.672, 32.0] => 1 (expected 1)
4 [1.0, 89.0, 66.0, 23.0, 94.0, 28.1, 0.167, 21.0] => 0 (expected 0)
5 [0.0, 137.0, 40.0, 35.0, 168.0, 43.1, 2.288, 33.0] => 1 (expected 1)

If you would like to know more about how to make predictions with Keras models, see the post:

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 10/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

How to Make Predictions with Keras

Keras Tutorial Summary

In this post, you discovered how to create your first neural network model using the powerful Keras
Python library for deep learning.

Specifically, you learned the six key steps in using Keras to create a neural network or deep learning
model, step-by-step including:

1. How to load data.

2. How to define a neural network in Keras.
3. How to compile a Keras model using the efficient numerical backend.
4. How to train a model on data. Start Machine Learning ×
5. How to evaluate a model on data.
6. How to make predictions with the model. You can master applied Machine Learning
without math or fancy degrees.
Do you have any questions about Keras or about thisFind out how in this free and practical course.
tutorial?
Ask your question in the comments and I will do my best to answer.
Email Address
Keras Tutorial Extensions
START MY EMAIL COURSE
Well done, you have successfully developed your first neural network using the Keras deep learning
library in Python.

This section provides some extensions to this tutorial that you might want to explore.

Tune the Model. Change the configuration of the model or training process and see if you can
improve the performance of the model, e.g. achieve better than 76% accuracy.
Save the Model. Update the tutorial to save the model to file, then load it later and use it to make
predictions (see this tutorial).
Summarize the Model. Update the tutorial to summarize the model and create a plot of model
layers (see this tutorial).
Separate Train and Test Datasets. Split the loaded dataset into a train and test set (split based on
rows) and use one set to train the model and the other set to estimate the performance of the
model on new data.
Plot Learning Curves. The fit() function returns a history
Start objectLearning
Machine that summarizes the loss and
accuracy at the end of each epoch. Create line plots of this data, called learning curves (see this
tutorial).
Learn a New Dataset. Update the tutorial to use a different tabular dataset, perhaps from the UCI
Machine Learning Repository.
Use Functional API. Update the tutorial to use the Keras Functional API for defining the model
(see this tutorial).

Further Reading
Are you looking for some more Deep Learning tutorials with Python and Keras?

Take a look at some of these:

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 11/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Related Tutorials
5 Step Life-Cycle for Neural Network Models in Keras
Multi-Class Classification Tutorial with the Keras Deep Learning Library
Regression Tutorial with the Keras Deep Learning Library in Python
How to Grid Search Hyperparameters for Deep Learning Models in Python With Keras

Books
Deep Learning (Textbook), 2016.
Deep Learning with Python (my book).

APIs
Keras Deep Learning Library Homepage Start Machine Learning ×
Keras API Documentation
You can master applied Machine Learning
without math or fancy degrees.
How did you go? Do you have any questions about deep learning?
Find out how in this free and practical course.
Post your questions in the comments below and I will do my best to help.

Email Address

Develop Deep Learning Projects with

START MY EMAIL Python!
COURSE

What If You Could Develop A Network in Minutes

...with just a few lines of Python

Discover how in my new Ebook:

Deep Learning With Python

It covers end-to-end projects on topics like:

Multilayer Perceptrons, Convolutional Nets and Recurrent Neural Nets, and
more...

Finally Bring Deep Learning To

Your Own Projects
Skip the Academics. Just Results.

Start Machine Learning

SEE WHAT'S INSIDE

Tweet Share Share

About Jason Brownlee

Jason Brownlee, PhD is a machine learning specialist who teaches developers how to get results
with modern machine learning methods via hands-on tutorials.
View all posts by Jason Brownlee →

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 12/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

 How to Develop an Information Maximizing GAN (InfoGAN) in Keras

How to Implement a Semi-Supervised GAN (SGAN) From Scratch in Keras 

994 Responses to Your First Deep Learning Project in Python with Keras
Step-By-Step

REPLY 
Saurav May 27, 2016 at 11:08 pm #

The input layer doesn’t have any activation function, but still activation=”relu” is mentioned in
the first layer of the model. Why?
Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees.
Jason Brownlee May 28, 2016 at 6:32 am # Find out how in this free and practical course. REPLY 

Hi Saurav,
Email Address
The first layer in the network here is technically a hidden layer, hence it has an activation function.

START MY EMAIL COURSE

REPLY 
sam Johnson December 21, 2016 at 2:44 am #

Why have you made it a hidden layer though? the input layer is not usually represented
as a hidden layer?

REPLY 
Jason Brownlee December 21, 2016 at 8:41 am #

Hi sam,

Note this line:

1 model.add(Dense(12, input_dim=8, init='uniform', activation='relu'))

Start Machine Learning
It does a few things.

It defines the input layer as having 8 inputs.

It defines a hidden layer with 12 neurons, connected to the input layer that use relu
activation function.
It initializes all weights using a sample of uniform random numbers.
Does that help?

Pavidevi May 17, 2017 at 2:31 am #

Hi Jason,

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 13/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

U have used two different activation functions so how can we know which activation
function fit the model?

Jason Brownlee May 17, 2017 at 8:38 am #

Sorry, I don’t understand the question.

Marco Cheung August 23, 2017 at 12:51 am #

Hi Jason,
Start Machine Learning ×
I am interested in deep learning and machine learning. You mentioned “It defines a
hidden layer with 12 neurons, connected
Youto
canthe input applied
master layer that use relu
Machine activation
Learning
function.” I wonder how can we determine themath
without number of neurons
or fancy in order to achieve a
degrees.
high accuracy rate of the model? Find out how in this free and practical course.

Thanks a lot!!!
Email Address

START MY EMAIL COURSE

Jason Brownlee August 23, 2017 at 6:55 am #

Use trial and error. We cannot specify the “best” number of neurons analytically.
We must test.

Ramzan Shahid November 10, 2017 at 4:32 am #

Sir, thanks for your tutorial. Would you like to make tutorial on stock Data
Prediction through Neural Network Model and training this on any stock data. If you have
on this so please share the link. Thanks

Start
Jason Brownlee November 10, 2017Machine
at 10:39 amLearning
#

I am reticent to post tutorials on stock market prediction given the random walk
hypothesis of security prices:
https://machinelearningmastery.com/gentle-introduction-random-walk-times-series-
forecasting-python/

Dhara Bhavsar August 28, 2019 at 9:54 pm #

Hi,

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 14/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

I would like to know more about activation function. How it is working? How many
activation functions? Using different activation function How much affect the output of
the model?

I would like to also know about the Hidden Layer. How the size of the hidden layer affect
the model?

Jason Brownlee August 29, 2019 at 6:09 am #

In this tutorial, we use relu in the hidden layers, learn more here:
https://machinelearningmastery.com/rectified-linear-activation-function-for-deep-
learning-neural-networks/
Start Machine Learning
The size of the layer impacts the capacity of the model, learn more here:
×
https://machinelearningmastery.com/how-to-control-neural-network-model-capacity-with-
You can master applied Machine Learning
nodes-and-layers/ without math or fancy degrees.
Find out how in this free and practical course.

Email Address REPLY 

dhani June 28, 2018 at 2:44 am #

hi how use cnn for pixel classification on mhd images

START MY EMAIL COURSE

REPLY 
Jason Brownlee June 28, 2018 at 6:22 am #

What is pixel classification? What are mhd images?

REPLY 
Tanmay Kulkarni February 11, 2020 at 5:50 am #

Hello! I want to know if there’s a way to know the values of all weights after each
updation?

Start Machine Learning

REPLY 
Jason Brownlee February 11, 2020 at 5:53 am #

Yes, you can save them to file or review them manually.

Often saving is achieved using a checkpoint:

https://machinelearningmastery.com/check-point-deep-learning-models-keras/

REPLY 
BlackBookKeeper August 18, 2018 at 10:15 pm #

runfile(‘C:/Users/Owner/Documents/untitled1.py’, wdir=’C:/Users/Owner/Documents’)
Traceback (most recent call last):
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 15/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

File “”, line 1, in

runfile(‘C:/Users/Owner/Documents/untitled1.py’, wdir=’C:/Users/Owner/Documents’)

File “C:\Users\Owner\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py”, line 705, in

runfile
execfile(filename, namespace)

File “C:\Users\Owner\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py”, line 102, in

execfile
exec(compile(f.read(), filename, ‘exec’), namespace)

File “C:/Users/Owner/Documents/untitled1.py”, line 13, in

model.add(Dense(12, input_dim=8, activation=’relu’))

File “C:\Users\Owner\Anaconda3\lib\site-packages\keras\engine\sequential.py”, line 160, in add

name=layer.name + ‘_input’)
Start Machine Learning ×
File “C:\Users\Owner\Anaconda3\lib\site-packages\keras\engine\input_layer.py”, line 177, in Input
You can master applied Machine Learning
input_tensor=tensor)
without math or fancy degrees.
File “C:\Users\Owner\Anaconda3\lib\site-packages\keras\legacy\interfaces.py”, line 91,
Find out how in this free and practical in wrapper
course.
return func(*args, **kwargs)

Email Address
File “C:\Users\Owner\Anaconda3\lib\site-packages\keras\engine\input_layer.py”, line 86, in __init__
name=self.name)

START MY EMAIL COURSE

File “C:\Users\Owner\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py”, line 515,
in placeholder
x = tf.placeholder(dtype, shape=shape, name=name)

File “C:\Users\Owner\AppData\Roaming\Python\Python36\site-
packages\tensorflow\python\ops\array_ops.py”, line 1530, in placeholder
return gen_array_ops._placeholder(dtype=dtype, shape=shape, name=name)

File “C:\Users\Owner\AppData\Roaming\Python\Python36\site-
packages\tensorflow\python\ops\gen_array_ops.py”, line 1954, in _placeholder
name=name)

File “C:\Users\Owner\AppData\Roaming\Python\Python36\site-
packages\tensorflow\python\framework\op_def_library.py”, line 767, in apply_op
op_def=op_def)

File “C:\Users\Owner\AppData\Roaming\Python\Python36\site-
Start
packages\tensorflow\python\framework\ops.py”, line Machine
2508, Learning
in create_op
set_shapes_for_outputs(ret)

File “C:\Users\Owner\AppData\Roaming\Python\Python36\site-
packages\tensorflow\python\framework\ops.py”, line 1894, in set_shapes_for_outputs
output.set_shape(s)

File “C:\Users\Owner\AppData\Roaming\Python\Python36\site-
packages\tensorflow\python\framework\ops.py”, line 443, in set_shape
self._shape = self._shape.merge_with(shape)

File “C:\Users\Owner\AppData\Roaming\Python\Python36\site-
packages\tensorflow\python\framework\tensor_shape.py”, line 550, in merge_with
stop = key.stop

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 16/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

File “C:\Users\Owner\AppData\Roaming\Python\Python36\site-
packages\tensorflow\python\framework\tensor_shape.py”, line 798, in as_shape
“””Returns this shape as a TensorShapeProto.”””

File “C:\Users\Owner\AppData\Roaming\Python\Python36\site-
packages\tensorflow\python\framework\tensor_shape.py”, line 431, in __init__
size for one or more dimension. e.g. TensorShape([None, 256])

File “C:\Users\Owner\AppData\Roaming\Python\Python36\site-
packages\tensorflow\python\framework\tensor_shape.py”, line 376, in as_dimension
other = as_dimension(other)

File “C:\Users\Owner\AppData\Roaming\Python\Python36\site-
packages\tensorflow\python\framework\tensor_shape.py”, line 32, in __init__
if value is None:
Start Machine Learning ×
TypeError: int() argument must be a string, a bytes-like object or a number, not ‘TensorShapeProto’
You can master applied Machine Learning
this error occurs when {model.add(Dense(12, input_dim=8, activation=’relu’))} this command is run
without math or fancy degrees.
any help? Find out how in this free and practical course.

Email Address
REPLY 
Jason Brownlee August 19, 2018 at 6:20 am #
START MY EMAIL COURSE
Save all code into a file and run it as follows:
https://machinelearningmastery.com/faq/single-faq/how-do-i-run-a-script-from-the-command-line

REPLY 
Penchalaiah December 8, 2019 at 6:24 pm #

Fantastic tutorial. The explanation is simple and precise. Thanks a lot

REPLY 
Jason Brownlee December 9, 2019 at 6:47 am #

Thanks!

Start Machine Learning

REPLY 
Geoff May 29, 2016 at 6:18 am #

Can you explain how to implement weight regularization into the layers?

REPLY 
Jason Brownlee June 15, 2016 at 5:50 am #

Yep, see here:

http://keras.io/regularizers/

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 17/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
afthab October 5, 2018 at 8:32 pm #

hey yo!!! how u r start coding in python

REPLY 
Jason Brownlee October 6, 2018 at 5:43 am #

Start here:
https://machinelearningmastery.com/faq/single-faq/how-do-i-get-started-with-python-
programming

Start Machine Learning ×

REPLY 
KWC June 14, 2016 at 12:08 pm # You can master applied Machine Learning
without math or fancy degrees.
Import statements if others need them: Find out how in this free and practical course.

from keras.models import Sequential

from keras.layers import Dense, Activation Email Address

START MY EMAIL COURSE

REPLY 
Jason Brownlee June 15, 2016 at 5:49 am #

Thanks.

I had them in Part 6, but I have also added them to Part 1.

REPLY 
Shiran January 20, 2020 at 11:30 am #

Great post!
Is it possible to train a neural network that receives as input a vector x and tries to predict
another vector y where both x and y are floats?

Start Machine Learning

REPLY 
Jason Brownlee January 20, 2020 at 2:07 pm #

Yes, this is called regression:

https://machinelearningmastery.com/regression-tutorial-keras-deep-learning-library-python/

REPLY 
Aakash Nain June 29, 2016 at 6:00 pm #

If there are 8 inputs for the first layer then why we have taken them as ’12’ in the following line :

model.add(Dense(12, input_dim=8, init=’uniform’, activation=’relu’))

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 18/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee June 30, 2016 at 6:47 am #

Hi Aakash.

The input layer is defined by the input_dim parameter, here set to 8.

The first hidden layer has 12 neurons.

REPLY 
Joshua July 2, 2016 at 12:04 am #

I ran your program and i have an error:

ValueError: could not convert string to float: Start Machine Learning ×
what could be the reason for this, and how may I solve it.
You can master applied Machine Learning
thanks.
without math or fancy degrees.
great post by the way.
Find out how in this free and practical course.

Email Address
REPLY 
Jason Brownlee July 2, 2016 at 6:20 am #
START MY EMAIL COURSE
It might be a copy-paste error. Perhaps try to copy and run the whole example listed in
section 6?

REPLY 
Akash September 28, 2018 at 11:12 am #

Hello sir, I am facing the same problem valueError: could not convert string to float: ‘”6’
also I am running the example from section 6.

REPLY 
Jason Brownlee September 28, 2018 at 3:00 pm #

I have some suggestions here:

Start Machine Learning
https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-
work-for-me

REPLY 
yashu October 5, 2018 at 8:28 pm #

jason can u plzz help me how to code

REPLY 
Jason Brownlee October 6, 2018 at 5:42 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 19/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Sorry, I cannot help you to write code.

REPLY 
KeyChy July 3, 2019 at 5:45 pm #

Maybe when you set all parameters in an extra column in your *.csv file. Than you schould
replace the delimiter from , to ; like:
dataset = numpy.loadtxt(“pima-indians-diabetes.csv”, delimiter=”;”)
This solved the Problem for me.

×
REPLY 
Jason Brownlee July 4, 2019 at 7:40 am #
Start Machine Learning
Thanks for sharing.
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
cheikh brahim July 5, 2016 at 7:40 pm #
Email Address
thank you for your simple and useful example.

START MY EMAIL COURSE

REPLY 
Jason Brownlee July 6, 2016 at 6:22 am #

You’re welcome cheikh.

REPLY 
Nikhil Thakur July 6, 2016 at 6:39 pm #

Hello Sir, I am trying to use Keras for NLP , specifically sentence classification. I have given the
model building part below. It’s taking quite a lot time to execute. I am using Pycharm IDE.

batch_size = 32
nb_filter = 250
filter_length = 3
Start Machine Learning
nb_epoch = 2
pool_length = 2
output_dim = 5
hidden_dims = 250

# Build the model

model1 = Sequential()

model1.add(Convolution1D(nb_filter, filter_length ,activation=’relu’,border_mode=’valid’,

input_shape=(len(embb_weights),dim), weights=[embb_weights]))

model1.add(Dense(hidden_dims))
model1.add(Dropout(0.2))

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 20/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

model1.add(Activation(‘relu’))

model1.add(MaxPooling1D(pool_length=pool_length))

model1.add(Dense(output_dim, activation=’sigmoid’))

sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)

model1.compile(loss=’mean_squared_error’,
optimizer=sgd,
metrics=[‘accuracy’])

REPLY 
Jason Brownlee July 7, 2016 at 7:31 am #

You may want a larger network. You mayStart Machine

also want Learning
to use a standard repeating structure
×
like CNN->CNN->Pool->Dense.
You can master applied Machine Learning
See this post on using a CNN: without math or fancy degrees.
Find out how in this free and practical course.
http://machinelearningmastery.com/handwritten-digit-recognition-using-convolutional-neural-
networks-python-keras/
Email Address
Later, you may also want to try some stacked LSTMs.

START MY EMAIL COURSE

REPLY 
Andre Norman July 15, 2016 at 10:40 am #

Hi Jason, thanks for the awesome example. Given that the accuracy of this model is 79.56%.
From here on, what steps would you take to improve the accuracy?

Given my nascent understanding of Machine Learning, my initial approach would have been:

Implement forward propagation, then compute the cost function, then implement back propagation, use
gradient checking to evaluate my network (disable after use), then use gradient descent.

However, this approach seems arduous compared to using Keras. Thanks for your response.

REPLY 
Jason Brownlee July 15, 2016 at 10:52 am #
Start Machine Learning
Hi Andre, indeed Keras makes working with neural nets so much easier. Fun even!

We may be maxing out on this problem, but here is some general advice for lifting performance.
– data prep – try lots of different views of the problem and see which is best at exposing the
structure of the problem to the learning algorithm (data transforms, feature engineering, etc.)
– algorithm selection – try lots of algorithms and see which one or few are best on the problem (try
on all views)
– algorithm tuning – tune well performing algorithms to get the most out of them (grid search or
random search hyperparameter tuning)
– ensembles – combine predictions from multiple algorithms (stacking, boosting, bagging, etc.)

For neural nets, there are a lot of things to tune, I think there are big gains in trying different network
topologies (layers and number of neurons per layer) in concert with training epochs and learning rate

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 21/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

(bigger nets need more training).

I hope that helps as a start.

REPLY 
Andre Norman July 18, 2016 at 7:19 am #

Awesome! Thanks Jason =)

REPLY 
Jason Brownlee July 18, 2016 at 8:03 am #

You’re welcome Andre.

Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees.
REPLY 
quentin August 7, 2017 at 8:41 pm # Find out how in this free and practical course.

Some interesting stuff here

Email Address
https://youtu.be/vq2nnJ4g6N0

START MY EMAIL COURSE

REPLY 
Jason Brownlee August 8, 2017 at 7:49 am #

Thanks for sharing. What did you like about it?

REPLY 
Romilly Cocking July 21, 2016 at 12:31 am #

Hi Jason, it’s a great example but if anyone runs it in an IPython/Jupyter notebook they are
likely to encounter an I/O error when running the fit step. This is due to a known bug in IPython.

The solution is to set verbose=0 like this

# Fit the model

model.fit(X, Y, nb_epoch=40, batch_size=10, verbose=0)
Start Machine Learning

REPLY 
Jason Brownlee July 21, 2016 at 5:36 am #

Great, thanks for sharing Romilly.

REPLY 
Anirban July 23, 2016 at 10:20 pm #

Great example. Have a query though. How do I now give a input and get the output (0 or 1).
Can you pls give the cmd for that.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 22/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Thanks

REPLY 
Jason Brownlee July 24, 2016 at 6:53 am #

You can call model.predict() to get predictions and round on each value to snap to a binary
value.

For example, below is a complete example showing you how to round the predictions and print them
to console.

1 # Create first network with Keras

2 from keras.models import Sequential
3 from keras.layers import Dense
4
5
import numpy
# fix random seed for reproducibility Start Machine Learning ×
6 seed = 7
7 numpy.random.seed(seed) You can master applied Machine Learning
8 # load pima indians dataset
without math or fancy degrees.
9 dataset = numpy.loadtxt("pima-indians-diabetes.csv", delimiter=",")
10 Find out how in this free and practical course.
# split into input (X) and output (Y) variables
11 X = dataset[:,0:8]
12 Y = dataset[:,8]
13 # create model Email Address
14 model = Sequential()
15 model.add(Dense(12, input_dim=8, init='uniform', activation='relu'))
16 model.add(Dense(8, init='uniform', activation='relu'))
START MY EMAIL COURSE
17 model.add(Dense(1, init='uniform', activation='sigmoid'))
18 # Compile model
19 model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
20 # Fit the model
21 model.fit(X, Y, nb_epoch=150, batch_size=10, verbose=2)
22 # calculate predictions
23 predictions = model.predict(X)
24 # round predictions
25 rounded = [round(x) for x in predictions]
26 print(rounded)

REPLY 
Debanjan March 27, 2017 at 12:04 pm #

Hi, Why you are not using any test set? You are predicting from the training set , I think.

Start Machine Learning

REPLY 
Jason Brownlee March 28, 2017 at 8:19 am #

Correct, it is just an example to get you started with Keras.

REPLY 
David June 26, 2017 at 12:24 am #

Jason, I’m not quite understanding how the predicted values ([1.0, 0.0, 1.0, 0.0, 1.0,…)
map to the real world problem. For instance, what does that first “1.0” in the results indicate?

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 23/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

I get that it’s a prediction of ‘true’ for diabetes…but to which patient is it predicting that—the first
in the list? So then the second result, “0.0,” is the prediction for the second patient/row in the
dataset?

REPLY 
Jason Brownlee June 26, 2017 at 6:08 am #

Remember the original file has 0 and 1 values in the final class column where 0 is
no onset of diabetes and 1 is an onset of diabetes.

We are predicting new values in this column.

We are making predictions for special rows, we pass in their medical info and predict the
onset of diabetes. We just happen to do this for a number of rows at a time.
Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees.
ami July 16, 2018 at 4:30 pm # Find out how in this free and practical course.

hello jason
Email Address
i am getting this error while calculating the predictions.

#calculate predictions
START MY EMAIL COURSE
predictions = model.predict(X)

#round predictions

rounded = [round(x) for x in predictions]

print(rounded)

—————————————————————————
TypeError Traceback (most recent call last)
in ()
2 predictions = model.predict(X)
3 #round predictions
—-> 4 rounded = [round(x) for x in predictions]
5 print(rounded)

in (.0)
Start Machine Learning
2 predictions = model.predict(X)
3 #round predictions
—-> 4 rounded = [round(x) for x in predictions]
5 print(rounded)

TypeError: type numpy.ndarray doesn’t define __round__ method

Jason Brownlee July 17, 2018 at 6:09 am #

Try removing the call to round().

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 24/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Rachel June 28, 2017 at 8:28 pm #

Hi Jason,
Can I ask why you use the same data X you fit the model to do the prediction?

# Fit the model

model.fit(X, Y, epochs = 150, batch_size = 10, verbose = 2)

# calculate predictions
predictions = model.predict(X)

Rachel

Start Machine Learning ×

REPLY 
Jason Brownlee June 29, 2017 at 6:34 am #
You can master applied Machine Learning
It is all I have at hand. X means data matrix.
without math or fancy degrees.
Find out how in this free and practical course.
Replace X in predict() with Xprime or whatever you like.

Email Address

REPLY 
jitendra March 27, 2018 at 7:20 pm # START MY EMAIL COURSE

hii, how will i feed the input (8,125,96,0,0,0.0,0.232,54) to get our output.

predictions = model.predict(X)
i mean insead of X i want to get output of 8,125,96,0,0,0.0,0.232,54.

REPLY 
Jason Brownlee March 28, 2018 at 6:24 am #

Wrap your input in an array, n-columns with one row, then pass that to the model.

Does that help?

Roman October 5, 2018 at 11:22 Start

pm # Machine Learning

Hello, trying to use predictions on similar neural network but keep getting errors
that input dimension has other shape.

Can you say how array must look on exampled neural network?

Jason Brownlee October 6, 2018 at 5:45 am #

For an MLP, data must be organized into a 2d array of samples x features

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 25/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Anirban July 23, 2016 at 10:52 pm #

I am not able to get to the last epoch. Getting error before that:
Epoch 11/150
390/768 [==============>……………]Traceback (most recent call last):.6921

ValueError: I/O operation on closed file

I could resolve this by varying the epoch and batch size.

Now to predict a unknown value, i loaded a new dataset and used predict cmd as below :
dataset_test = numpy.loadtxt(“pima-indians-diabetes_test.csv”,delimiter=”,”) –has only one row

X = dataset_test[:,0:8]
model.predict(X)
Start Machine Learning ×
But I am getting error :
X = dataset_test[:,0:8] You can master applied Machine Learning
without math or fancy degrees.
IndexError: too many indices for array
Find out how in this free and practical course.
Can you help pls.

Thanks Email Address

START MY EMAIL COURSE

REPLY 
Jason Brownlee July 24, 2016 at 6:55 am #

I see problems like this when you run from a notebook or from an IDE.

Consider running examples from the console to ensure they work.

Consider tuning off verbose output (verbose=0 in the call to fit()) to disable the progress bar.

REPLY 
David Kluszczynski July 28, 2016 at 12:42 am #

Hi Jason!
Loved the tutorial! I have a question however.
Is there a way to save the weights to a file after the model is trained for uses, such as kaggle?
Thanks, Start Machine Learning
David

REPLY 
Jason Brownlee July 28, 2016 at 5:47 am #

Thanks David.

You can save the network weights to file by calling model.save_weights(“model.h5”)

You can learn more in this post:

http://machinelearningmastery.com/save-load-keras-deep-learning-models/

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 26/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Alex Hopper July 29, 2016 at 5:45 am #

Hey, Jason! Thank you for the awesome tutorial! I’ve use your tutorial to learn about CNN. I
have one question for you… Supposing I want to use Keras to classicate images and I have 3 or more
classes to classify, How could my algorithm know about this classes? You know, I have to code what is a
cat, a dog and a horse. Is there any way to code this? I’ve tried it:

target_names = [‘class 0(Cats)’, ‘class 1(Dogs)’, ‘class 2(Horse)’]

print(classification_report(np.argmax(Y_test,axis=1), y_pred,target_names=target_names))

But my results are not classifying correctly.

precision recall f1-score support

class 0(Cat) 0.00 0.00 0.00 17
Start Machine Learning ×
class 1(Dog) 0.00 0.00 0.00 14
class 2(Horse) 0.99 1.00 0.99 2526 You can master applied Machine Learning
avg / total 0.98 0.99 0.98 2557 without math or fancy degrees.
Find out how in this free and practical course.

Email Address
REPLY 
Jason Brownlee July 29, 2016 at 6:41 am #

Great question Alex. START MY EMAIL COURSE

This is an example of a multi-class classification problem. You must use a one hot encoding on the
output variable to be able to model it with a neural network and specify the number of classes as the
number of outputs on the final layer of your network.

I provide a tutorial with the famous iris dataset that has 3 output classes here:
http://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/

REPLY 
Alex Hopper August 1, 2016 at 1:22 am #

Thank you.
I’ll check it.

Start Machine Learning

REPLY 
Jason Brownlee August 1, 2016 at 6:25 am #

No problem Alex.

REPLY 
Anonymouse August 2, 2016 at 11:28 pm #

This was really useful, thank you

I’m using keras (with CNNs) for sentiment classification of documents and I’d like to improve the
performance, but I’m completely at a loss when it comes to tuning the parameters in a non-arbitrary way.
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 27/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Could you maybe point me somewhere that will help me go about this in a more systematic fashion?
There must be some heuristics or rules-of-thumb that could guide me.

REPLY 
Jason Brownlee August 3, 2016 at 8:09 am #

I have a tutorial coming out soon (next week) that provide lots of examples of tuning the
hyperparameters of a neural network in Keras, but limited to MLPs.

For CNNs, I would advise tuning the number of repeating layers (conv + max pool), the number of
filters in repeating block, and the number and size of dense layers at the predicting part of your
network. Also consider using some fixed layers from pre-trained models as the start of your network
(e.g. VGG) and try just training some input and output layers around it for your problem.
Start Machine Learning ×
I hope that helps as a start.
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Shopon August 14, 2016 at 5:04 pm #

Email
Hello Jason , My Accuracy is : 0.0104 , but yours Address
is 0.7879 and my loss is : -9.5414 . Is there
any problem with the dataset ? I downloaded the dataset from a different site .

START MY EMAIL COURSE

REPLY 
Jason Brownlee August 15, 2016 at 12:36 pm #

I think there might be something wrong with your implementation or your dataset. Your
numbers are way out.

REPLY 
mohamed August 15, 2016 at 9:30 am #

after training, how i can use the trained model on new sample

REPLY 
Jason Brownlee August 15, 2016 at 12:36 pmStart
# Machine Learning

You can call model.predict()

See an above comment for a specific code example.

REPLY 
Omachi Okolo August 16, 2016 at 10:21 pm #

Hi Jason,
i’m a student conducting a research on how to use artificial neural network to predict the business
viability of potential software projects.
I intend to use python as a programming language. The application of ANN fascinates me but i’m new to
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 28/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

machine learning and python. Can you help suggest how to go about this.
Many thanks

REPLY 
Jason Brownlee August 17, 2016 at 9:51 am #

Consider getting a good grounding in how to work through a machine learning problem end
to end in python first.

Here is a good tutorial to get you started:

http://machinelearningmastery.com/machine-learning-in-python-step-by-step/

Start Machine Learning ×

REPLY 
Agni August 17, 2016 at 6:23 am #
You can master applied Machine Learning
Dear Jeson, this is a great tutorial for beginners. It willmath
without satisfy the need
or fancy of many students who
degrees.
are looking for the initial help. But I have a question. Could youhow
Find out please light
in this freeon a few
and things:
practical i) how to test
course.
the trained model using test dataset (i.e., loading of test dataset and applied the model and suppose the
test file name is test.csv) ii) print the accuracy obtainedEmail
on test dataset iii) the o/p has more than 2 class
Address
(suppose 4-class classification problem).
Please show the whole program to overcome any confusion.
Thanks a lot. START MY EMAIL COURSE

REPLY 
Jason Brownlee August 17, 2016 at 10:03 am #

I provide an example elsewhere in the comments, you can also see how to make
predictions on new data in this post:
http://machinelearningmastery.com/5-step-life-cycle-neural-network-models-keras/

For an example of multi-class classification, you can see this tutorial:

http://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/

REPLY 
Doron Vetlzer August 17, 2016 at 9:29 am #
Start Machine Learning
I am trying to build a Neural Network with some recursive connections but not a full recursive
layer, how do I do this in Keras?

REPLY 
Doron Vetlzer August 17, 2016 at 9:31 am #

I could print a diagram of the network but what I want Basically is that each neuron in the
current time frame to know only its own previous output and not the output of all the neurons in the
output layer.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 29/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee August 17, 2016 at 10:04 am #

I don’t know off hand Doron.

REPLY 
Doron Veltzer August 23, 2016 at 2:28 am #

Thanks for replying though, have a good day.

REPLY 
sairam August 30, 2016 at 8:49 am #
Start Machine Learning ×
Hello Jason,

This is a great tutorial . Thanks for sharing. You can master applied Machine Learning
without math or fancy degrees.
I am having a dataset of 100 finger prints and i want toFind
extract minutiae
out how in thisof 100
free finger
and prints
practical using python
course.
( Keras). Can you please advise where to start? I am really confused.

Email Address

REPLY 
Jason Brownlee August 31, 2016 at 8:43 am # START MY EMAIL COURSE

If your fingerprints are images, you may want to consider using convolutional neural
networks (CNNs) that are much better at working image data.

See this tutorial on digit recognition for a start:

http://machinelearningmastery.com/handwritten-digit-recognition-using-convolutional-neural-
networks-python-keras/

REPLY 
padmashri July 6, 2017 at 10:12 pm #

Hi Jason
Thanks for this great tutorial, i am new to machine learning i went through your basic tutorial on
keras and also handwritten-digit-recognition. I would like to understand how i can train a set of
Start
image data, for eg. the set of image data can be Machine
some Learning
thing like square, circle, pyramid.
pl. let me know how the input data needs to fed to the program and how we need to export the
model.

REPLY 
Jason Brownlee July 9, 2017 at 10:30 am #

Start by preparing a high-quality dataset.

REPLY 
CM September 1, 2016 at 4:23 pm #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 30/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Hi Jason,

Thanks for the great article. But I had 1 query.

Are there any inbuilt functions in keras that can give me the feature importance for the ANN model?

If not, can you suggest a technique I can use to extract variable importance from the loss function? I am
considering an approach similar to that used in RF which involves permuting the values of the selected
variable and calculating the relative increase in loss.

Regards,
CM

Jason Brownlee September 2, 2016 at 8:07 am #

Start Machine Learning ×
REPLY 

I don’t believe so CM.

You can master applied Machine Learning
without
I would suggest using a wrapper method and evaluate math of
subsets or features
fancy degrees.
to develop a feature
importance/feature selection report. Find out how in this free and practical course.

I talk a lot more about feature selection in this post:

Email Address
http://machinelearningmastery.com/an-introduction-to-feature-selection/

I provide an example of feature selection in scikit-learn here:

START MY EMAIL COURSE
http://machinelearningmastery.com/feature-selection-machine-learning-python/

I hope that helps as a start.

REPLY 
Minesh Jethva May 15, 2017 at 7:49 pm #

have you develop any progress for this approach? I also have same problem.

REPLY 
Kamal September 7, 2016 at 2:09 am #

Dear Jason, I am new to Deep learning. Being a novice, I am asking you a technical question
which may seem silly. My question is that- can we use features (for example length of the sentence etc.)
Start Machine Learning
of a sentence while classifying a sentence ( suppose the o/p are +ve sentence and -ve sentence) using
deep neural network?

REPLY 
Jason Brownlee September 7, 2016 at 10:27 am #

Great question Kamal, yes you can. I would encourage you to include all such features and
see which give you a bump in performance.

REPLY 
Saurabh September 11, 2016 at 12:42 pm #
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 31/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Hi, How would I use this on a dataset that has multiple outputs? For example a dataset with
output A and B where A could be 0 or 1 and B could be 3 or 4 ?

REPLY 
Jason Brownlee September 12, 2016 at 8:30 am #

You could use two neurons in the output layer and normalize the output variables to both be
in the range of 0 to 1.

This tutorial on multi-class classification might give you some ideas:

http://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/

Start Machine Learning ×

REPLY 
Tom_P September 17, 2016 at 1:47 pm #
You can master applied Machine Learning
Hi Jason, without math or fancy degrees.
The tutorial looks really good but unfortunately I keep Find
getting
outan error
how when
in this freeimporting Dense
and practical from
course.
keras.layers, I get the error : AttributeError: module ‘theano’ has no attribute ‘gof’
I have tried reinstalling Theano but it has not fixed the issue.
Email Address
Best wishes
Tom START MY EMAIL COURSE

REPLY 
Jason Brownlee September 18, 2016 at 7:57 am #

Hi Tom, sorry to hear that. I have not seen this problem before.

Have you searched google? I can see a few posts and it might be related to your version of scipy or
similar.

Let me know how you go.

REPLY 
shudhan September 21, 2016 at 5:54 pm #

Hey Jason, Start Machine Learning

Can you please make a tutorial on how to add additional train data into the already trained model? This
will be helpful for the bigger data sets. I read that warm start is used for random forest. But not sure how
to implement as algorithm. A generalised version of how to implement would be good. Thank You!

REPLY 
Jason Brownlee September 22, 2016 at 8:08 am #

Great question Shudhan!

Yes, you could save your weights, load them later into a new network topology and start training on
new data again.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 32/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

I’ll work out an example in coming weeks, time permitting.

REPLY 
Joanna September 22, 2016 at 1:09 am #

Hi Jason,
first of all congratulations for this amazing work that you have done!
Here is my question:
What about if my .csv file includes also both nominal and numerical attributes?
Should I change my nominal values to numerical?

Thank you in advance

Start Machine Learning ×

REPLY 
Jason Brownlee September 22, 2016 at 8:19You
am #can master applied Machine Learning
without math or fancy degrees.
Hi Joanna, yes. Find out how in this free and practical course.

You can use a label encoder to convert nominal to integer, and then even convert the integer to one
hot encoding. Email Address

This post will give you code you can use:

START MY EMAIL COURSE
http://machinelearningmastery.com/data-preparation-gradient-boosting-xgboost-python/

REPLY 
ATM October 2, 2016 at 5:47 am #

A small bug:-
Line 25 : rounded = [round(x) for x in predictions]

should have numpy.round instead, for the code to run!

Great tutorial, regardless. The best i’ve seen for intro to ANN in python. Thanks!

REPLY 
Jason Brownlee October 2, 2016 at 8:20 am #

Start Machine Learning

Perhaps it’s your version of Python or environment?

In Python 2.7 the round() function is built-in.

REPLY 
AC January 14, 2017 at 2:11 am #

If there is comment for python3, should be better.

#use unmpy.round instead, if using python3,

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 33/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee January 15, 2017 at 5:24 am #

Thanks for the note AC.

REPLY 
Ash October 9, 2016 at 1:36 am #

This is simple to grasp! Great post! How can we perform dropout in keras?

REPLY 
Jason Brownlee October 9, 2016 at 6:49 am #
Start Machine Learning ×
Thanks Ash.
You can master applied Machine Learning
You can learn about drop out with Keras here: without math or fancy degrees.
http://machinelearningmastery.com/dropout-regularization-deep-learning-models-keras/
Find out how in this free and practical course.

Email Address
REPLY 
Homagni Saha October 14, 2016 at 4:15 am #
START MY EMAIL COURSE
Hello Jason,
You are using model.predict in the end to predict the results. Is it possible to save the model somewhere
in the harddisk and transfer it to another machine(turtlebot running on ROS for my instance) and then
use the model directly on turtlebot to predict the results?
Please tell me how
Thanking you
Homagni Saha

REPLY 
Jason Brownlee October 14, 2016 at 9:07 am #

Hi Homagni, great question.

Absolutely!
Start Machine Learning
Learn exactly how in this tutorial I wrote:
http://machinelearningmastery.com/save-load-keras-deep-learning-models/

REPLY 
Rimi October 16, 2016 at 8:21 pm #

Hi Jason,
I implemented you code to begin with. But I am getting an accuracy of 45.18% with the same parameters
and everything.
Cant figure out why.
Thanks

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 34/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee October 17, 2016 at 10:29 am #

There does sound like a problem there Rimi.

Confirm the code and data match exactly.

REPLY 
Ankit October 26, 2016 at 8:12 pm #

Hi Jason,
I am little confused with first layer parameters. You said that first layer has 12 neurons and expects 8
input variables.
Start Machine Learning ×
Why there is a difference between number of neurons, input_dim for first layer.
You can master applied Machine Learning
Regards, without math or fancy degrees.
Ankit Find out how in this free and practical course.

Email Address
REPLY 
Jason Brownlee October 27, 2016 at 7:45 am #
START MY EMAIL COURSE
Hi Ankit,

The problem has 8 input variables and the first hidden layer has 12 neurons. Inputs are the columns
of data, these are fixed. The Hidden layers in general are whatever we design based on whatever
capacity we think we need to represent the complexity of the problem. In this case, we have chosen
12 neurons for the first hidden layer.

I hope that is clearer.

REPLY 
Tom October 27, 2016 at 3:04 am #

Hi,
I have a data , IRIS like data but with more colmuns.
I want to use MLP and DBN/CNNClassifier (or any other Deep Learning classificaiton algorithm) on my
Start Machine Learning
data to see how correctly it does classified into 6 groups.

Previously using DEEP LEARNING FOR J, today first time see KERAS.
does KERAS has examples (code examples) of DL Classification algorithms?

Kindly,
Tom

REPLY 
Jason Brownlee October 27, 2016 at 7:48 am #

Yes Tom, the example in this post is an example of a neural network (deep learning) applied
to a classification problem.
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 35/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Rumesa October 30, 2016 at 1:57 am #

I have installed theano but it gives me the error of tensorflow.is it mendatory to install both
packages? because tensorflow is not supported on wndows.the only way to get it on windows is to install
virtual machine

REPLY 
Jason Brownlee October 30, 2016 at 8:57 am #

Keras will work just fine with Theano.

Start Machine Learning ×
Just install Theano, and configure Keras to use the Theano backend.
You can master applied Machine Learning
More information about configuring the Keras backend here:
without math or fancy degrees.
http://machinelearningmastery.com/introduction-python-deep-learning-library-keras/
Find out how in this free and practical course.

Email Address
REPLY 
Rumesa October 31, 2016 at 4:36 am #
START MY EMAIL COURSE
hey jason I have run your code but got the following error.Although I have aready
installed theano backend.help me out.I just stuck.

Using TensorFlow backend.

Traceback (most recent call last):
File “C:\Users\pc\Desktop\first.py”, line 2, in
from keras.models import Sequential
File “C:\Users\pc\Anaconda3\lib\site-packages\keras\__init__.py”, line 2, in
from . import backend
File “C:\Users\pc\Anaconda3\lib\site-packages\keras\backend\__init__.py”, line 64, in
from .tensorflow_backend import *
File “C:\Users\pc\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py”, line 1, in
import tensorflow as tf
ImportError: No module named ‘tensorflow’
>>>
Start Machine Learning

REPLY 
Jason Brownlee October 31, 2016 at 5:34 am #

Change the backend used by Keras from TensorFlow to Theano.

You can do this either by using the command line switch or changing the Keras config file.

See the link I posted in the previous post for instructions.

REPLY 
Maria January 6, 2017 at 1:05 pm #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 36/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Hello Rumesa!
Have you solved your problem? I have the same one. Everywhere is the same answer with
keras.json file or envirinment variable but it doesn’t work. Can you tell me what have
worked for you?

REPLY 
Jason Brownlee January 7, 2017 at 8:20 am #

Interesting.

Maybe there is an issue with the latest version and a tight coupling to tensorflow? I have not
seen this myself.

Perhaps it might be worth testing prior versions of Keras, such as 1.1.0?

Start Machine Learning ×
Try this:
You can master applied Machine Learning
1 pip install --upgrade --no-deps keras==1.1.0
without math or fancy degrees.
Find out how in this free and practical course.

Email Address REPLY 

Alexon November 1, 2016 at 6:54 am #

Hi Jason,
START MY EMAIL COURSE
First off, thanks so much for creating these resources, I have been keeping an eye on your newsletter for
a while now, and I finally have the free time to start learning more about it myself, so your work has been
really appreciated.

My question is: How can I set/get the weights of each hidden node?

I am planning to create several arrays randomized weights, then use a genetic algorithm to see which
weight array performs the best and improve over generations. How would be the best way to go about
this, and if I use a “relu” activation function, am I right in thinking these randomly generated weights
should be between 0 and 0.05?

Many thanks for your help

Alexon

Start Machine Learning REPLY 

Jason Brownlee November 1, 2016 at 8:05 am #

Thanks Alexon,

You can get and set the weights from a network.

You can learn more about how to do this in the context of saving the weights to file here:
http://machinelearningmastery.com/save-load-keras-deep-learning-models/

I hope that helps as a start, I’d love to hear how you go.

REPLY 
Alexon November 6, 2016 at 6:36 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 37/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Thats great, thanks for pointing me in the right direction.

I’d be happy to let you know how it goes, but might take a while as this is very much a
“when I can find the time” project between jobs

Cheers!

REPLY 
Arnaldo Gunzi November 2, 2016 at 10:17 pm #

Nice introduction, thanks!

Start
Jason Brownlee November 3, 2016 at 7:59 am # Machine Learning ×
REPLY 

I’m glad you found it useful Arnaldo. You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Abbey November 14, 2016 at 11:05 pm #
Email Address

Good day
START MY EMAIL COURSE
I have a question, how can I represent a character as a vector that could be an input for the neural
network to predict the word meaning and trained using LSTM

For instance, I have bf to predict boy friend or best friend and similarly I have 2mor to predict tomorrow. I
need to encode all the input as a character represented as vector, so that it can be train with RNN/LSTM
to predict the output.

Thank you.

Kind Regards

REPLY 
Jason Brownlee November 15, 2016 at 7:54 am #

Hi Abbey, You can map characters to integers to get integer vectors.

Start Machine Learning

REPLY 
Abbey November 15, 2016 at 6:17 pm #

Thank you Jason, if i map characters to integers value to get vectors using English
Alphabets, numbers and special characters

The question is how will LSTM predict the character. Please example in more details for me.

Regards

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 38/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee November 16, 2016 at 9:27 am #

Hi Abbey,

If your output values are also characters, you can map them onto integers, and reverse the
mapping to convert the predictions back to text.

Abbey November 16, 2016 at 8:39 pm #

The output value of the characters encoding will be text

Start Machine Learning ×

REPLY 
Abbey November 15, 2016 at 6:22 pm # You can master applied Machine Learning
without math or fancy degrees.
Thank you, Jason, if I map charactersFind
to integers value
out how in to get
this free vectors
and representation
practical course. of
the informal text using English Alphabets, numbers and special characters

The question is how will LSTM predict the character

Emailor words that have close meaning to the
Address
input value. Please example in more details for me. I understand how RNN/LSTM work based
on your tutorial example but the logic in designing processing is what I am stress with.
START MY EMAIL COURSE
Regards

REPLY 
Ammar November 27, 2016 at 10:35 am #

hi Jason,
i am trying to implement CNN one dimention on my data. so, i bluit my network.
the issue is:
def train_model(model, X_train, y_train, X_test, y_test):
X_train = X_train.reshape(-1, 1, 41)
X_test = X_test.reshape(-1, 1, 41)

numpy.random.seed(seed)
model.fit(X_train, y_train, validation_data=(X_test, y_test), nb_epoch=100, batch_size=64)
# Final evaluation of the model Start Machine Learning
scores = model.evaluate(X_test, y_test, verbose=0)
print(“Accuracy: %.2f%%” % (scores[1] * 100))
this method above does not work and does not give me any error message.
could you help me with this please?

REPLY 
Jason Brownlee November 28, 2016 at 8:40 am #

Hi Ammar, I’m surprised that there is no error message.

Perhaps run from the command line and add some print() statements to see exactly where it stops.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 39/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
KK November 28, 2016 at 6:55 pm #

Hi Jason
Great work. I have another doubt. How can we apply this to text mining. I have a csv file containing
review document and label. I want to apply classify the documents based on the text available. Can U do
this favor.

REPLY 
Jason Brownlee November 29, 2016 at 8:48 am #

I would recommend converting the chars to ints and then using an Embedding layer.
Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees. REPLY 
Alex M November 30, 2016 at 10:52 pm #
Find out how in this free and practical course.

Mr Jason, this is great tutorial but I am stack with some errors.

Email Address
First I can’t load data set correctly, tried to correct error but can’t make it. ( FileNotFoundError: [Errno 2]
No such file or directory: ‘pima-indians-diabetes.csv’ ).
START MY EMAIL COURSE
Second: While trying to evaluate the model it says (X is not defined) May be this is because uploading
failed.

Thanks!

REPLY 
Jason Brownlee December 1, 2016 at 7:29 am #

You need to download the file and place it in your current working directory Alex.

Does that help?

REPLY 
Alex M December 1, 2016 at 6:45 pm #
Start Machine Learning
Sir, it is now successful….
Thanks!

REPLY 
Jason Brownlee December 2, 2016 at 8:15 am #

Glad to hear it Alex.

REPLY 
Bappaditya December 2, 2016 at 7:35 pm #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 40/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Hi Jason,

First of all a special thanks to you for providing such a great tutorial. I am very new to machine
learning and truly speaking i had no background in data science. The concept of ML overwhelmed me
and now i have a desire to be an expert of this field. I need your advice to start from a scratch. Also i am
a PhD student in Computer Engineering ( computer hardware )and i want to apply it as a tool for fault
detection and testing for ICs.Can you provide me some references on this field?

REPLY 
Jason Brownlee December 3, 2016 at 8:29 am #

Hi Bappaditya,

My best advice for getting started is here:

Start Machine Learning ×
http://machinelearningmastery.com/start-here/#getstarted
You can
I believe machine learning and deep learning are good master
tools applied
for use Machine Learning
on problems in fault detection. A
without math
good place to find references is here http://scholar.google.com or fancy degrees.
Find out how in this free and practical course.
Best of luck with your project.

Email Address

REPLY 
Alex M December 3, 2016 at 8:00 pm # START MY EMAIL COURSE

Well as usual in our daily coding life errors happen, now I have this error how can I correct it?
Thanks!

” —————————————————————————
NoBackendError Traceback (most recent call last)
in ()
16 import librosa.display
17 audio_path = (‘/Users/MA/Python Notebook/OK.mp3’)
—> 18 y, sr = librosa.load(audio_path)

C:\Users\MA\Anaconda3\lib\site-packages\librosa\core\audio.py in load(path, sr, mono, offset, duration,

dtype)
107
108 y = []
–> 109 with audioread.audio_open(os.path.realpath(path)) as input_file:
Start Machine Learning
110 sr_native = input_file.samplerate
111 n_channels = input_file.channels

C:\Users\MA\Anaconda3\lib\site-packages\audioread\__init__.py in audio_open(path)
112
113 # All backends failed!
–> 114 raise NoBackendError()

NoBackendError:

That is the error I am getting just when trying to load a song into librosa…
Thanks!! @Jason Brownlee

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 41/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee December 4, 2016 at 5:30 am #

Sorry, this looks like an issue with your librosa library, not a machine learning issue. I can’t
give you expert advice, sorry.

REPLY 
Alex M December 4, 2016 at 10:30 pm #

Thanks I have managed to correct the error…

Happy Sunday to you all……

Start Machine Learning ×
You can master applied Machine Learning
without
Jason Brownlee December 5, 2016 at 6:49 am # math or fancy degrees. REPLY 

Find out how in this free and practical course.

Glad to hear it Alex.
Email Address

START MY EMAIL COURSE REPLY 

ayush June 19, 2018 at 3:27 am #

how did you solved the problem?

REPLY 
Lei December 4, 2016 at 10:52 pm #

Hi, Jason, thank you for your amazing examples.

I run the same code on my laptop. But I did not get the same results. What could be the possible
reasons?
I am using windows 8.1 64bit+eclipse+anaconda 4.2+theano 0.9.4+CUDA7.5
I got results like follows.

……
Epoch 145/150
Start Machine Learning
10/768 […………………………] – ETA: 0s – loss: 0.3634 – acc: 0.8000
80/768 [==>………………………] – ETA: 0s – loss: 0.4066 – acc: 0.7750
150/768 [====>…………………….] – ETA: 0s – loss: 0.4059 – acc: 0.8067
220/768 [=======>………………….] – ETA: 0s – loss: 0.4047 – acc: 0.8091
300/768 [==========>……………….] – ETA: 0s – loss: 0.4498 – acc: 0.7867
380/768 [=============>…………….] – ETA: 0s – loss: 0.4595 – acc: 0.7895
450/768 [================>………….] – ETA: 0s – loss: 0.4568 – acc: 0.7911
510/768 [==================>………..] – ETA: 0s – loss: 0.4553 – acc: 0.7882
580/768 [=====================>……..] – ETA: 0s – loss: 0.4677 – acc: 0.7776
660/768 [========================>…..] – ETA: 0s – loss: 0.4697 – acc: 0.7788
740/768 [===========================>..] – ETA: 0s – loss: 0.4611 – acc: 0.7838

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 42/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

768/768 [==============================] – 0s – loss: 0.4614 – acc: 0.7799

Epoch 146/150

10/768 […………………………] – ETA: 0s – loss: 0.3846 – acc: 0.8000

90/768 [==>………………………] – ETA: 0s – loss: 0.5079 – acc: 0.7444
170/768 [=====>……………………] – ETA: 0s – loss: 0.4500 – acc: 0.7882
250/768 [========>…………………] – ETA: 0s – loss: 0.4594 – acc: 0.7840
330/768 [===========>………………] – ETA: 0s – loss: 0.4574 – acc: 0.7818
400/768 [==============>……………] – ETA: 0s – loss: 0.4563 – acc: 0.7775
470/768 [=================>…………] – ETA: 0s – loss: 0.4654 – acc: 0.7723
540/768 [====================>………] – ETA: 0s – loss: 0.4537 – acc: 0.7870
620/768 [=======================>……] – ETA: 0s – loss: 0.4615 – acc: 0.7806
690/768 [=========================>….] – ETA: 0s – loss: 0.4631 – acc: 0.7739
750/768 [============================>.] – ETA: 0s – loss: 0.4649 – acc: 0.7733
768/768 [==============================] – 0sStart Machine
– loss: 0.4636 Learning
– acc: 0.7734
×
Epoch 147/150
You can master applied Machine Learning
10/768 […………………………] – ETA: 0s – loss: 0.3561 – acc:
without 0.9000
math or fancy degrees.
Find out
90/768 [==>………………………] – ETA: 0s – loss: 0.4167 how0.8556
– acc: in this free and practical course.
170/768 [=====>……………………] – ETA: 0s – loss: 0.4824 – acc: 0.8059
250/768 [========>…………………] – ETA: 0s – loss:Email
0.4534 – acc: 0.8080
Address
330/768 [===========>………………] – ETA: 0s – loss: 0.4679 – acc: 0.7848
400/768 [==============>……………] – ETA: 0s – loss: 0.4590 – acc: 0.7950
START
460/768 [================>………….] – ETA: 0s – loss: MY EMAIL
0.4619 – acc:COURSE
0.7913
530/768 [===================>……….] – ETA: 0s – loss: 0.4562 – acc: 0.7868
600/768 [======================>…….] – ETA: 0s – loss: 0.4497 – acc: 0.7883
680/768 [=========================>….] – ETA: 0s – loss: 0.4525 – acc: 0.7853
760/768 [============================>.] – ETA: 0s – loss: 0.4568 – acc: 0.7803
768/768 [==============================] – 0s – loss: 0.4561 – acc: 0.7812
Epoch 148/150

10/768 […………………………] – ETA: 0s – loss: 0.4183 – acc: 0.9000

80/768 [==>………………………] – ETA: 0s – loss: 0.3674 – acc: 0.8750
160/768 [=====>……………………] – ETA: 0s – loss: 0.4340 – acc: 0.8250
240/768 [========>…………………] – ETA: 0s – loss: 0.4799 – acc: 0.7583
320/768 [===========>………………] – ETA: 0s – loss: 0.4648 – acc: 0.7719
400/768 [==============>……………] – ETA: 0s – loss: 0.4596 – acc: 0.7775
470/768 [=================>…………] – ETA: 0s – loss: 0.4475 – acc: 0.7809
540/768 [====================>………] – ETA: 0s – loss:
Start 0.4545Learning
Machine – acc: 0.7778
620/768 [=======================>……] – ETA: 0s – loss: 0.4590 – acc: 0.7742
690/768 [=========================>….] – ETA: 0s – loss: 0.4769 – acc: 0.7652
760/768 [============================>.] – ETA: 0s – loss: 0.4748 – acc: 0.7658
768/768 [==============================] – 0s – loss: 0.4734 – acc: 0.7669
Epoch 149/150

10/768 […………………………] – ETA: 0s – loss: 0.3043 – acc: 0.9000

90/768 [==>………………………] – ETA: 0s – loss: 0.4913 – acc: 0.7111
170/768 [=====>……………………] – ETA: 0s – loss: 0.4779 – acc: 0.7588
250/768 [========>…………………] – ETA: 0s – loss: 0.4794 – acc: 0.7640
320/768 [===========>………………] – ETA: 0s – loss: 0.4957 – acc: 0.7562
370/768 [=============>…………….] – ETA: 0s – loss: 0.4891 – acc: 0.7703

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 43/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

450/768 [================>………….] – ETA: 0s – loss: 0.4737 – acc: 0.7867

520/768 [===================>……….] – ETA: 0s – loss: 0.4675 – acc: 0.7865
600/768 [======================>…….] – ETA: 0s – loss: 0.4668 – acc: 0.7833
680/768 [=========================>….] – ETA: 0s – loss: 0.4677 – acc: 0.7809
760/768 [============================>.] – ETA: 0s – loss: 0.4648 – acc: 0.7803
768/768 [==============================] – 0s – loss: 0.4625 – acc: 0.7826
Epoch 150/150

10/768 […………………………] – ETA: 0s – loss: 0.2751 – acc: 1.0000

100/768 [==>………………………] – ETA: 0s – loss: 0.4501 – acc: 0.8100
170/768 [=====>……………………] – ETA: 0s – loss: 0.4588 – acc: 0.8059
250/768 [========>…………………] – ETA: 0s – loss: 0.4299 – acc: 0.8200
310/768 [===========>………………] – ETA: 0s – loss: 0.4298 – acc: 0.8129
380/768 [=============>…………….] – ETA: 0s – loss: 0.4365 – acc: 0.8053
460/768 [================>………….] – ETA: 0s –Start Machine
loss: 0.4469 Learning
– acc: 0.7957
×
540/768 [====================>………] – ETA: 0s – loss: 0.4436 – acc: 0.8000
You can master applied Machine Learning
620/768 [=======================>……] – ETA: 0s – loss: 0.4570 – acc: 0.7871
without math or fancy degrees.
690/768 [=========================>….] – ETA: 0s – loss: 0.4664 – acc: 0.7783
Find out how in this free and practical course.
760/768 [============================>.] – ETA: 0s – loss: 0.4617 – acc: 0.7789
768/768 [==============================] – 0s – loss: 0.4638 – acc: 0.7773
Email Address
32/768 [>………………………..] – ETA: 0s
448/768 [================>………….] – ETA: 0sacc: 79.69%
START MY EMAIL COURSE

REPLY 
Jason Brownlee December 5, 2016 at 6:50 am #

There is randomness in the learning process that we cannot control for yet.

See this post:

http://machinelearningmastery.com/randomness-in-machine-learning/

REPLY 
Nanya December 10, 2016 at 2:55 pm #

Hello Jason Brownlee,Thx for sharing~

I’m new in deep learning.And I am wondering can what you dicussed here:”Keras” be used to build a
Start Machine
CNN in tensorflow and train some csv fiels for classification.May Learning
be this is a stupid question,but waiting
for you reply.I’m working on my graduation project for Word sense disambiguation with cnn,and just can’t
move on.Hope for your heip~Bese wishes!

REPLY 
Jason Brownlee December 11, 2016 at 5:22 am #

Sorry Nanya, I’m not sure I understand your question. Are you able to rephrase it?

REPLY 
Anon December 16, 2016 at 12:51 am #
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 44/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

I’ve just installed Anaconda with Keras and am using python 3.5.
It seems there’s an error with the rounding using Py3 as opposed to Py2. I think it’s because of
this change: https://github.com/numpy/numpy/issues/5700

I removed the rounding and just used print(predictions) and it seemed to work outputting floats instead.

Does this look correct?

…
Epoch 150/150
0s – loss: 0.4593 – acc: 0.7839
[[ 0.79361773]
[ 0.10443526]
[ 0.90862554]
…,
[ 0.33652252] Start Machine Learning ×
[ 0.63745886]
You can master applied Machine Learning
[ 0.11704451]]
without math or fancy degrees.
Find out how in this free and practical course.

Email
Jason Brownlee December 16, 2016 at 5:44 am # Address REPLY 

Nice, it does look good!

START MY EMAIL COURSE

REPLY 
Florin Claudiu Mihalache December 19, 2016 at 2:37 am #

Hi Jason Brownlee
I tried to modified your exemple for my problem (Letter Recognition
,http://archive.ics.uci.edu/ml/datasets/Letter+Recognition).
My data set look like http://archive.ics.uci.edu/ml/machine-learning-databases/letter-recognition/letter-
recognition.data (T,2,8,3,5,1,8,13,0,6,6,10,8,0,8,0,8) .I try to split the data in input and ouput like this :

X = dataset[:,1:17]
Y = dataset[:,0]
but a have some error (something related that strings are not recognized) .
I tried to modified each letter whit the ASCII code (A became 65 and so on).The string error disappeared.
The program compiles now but the output look like thisStart
: Machine Learning
17445/20000 [=========================>….] – ETA: 0s – loss: -1219.4768 – acc:0.0000e+00
17605/20000 [=========================>….] – ETA: 0s – loss: -1219.4706 – acc:0.0000e+00
17730/20000 [=========================>….] – ETA: 0s – loss: -1219.4566 – acc:0.0000e+00
17890/20000 [=========================>….] – ETA: 0s – loss: -1219.4071 – acc:0.0000e+00
18050/20000 [==========================>…] – ETA: 0s – loss: -1219.4599 – acc:0.0000e+00
18175/20000 [==========================>…] – ETA: 0s – loss: -1219.3972 – acc:0.0000e+00
18335/20000 [==========================>…] – ETA: 0s – loss: -1219.4642 – acc:0.0000e+00
18495/20000 [==========================>…] – ETA: 0s – loss: -1219.5032 – acc:0.0000e+00
18620/20000 [==========================>…] – ETA: 0s – loss: -1219.4391 – acc:0.0000e+00
18780/20000 [===========================>..] – ETA: 0s – loss: -1219.5652 – acc:0.0000e+00
18940/20000 [===========================>..] – ETA: 0s – loss: -1219.5520 – acc:0.0000e+00
19080/20000 [===========================>..] – ETA: 0s – loss: -1219.5381 – acc:0.0000e+00
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 45/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

19225/20000 [===========================>..] – ETA: 0s – loss: -1219.5182 – acc:0.0000e+00

19385/20000 [============================>.] – ETA: 0s – loss: -1219.6742 – acc:0.0000e+00
19535/20000 [============================>.] – ETA: 0s – loss: -1219.7030 – acc:0.0000e+00
19670/20000 [============================>.] – ETA: 0s – loss: -1219.7634 – acc:0.0000e+00
19830/20000 [============================>.] – ETA: 0s – loss: -1219.8336 – acc:0.0000e+00
19990/20000 [============================>.] – ETA: 0s – loss: -1219.8532 – acc:0.0000e+00
20000/20000 [==============================] – 1s – loss: -1219.8594 – acc: 0.0000e+00
18880/20000 [===========================>..] – ETA: 0sacc: 0.00%

I do not understand why. Can you please help me

REPLY 
Anon December 26, 2016 at 6:44 am #
Start Machine Learning ×
What version of Python are you running?
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
karishma sharma December 22, 2016 at 10:03 am #
Email Address
Hi Jason,

Since the epoch is set to 150 and batch size is 10, does the training algorithm pick 10 training examples
START MY EMAIL COURSE
at random in each iteration, given that we had only 768 total in X. Or does it sample randomly after it has
finished covering all.

Thanks

REPLY 
Jason Brownlee December 23, 2016 at 5:27 am #

Good question,

It iterates over the dataset 150 times and within one epoch it works through 10 rows at a time before
doing an update to the weights. The patterns are shuffled before each epoch.

I hope that helps.

Start Machine Learning

REPLY 
Kaustuv January 9, 2017 at 4:57 am #

Hi Jason
Thanks a lot for this blog. It really helps me to start learning deep learning which was in a planning state
for last few months. Your simple enrich blogs are awsome. No questions from my side before completing
all tutorials.
One question regarding availability of your book. How can I buy those books from India ?

REPLY 
Jason Brownlee January 9, 2017 at 7:53 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 46/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

All my books and training are digital, you can purchase them from here:
http://machinelearningmastery.com/products

REPLY 
Stephen Wilson January 15, 2017 at 4:00 pm #

Hi Jason, firstly your work here is a fantastic resource and I am very thankful for the effort you
put in.
I am a slightly-better-than-beginner at python and an absolute novice at ML, I wonder if you could help
me classify my problem and find an angle to work at it from.

My data is thus:
Column Names: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, Result
Values: 4, 4, 6, 6, 3, 2, 5, 5, 0, 0, 0, 0, 0, 0, 0, 4 Start Machine Learning ×
I want to find the percentage chance of each Column Names category being the Result based off the
You can master applied Machine Learning
configuration of all the values present from 1-15. Thenwithout
if need math
be compare
or fancythe configuration of Values
degrees.
with another row of values to find the same, Resulting Find
in theouttotal
howneeded calculation
in this free as: course.
and practical
Column Names: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, Result
Values: 4, 4, 6, 6, 3, 2, 5, 5, 0, 0, 0, 0, 0, 0, 0, 4 Email Address
Values2: 7, 3, 5, 1, 4, 8, 6, 2, 9, 9, 9, 9, 9, 9, 9

I apologize if my explanation is not clear, and appreciateSTART

any help
MY you can
EMAIL give me thank you.
COURSE

REPLY 
Jason Brownlee January 16, 2017 at 10:39 am #

Hi Stephen,

This process might help you work through your problem:

http://machinelearningmastery.com/start-here/#process

Specifically the first step in defining your problem.

Let me know how you go.

REPLY 
Rohit January 16, 2017 at 10:37 pm # Start Machine Learning

Thanks Jason for such a nice and concise example.

Just wanted to ask if it is possible to save this model in a file and port it to may be an Android or iOS
device? If so, what are the libraries available for the same?

Thanks

Rohit

REPLY 
Jason Brownlee January 17, 2017 at 7:38 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 47/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Thanks Rohit,

Here’s an example of saving a Keras model to file:

http://machinelearningmastery.com/save-load-keras-deep-learning-models/

I don’t know about running Keras on an Android or iOS device. Let me know how you go.

REPLY 
zaheer khan June 16, 2017 at 7:17 pm #

Dear Jason, Thanks for sharing this article.

I am novice to the deep learning, and my apology if my question is not clear. my question is
could we call all that functions and program from any .php,.aspx, or .html webpage. i mean i load
the variables and other files selection from user interface and then make them input to this
Start Machine Learning ×
functions.

will be waiting for your kind reply. You can master applied Machine Learning
thanks in advance. without math or fancy degrees.
zaheer Find out how in this free and practical course.

Email Address

REPLY 
Jason Brownlee June 17, 2017 at 7:25 am #
START MY EMAIL COURSE

Perhaps, this sounds like a systems design question, not really machine learning.

I would suggest you gather requirements, assess risks like any software engineering project.

REPLY 
Hsiang January 18, 2017 at 3:35 pm #

Hi, Jason

Thank you for your blog! It is wonderful!

I used tensorflow as backend, and implemented the procedures using Jupyter.

I did “source activate tensorflow” -> “ipython notebook”.
I can successfully use Keras and import tensorflow.

However, it seems that such environment doesn’t support Startpandas andLearning

Machine sklearn.
Do you have any way to incorporate pandas, sklearn and keras?
(I wish to use sklearn to revisit the classification problem and compare the accuracy with the deep
learning method. But I also wish to put the works together in the same interface.)

Thanks!

REPLY 
Jason Brownlee January 19, 2017 at 7:24 am #

Sorry, I do not use notebooks myself. I cannot offer you good advice.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 48/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Hsiang January 19, 2017 at 12:53 pm #

Thanks, Jason!
Actually the problem is not on notebooks. Even I used the terminal mode, i.e. doing “source
activate tensorflow” only. It failed to import sklearn. Does that mean tensorflow library is not
compatible with sklearn? Thanks again!

REPLY 
Jason Brownlee January 20, 2017 at 10:17 am #

Sorry Hsiang, I don’t have experience using sklearn and tensorflow with virtual
environments.
Start Machine Learning ×
You can master applied Machine Learning

Hsiang January 21, 2017 at 12:46without

am #
math or fancy degrees.
Find out how in this free and practical course.
Thank you!
Email Address

START
Jason Brownlee January 21, 2017 MYam
at 10:34 EMAIL
# COURSE

You’re welcome Hsiang.

REPLY 
keshav bansal January 24, 2017 at 12:45 am #

hello sir,
A very informative post indeed . I know my question is a very trivial one but can you please show me
how to predict on a explicitly mentioned data tuple say v=[6,148,72,35,0,33.6,0.627,50]
thanks for the tutorial anyway

REPLY 
Jason Brownlee January 24, 2017 at 11:04 am #
Start Machine Learning

Hi keshav,

You can make predictions by calling model.predict()

REPLY 
CATRINA WEBB January 25, 2017 at 9:06 am #

When I rerun the file (without predictions) does it reset the model and weights?

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 49/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Ericson January 30, 2017 at 8:04 pm #

excuse me sir, i wanna ask you a question about this paragraph”dataset = numpy.loadtxt(“pima-
indians-diabetes.csv”,delimiter=’,’)”, i used the mac and downloaded the dataset,then i exchanged the
text into csv file. Running the program

,hen i got:{Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 12:39:47)

[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type “copyright”, “credits” or “license()” for more information.
>>>
============ RESTART: /Users/luowenbin/Documents/database_test.py ============
Using TensorFlow backend.

Traceback (most recent call last):

Start Machine Learning ×
File “/Users/luowenbin/Documents/database_test.py”, line 9, in
dataset = numpy.loadtxt(“pima-indians-diabetes.csv”,delimiter=’,’)
You can master applied Machine Learning
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
without math or fancy degrees.
packages/numpy/lib/npyio.py”, line 985, in loadtxt Find out how in this free and practical course.
items = [conv(val) for (conv, val) in zip(converters, vals)]
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
Email Address
packages/numpy/lib/npyio.py”, line 687, in floatconv
return float(x)
ValueError: could not convert string to float: book START MY EMAIL COURSE
>>> }
How can i solve this problem? give me a hand thank you!

REPLY 
Jason Brownlee February 1, 2017 at 10:22 am #

Hi Ericson,

Confirm that the contents of “pima-indians-diabetes.csv” meet your expectation of a list of CSV lines.

REPLY 
Sukhpal February 7, 2017 at 9:00 pm #

excuse me sir,when i run this code for my data set Machine

Start ,I encounter this problem…please help me
Learning
finding solution to this problem
runfile(‘C:/Users/sukhpal/.spyder/temp.py’, wdir=’C:/Users/sukhpal/.spyder’)
Using TensorFlow backend.
Traceback (most recent call last):

File “”, line 1, in

runfile(‘C:/Users/sukhpal/.spyder/temp.py’, wdir=’C:/Users/sukhpal/.spyder’)

File “C:\Users\sukhpal\Anaconda2\lib\site-packages\spyder\utils\site\sitecustomize.py”, line 866, in

runfile
execfile(filename, namespace)

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 50/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

File “C:\Users\sukhpal\Anaconda2\lib\site-packages\spyder\utils\site\sitecustomize.py”, line 87, in

execfile
exec(compile(scripttext, filename, ‘exec’), glob, loc)

File “C:/Users/sukhpal/.spyder/temp.py”, line 1, in

from keras.models import Sequential

File “C:\Users\sukhpal\Anaconda2\lib\site-packages\keras\__init__.py”, line 2, in

from . import backend

File “C:\Users\sukhpal\Anaconda2\lib\site-packages\keras\backend\__init__.py”, line 67, in

from .tensorflow_backend import *

File “C:\Users\sukhpal\Anaconda2\lib\site-packages\keras\backend\tensorflow_backend.py”, line 1, in

import tensorflow as tf

ImportError: No module named tensorflow Start Machine Learning ×

You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course. REPLY 
Jason Brownlee February 8, 2017 at 9:34 am #

This is a change with the most recent version of Address

Email tensorflow, I will investigate and change the
example.

For now, consider installing and using an older version of tensorflow.

START MY EMAIL COURSE

REPLY 
Will February 14, 2017 at 5:33 am #

Great tutorial! Amazing amount of work you’ve put in and great marketing skills (I also have an
email list, ebooks and sequence, etc). I ran this in Jupyter notebook… I noticed the 144th epoch (acc
.7982) had more accuracy than at 150. Why is that?

P.S. i did this for the print: print(numpy.round(predictions))

It seems to avoid a list of arrays which when printing includes the dtype (messy)

REPLY 
Jason Brownlee February 14, 2017 at 10:07 am #
Start Machine Learning
Thanks Will.

The model will fluctuate in performance while learning. You can configure triggered check points to
save the model if/when conditions like a decrease in train/validation performance is detected. Here’s
an example:
http://machinelearningmastery.com/check-point-deep-learning-models-keras/

REPLY 
Sukhpal February 14, 2017 at 3:50 pm #

Please help me to find out this error

runfile(‘C:/Users/sukhpal/.spyder/temp.py’, wdir=’C:/Users/sukhpal/.spyder’)ERROR: execution aborted
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 51/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee February 15, 2017 at 11:32 am #

I’m not sure Sukhpal.

Consider getting code working from the command line, I don’t use IDEs myself.

REPLY 
Kamal February 14, 2017 at 5:15 pm #

please help me to find this error find this error

Epoch 194/195
Start Machine Learning ×
195/195 [==============================] – 0s – loss: 0.2692 – acc: 0.8667
Epoch 195/195 You can master applied Machine Learning
195/195 [==============================] – 0swithout – loss: math
0.2586
or – acc: degrees.
fancy 0.8667
195/195 [==============================] – 0sFind out how in this free and practical course.
Traceback (most recent call last):

Email Address

REPLY 
Jason Brownlee February 15, 2017 at 11:32 amSTART
# MY EMAIL COURSE

What was the error exactly Kamal?

REPLY 
Kamal February 15, 2017 at 3:24 pm #

sir when i run the code on my data set

then it doesnot show overall accuracy although it shows the accuracy and loss for the whole iterations

REPLY 
Jason Brownlee February 16, 2017 at 11:06 am #

I’m not sure I understand your question Kamal, please you could restate it?
Start Machine Learning

REPLY 
Val February 15, 2017 at 9:00 pm #

Hi Jason, im just starting deep learning in python using keras and theano. I have followed the
installation instructions without a hitch. Tested some examples but when i run this one line by line i get a
lot of exceptions and errors once i run the “model.fit(X,Y, nb_epochs=150, batch_size=10”

REPLY 
Jason Brownlee February 16, 2017 at 11:06 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 52/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

What errors are you getting?

REPLY 
CrisH February 17, 2017 at 8:12 pm #

Hi, how do I know what number to use for random.seed() ? I mean you use 7, is there any
reason for that? Also is it enough to use it only once, in the beginning of the code?

REPLY 
Jason Brownlee February 18, 2017 at 8:38 am #

You can use any number CrisH. The fixed random seed makes the example reproducible.
Start Machine Learning ×
You can learn more about randomness and random seeds in this post:
http://machinelearningmastery.com/randomness-in-machine-learning/
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
kk February 18, 2017 at 1:53 am #
Email Address

am new to deep learning and found this great tutorial. keep it up and look forward!!
START MY EMAIL COURSE

REPLY 
Jason Brownlee February 18, 2017 at 8:41 am #

Thanks!

REPLY 
Iqra Ameer February 21, 2017 at 5:20 am #

HI, I have a problem in execution the above example as it. It seems that it’s not running properly
and stops at Using TensorFlow backend.

Epoch 147/150
768/768 [==============================] – 0s – loss: 0.4709 – acc: 0.7878
Epoch 148/150 Start Machine Learning
768/768 [==============================] – 0s – loss: 0.4690 – acc: 0.7812
Epoch 149/150
768/768 [==============================] – 0s – loss: 0.4711 – acc: 0.7721
Epoch 150/150
768/768 [==============================] – 0s – loss: 0.4731 – acc: 0.7747
32/768 [>………………………..] – ETA: 0sacc: 76.43%

I am new in this field, could you please guide me about this error.
I also executed on another data set, it stops with the same behavior.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 53/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee February 21, 2017 at 9:39 am #

What is the error exactly? The example hangs?

Maybe try the Theano backend and see if that makes a difference. Also make sure all of your
libraries are up to date.

REPLY 
Iqra Ameer February 22, 2017 at 5:47 am #

Dear Jason,
Thank you so much for your valuable suggestions. I tried Theano backend and also updated all my
libraries, but again it hanged at:
Start Machine Learning ×
768/768 [==============================] – 0s – loss: 0.4656 – acc: 0.7799
You can master applied Machine Learning
Epoch 149/150
without math or fancy degrees.
768/768 [==============================] – 0s – loss: 0.4589 – acc: 0.7826
Find out how in this free and practical course.
Epoch 150/150
768/768 [==============================] – 0s – loss: 0.4611 – acc: 0.7773
32/768 [>………………………..] – ETA: 0sacc: 78.91% Email Address

START MY EMAIL COURSE

REPLY 
Jason Brownlee February 22, 2017 at 10:05 am #

I’m sorry to hear that, I have not seen this issue before.

Perhaps a RAM issue or a CPU overheating issue? Are you able to try different hardware?

REPLY 
frd March 8, 2017 at 2:50 am #

Hi!

Were you able to find a solution for that?

I’m having exactly the same problem

(…) Start Machine Learning

Epoch 149/150
768/768 [==============================] – 0s – loss: 0.4593 – acc: 0.7773
Epoch 150/150
768/768 [==============================] – 0s – loss: 0.4586 – acc: 0.7891
32/768 [>………………………..] – ETA: 0sacc: 76.69%

REPLY 
Bhanu February 23, 2017 at 1:51 pm #

Hello sir,
i want to ask wether we can convert this code to deep learning wid increasing number of layers..
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 54/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee February 24, 2017 at 10:12 am #

Sure you can increase the number of layers, try it and see.

REPLY 
Ananya Mohapatra February 28, 2017 at 6:40 pm #

hello sir,
could you please tell me how do i determine the no.of neurons in each layer, because i am using a
different datset and am unable to know the no.of neurons in each layer
Start Machine Learning ×
You can master applied Machine Learning
REPLY 
Jason Brownlee March 1, 2017 at 8:33 am #without math or fancy degrees.
Find out how in this free and practical course.
Hi Ananya, great question.

Sorry, there is no good theory on how to configure aEmail Address

neural net.

You can configure the number of neurons in a layer by trial and error. Also consider tuning the
number of epochs and batch size at the same time. START MY EMAIL COURSE

REPLY 
Ananya Mohapatra March 1, 2017 at 4:42 pm #

thank you so much sir. It worked !

REPLY 
Jason Brownlee March 2, 2017 at 8:11 am #

Glad to here it Ananya.

Start Machine Learning REPLY 

Jayant Sahewal February 28, 2017 at 8:11 pm #

Hi Jason,

really helpful blog. I have a question about how much time does it take to converge?

I have a dataset with around 4000 records, 3 input columns and 1 output column. I came up with the
following model

def create_model(dropout_rate=0.0, weight_constraint=0, learning_rate=0.001, activation=’linear’):

# create model
model = Sequential()
model.add(Dense(6, input_dim=3, init=’uniform’, activation=activation,
W_constraint=maxnorm(weight_constraint)))

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 55/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

model.add(Dropout(dropout_rate))
model.add(Dense(1, init=’uniform’, activation=’sigmoid’))
# Optimizer
optimizer = Adam(lr=learning_rate)
# Compile model
model.compile(loss=’binary_crossentropy’, optimizer=optimizer, metrics=[‘accuracy’])
return model

# create model
model = KerasRegressor(build_fn=create_model, verbose=0)
# define the grid search parameters
batch_size = [10]
epochs = [100]
weight_constraint = [3]
dropout_rate = [0.9] Start Machine Learning ×
learning_rate = [0.01]
You can master applied Machine Learning
activation = [‘linear’]
without math or fancy degrees.
param_grid = dict(batch_size=batch_size, nb_epoch=epochs, dropout_rate=dropout_rate, \
Find out how in this free and practical course.
weight_constraint=weight_constraint, learning_rate=learning_rate, activation=activation)
grid = GridSearchCV(estimator=model, param_grid=param_grid, n_jobs=-1, cv=5)
grid_result = grid.fit(X_train, Y_train) Email Address

I have a 32 core machine with 64 GB RAM and it does not converge even in more than an hour. I can
START
see all the cores busy, so it is using all the cores for training. MY EMAIL
However, if COURSE
I change the input neurons to 3
then it converges in around 2 minutes.

Keras version: 1.1.1

Tensorflow version: 0.10.0rc0
theano version: 0.8.2.dev-901275534cbfe3fbbe290ce85d1abf8bb9a5b203

It’s using Tensorflow backend. Can you help me understand what is going on or point me in the right
direction? Do you think switching to theano will help?

Best,
Jayant

REPLY 
Jason Brownlee March 1, 2017 at 8:36 am #
Start Machine Learning
This post might help you tune your deep learning model:
http://machinelearningmastery.com/improve-deep-learning-performance/

I hope that helps as a start.

REPLY 
Animesh Mohanty March 1, 2017 at 9:21 pm #

hello sir,
could you please tell me how can i plot the results of the code on a graph . I made a few adjustments to
the code so as to run it on a different dataset.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 56/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee March 2, 2017 at 8:16 am #

What do you want to plot exactly Animesh?

REPLY 
Animesh Mohanty March 2, 2017 at 4:56 pm #

Accuracy vs no.of neurons in the input layer and the no.of neurons in the hidden layer

REPLY 
param March 2, 2017 at 12:15 am #
Start Machine Learning ×
sir can u plz explain
the different attributes used in this statement You can master applied Machine Learning
without math or fancy degrees.
print(“%s: %.2f%%” % (model.metrics_names[1], scores[1]*100))
Find out how in this free and practical course.

Email Address
REPLY 
param March 2, 2017 at 12:16 am #

precisely,what is model.metrics_names START MY EMAIL COURSE

REPLY 
Jason Brownlee March 2, 2017 at 8:22 am #

model.metrics_names is a list of names of the metrics collected during training.

More details here:

https://keras.io/models/sequential/

REPLY 
Jason Brownlee March 2, 2017 at 8:20 am #

Hi param,
Start Machine Learning
It is using string formatting. %s formats a string, %.2f formats a floating point value with 2 decimal
places, %% includes a percent symbol.

You can learn more about the print function here:

https://docs.python.org/3/library/functions.html#print

More info on string formatting here:

https://pyformat.info/

REPLY 
Vijin K P March 2, 2017 at 4:01 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 57/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Hi Jason,

It was an awesome post. Could you please tell me how to we decide the following in a DNN 1. number of
neurons in the hidden layers
2. number of hidden layers

Thanks.
Vijin

REPLY 
Jason Brownlee March 2, 2017 at 8:22 am #

Great question Vijin.

Start
Generally, trial and error. There are no good theories Machine
on how to configureLearning
a neural network. ×
You can master applied Machine Learning
without math or fancy degrees.
Vijin K P March 3, 2017 at 5:23 am # Find out how in this free and practical course. REPLY 

We do cross validation, grid search etc to find the hyper parameters in machine
Email Address
algorithms. Similarly can we do anything to identify the above parameters??

START MY EMAIL COURSE

REPLY 
Jason Brownlee March 3, 2017 at 7:46 am #

Yes, we can use grid search and tuning for neural nets.

The stochastic nature of neural nets means that each experiment (set of configs) will have to
be run many times (30? 100?) so that you can take the mean performance.

More general info on tuning neural nets here:

http://machinelearningmastery.com/improve-deep-learning-performance/

More on randomness and stochastic algorithms here:

http://machinelearningmastery.com/randomness-in-machine-learning/

Bogdan March 2, 2017 at 11:48 pm # Start Machine Learning REPLY 

Jason, Please tell me about these lines in your code:

seed = 7
numpy.random.seed(seed)

What do they do? And why do they do it?

One more question is why do you call the last section Bonus:Make a prediction?
I thought this what ANN was created for. What the point if your network’s output is just what you have
already know?

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 58/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee March 3, 2017 at 7:44 am #

They seed the random number generator so that it produces the same sequence of random
numbers each time the code is run. This is to ensure you get the same result as me.

I’m not convinced it works with Keras though.

More on randomness in machine learning here:

http://machinelearningmastery.com/randomness-in-machine-learning/

I was showing how to build and evaluate the model in this tutorial. The part about standalone
prediction was an add-on.

Sounak sahoo March 3, 2017 at 7:39 pm # Start Machine Learning ×

REPLY 

You
what exactly is the work of “seed” in the neural can master
network code?applied
what Machine Learning
does it do?
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Jason Brownlee March 6, 2017 at 10:44 am #Email Address

Seed refers to seeding the random number generator so that the same sequence of
START
random numbers is generated each time the example MY EMAIL COURSE
is run.

The aim is to make the examples 100% reproducible, but this is hard with symbolic math libs like
Theano and TensorFlow backends.

For more on randomness in machine learning, see this post:

http://machinelearningmastery.com/randomness-in-machine-learning/

REPLY 
Priya Sundari March 3, 2017 at 10:19 pm #

hello sir
could you plz tell me what is the role of optimizer and binary_crossentropy exactly? it is written that
optimizer is used to search through the weights of the network which weights are we talking about
exactly?
Start Machine Learning

REPLY 
Jason Brownlee March 6, 2017 at 10:48 am #

Hi Priya,

You can learn more about the fundamentals of neural nets here:
http://machinelearningmastery.com/neural-networks-crash-course/

REPLY 
Bogdan March 3, 2017 at 10:23 pm #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 59/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

If I am not mistaken, those lines I commented about used when we write

init = ‘uniform’

REPLY 
Bogdan March 3, 2017 at 10:44 pm #

Could you explain in more details what is the batch size?

×
REPLY 
Jason Brownlee March 6, 2017 at 10:50 am #
Start Machine Learning
Hi Bogdan,
You can master applied Machine Learning
Batch size is how many patterns to show to the network
withoutbefore
math the weights
or fancy are updated with the
degrees.
accumulated errors. The smaller the batch, the faster
Findthe
out learning, but
how in this also
free andthe more noisy
practical the
course.
learning (higher variance).
Email
Try exploring different batch sizes and see the effect Address
on the train and test performance over each
epoch.

START MY EMAIL COURSE

REPLY 
Mohammad March 7, 2017 at 6:50 am #

Dear Jason
Firstly, thanks for your great tutorials.
I am trying to classify computer networks packets using first 500 bytes of every packet to identify its
protocol. I am trying to use 1d convolution. for simpler task,I just want to do binary classification and then
tackle multilabel classification for 10 protocols. Here is my code but the accuracy which is like .63. how
can I improve the performance? should I Use RNNs?
########
model=Sequential()
model.add(Convolution1D(64,10,border_mode=’valid’,
activation=’relu’,subsample_length=1, input_shape=(500, 1)))
#model.add(Convolution2D(32,5,5,border_mode=’valid’,input_shape=(1,28,28),))
Start Machine Learning
model.add(MaxPooling1D(2))
model.add(Flatten())
model.add(Dense(200,activation=’relu’))
model.add(Dense(1,activation=’sigmoid’))
model.compile(loss=’binary_crossentropy’,
optimizer=’adam’,metrics=[‘accuracy’])
model.fit(train_set, y_train,
batch_size=250,
nb_epoch=30,
show_accuracy=True)
#x2= get_activations(model, 0,xprim )

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 60/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

#score = model.evaluate(t, y_test, show_accuracy = True, verbose = 0)

#print(score[0])

REPLY 
Jason Brownlee March 7, 2017 at 9:37 am #

This post lists some ideas to try an lift performance:

http://machinelearningmastery.com/improve-deep-learning-performance/

REPLY 
Damiano March 7, 2017 at 10:13 pm #

Hi Jason, thank you so much for this awesomeStart Machine

tutorial. Learning
I have just started with python and
×
machine learning.
You can master applied Machine Learning
I am joking with the code doing few changes, for example i have changed..
without math or fancy degrees.
this: Find out how in this free and practical course.

# create model
model = Sequential() Email Address
model.add(Dense(250, input_dim=8, init=’uniform’, activation=’relu’))
model.add(Dense(200, init=’uniform’, activation=’relu’)) START MY EMAIL COURSE
model.add(Dense(200, init=’uniform’, activation=’relu’))
model.add(Dense(1, init=’uniform’, activation=’sigmoid’))

and this:

model.fit(X, Y, nb_epoch=250, batch_size=10)

then i would like to pass some arrays for prediction so…

new_input = numpy.array([[3,88,58,11,54,24.8,267,22],[6,92,92,0,0,19.9,188,28],
[10,101,76,48,180,32.9,171,63], [2,122,70,27,0,36.8,0.34,27], [5,121,72,23,112,26.2,245,30]])

predictions = model.predict(new_input)
print predictions # [1.0, 1.0, 1.0, 0.0, 1.0]

is this correct? In this example i used the same series of training (that have 0 class), but i am getting
wrong results. Only one array is correctly predicted.

Thank you so much! Start Machine Learning

REPLY 
Jason Brownlee March 8, 2017 at 9:41 am #

Looks good. Perhaps you could try changing the configuration of your model to make it
more skillful?

See this post:

http://machinelearningmastery.com/improve-deep-learning-performance/

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 61/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
ANJI March 13, 2017 at 8:48 pm #

hello sir,
could you please tell me to rectify my error below it is raised while model is training:

str(array.shape))
ValueError: Error when checking model input: expected convolution2d_input_1 to have 4 dimensions,
but got array with shape (68, 28, 28).

REPLY 
Jason Brownlee March 14, 2017 at 8:17 am #

It looks like you are working with CNN, not related to this tutorial.
Start Machine Learning ×
Consider trying this tutorial to get familiar with CNNs:
You can master applied Machine Learning
http://machinelearningmastery.com/handwritten-digit-recognition-using-convolutional-neural-
without math or fancy degrees.
networks-python-keras/
Find out how in this free and practical course.

Email Address
REPLY 
Rimjhim March 14, 2017 at 8:21 pm #
START MY EMAIL COURSE
I want a neural that can predict sin values. Further from a given data set i need to determine the
function(for example if the data is of tan or cos, then how to determine that data is of tan only or cos
only)

Thanks in advance

REPLY 
Sudarshan March 15, 2017 at 11:19 pm #

Keras just updated to Keras 2.0. I have an updated version of this code here:
https://github.com/sudarshan85/keras-projects/tree/master/mlm/pima_indians

REPLY 
Jason Brownlee March 16, 2017 at 7:59 am #
Start Machine Learning

Nice work.

REPLY 
subhasish March 16, 2017 at 5:09 pm #

hello sir,
can we use PSO (particle swarm optimisation) in this? if so can you tell how?

REPLY 
Jason Brownlee March 17, 2017 at 8:25 am #
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 62/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Sorry, I don’t have an example of PSO for fitting neural network weights.

REPLY 
Ananya Mohapatra March 16, 2017 at 10:03 pm #

hello sir,
what type of neural network is used in this code? as there are 3 types of Neural network that are…
feedforward, radial basis function and recurrent neurak network.

REPLY 
Jason Brownlee March 17, 2017 at 8:28 am #

Start
A multilayer perceptron (MLP) neural network. Machine
A classic Learning
type from the 1980s.
×
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course. REPLY 
Diego March 17, 2017 at 3:58 am #

got this error while compiling.. Email Address

sigmoid_cross_entropy_with_logits() got an unexpected keyword argument ‘labels’

START MY EMAIL COURSE

REPLY 
Jason Brownlee March 17, 2017 at 8:30 am #

Perhaps confirm that your libraries are all up to date (Keras, Theano or TensorFlow)?

REPLY 
Rohan March 20, 2017 at 5:20 am #

Hi Jason!

I am trying to use two odd frames of a video to predict the even one. Thus I need to give two images as
input to the network and get one image as output. Can you help me with the syntax for the first
model.add()? I have X_train of dimension (190, 2, 240, 320, 3) where 190 are the number of odd pairs, 2
are the two odd images, and (240,320,3) are the (height, width,
Start depth)Learning
Machine of each image.

REPLY 
Herli Menezes March 21, 2017 at 8:33 am #

Hello, Jason,
Thanks for your good tutorial. However i found some issues:
Warnings like these:

1 – Warning (from warnings module):

File “/usr/lib/python2.7/site-packages/keras/legacy/interfaces.py”, line 86
‘ call to the Keras 2 API: ' + signature)

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 63/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

UserWarning: Update your Dense call to the Keras 2 API: Dense(12, activation=”relu”,
kernel_initializer=”uniform”, input_dim=8)

2 - Warning (from warnings module):

File "/usr/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 86
' call to the Keras 2 API: ‘ + signature)
UserWarning: Update your Dense call to the Keras 2 API: Dense(8, activation="relu",
kernel_initializer="uniform")

3 – Warning (from warnings module):

File “/usr/lib/python2.7/site-packages/keras/legacy/interfaces.py”, line 86
‘ call to the Keras 2 API: ' + signature)
UserWarning: Update your Dense call to the Keras 2 API: Dense(1, activation=”sigmoid”,
kernel_initializer=”uniform”)

3 - Warning (from warnings module): Start Machine Learning ×

File "/usr/lib/python2.7/site-packages/keras/models.py", line 826
You can master applied Machine Learning
warnings.warn('The nb_epoch argument in fit '
without math or fancy degrees.
UserWarning: The nb_epoch argument in fit has been renamed epochs`.
Find out how in this free and practical course.
I think these are due to some package update..

But, the output of predictions was an array of zeros… Email Address

such as: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
START MY EMAIL COURSE
0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ….0.0]

I am running in a Linux Machine, Fedora 24,

Python 2.7.13 (default, Jan 12 2017, 17:59:37)
[GCC 6.3.1 20161221 (Red Hat 6.3.1-1)] on linux2

Why?

Thank you!

REPLY 
Jason Brownlee March 21, 2017 at 8:45 am #

These look like warnings related to the recent Keras 2.0 release.

They look like just warning and that you can still run the example.
Start Machine Learning
I do not know why you are getting all zeros. I will investigate.

REPLY 
Ananya Mohapatra March 21, 2017 at 6:21 pm #

hello sir,
can you please help me build a recurrent neural network with the above given dataset. i am having a bit
trouble in building the layers…

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 64/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee March 22, 2017 at 7:56 am #

Hi Ananya ,

The Pima Indian diabetes dataset is a binary classification problem. It is not appropriate for a
Recurrent Neural Network as there is no sequence information to learn.

REPLY 
Ananya Mohapatra March 22, 2017 at 8:04 pm #

sir so could you tell on which type of dataset would the recurrent neural network
accurately work? i have the dataset of EEG signals of epileptic patients…will recurrent network
work on this? Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees.
Jason Brownlee March 23, 2017 atFind
8:49out
am how
# in this free and practical course. REPLY 

It may if it is regular enough.

Email Address
LSTMs are excellent at sequence problems that have regularity or clear signals to detect.

START MY EMAIL COURSE

REPLY 
Shane March 22, 2017 at 5:18 am #

Hi Jason, I have a quick question related to an error I am receiving when running the code in
the tutorial…

When I run

# Compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

Python returns the following error:

sigmoid_cross_entropy_with_logits() got an unexpected keyword argument ‘labels’

Start Machine Learning

REPLY 
Jason Brownlee March 22, 2017 at 8:09 am #

Sorry, I have not seen this error Shane.

Perhaps check that your environment is up to date with the latest versions of the deep learning
libraries?

REPLY 
Tejes March 24, 2017 at 1:04 am #

Hi Jason,
Thanks for this awesome post.
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 65/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

I ran your code with tensorflow back end, just out of curiosity. The accuracy returned was different every
time I ran the code. That didn’t happen with Theano. Can you tell me why?

Thanks in advance!

REPLY 
Jason Brownlee March 24, 2017 at 7:56 am #

You will get different accuracy each time you run the code because neural networks are
stochastic.

This is not related to the backend (I expect).

More on randomness in machine learning here:

Start Machine
http://machinelearningmastery.com/randomness-in-machine-learning/ Learning ×
You can master applied Machine Learning
without math or fancy degrees.
Saurabh Bhagvatula March 27, 2017 at 9:49 pmFind
# out how in this free and practical course. REPLY 

Hi Jason,
Email Address
I’m new to deep learning and learning it from your tutorials, which previously helped me understand
Machine Learning very well.
In the following code, I want to know why the number of START MYdiffer
neurons EMAIL COURSE
from input_dim in first layer of
Nueral Net.
# create model
model = Sequential()
model.add(Dense(12, input_dim=8, init=’uniform’, activation=’relu’))
model.add(Dense(8, init=’uniform’, activation=’relu’))
model.add(Dense(1, init=’uniform’, activation=’sigmoid’))

REPLY 
Jason Brownlee March 28, 2017 at 8:22 am #

You can specify the number of inputs via “input_dim”, you can specify the number of
neurons in the first hidden layer as the first parameter to Dense().

Start Machine Learning

REPLY 
Saurabh Bhagvatula March 28, 2017 at 4:15 pm #

Thanx a lot.

REPLY 
Jason Brownlee March 29, 2017 at 9:05 am #

You’re welcome.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 66/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Nalini March 29, 2017 at 2:52 am # REPLY 

Hi Jason

while running this code for k fold cross validation it is not working.please give the code for k fold cross
validation in binary class

REPLY 
Jason Brownlee March 29, 2017 at 9:10 am #

Generally neural nets are too slow/large for k-fold cross validation.

Nevertheless, you can use a sklearn wrapper for a keras model and use it with any sklearn
resampling method:
Start Machine Learning
http://machinelearningmastery.com/evaluate-performance-machine-learning-algorithms-python-
×
using-resampling/
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
trangtruong March 29, 2017 at 7:04 pm #
Email Address
Hi Jason, why i use function evaluate to get accuracy score my model with test dataset, it return
result >1, i can’t understand.
START MY EMAIL COURSE

REPLY 
enixon April 3, 2017 at 3:08 am #

Hey Jason, thanks for this great article! I get the following error when running the code above:

TypeError: Received unknown keyword arguments: {‘epochs’: 150}

Any ideas on why that might be? I can’t get ‘epochs’, nb_epochs, etc to work…

REPLY 
Jason Brownlee April 4, 2017 at 9:07 am #

You need to update to Keras version 2.0 or higher.

Start Machine Learning

REPLY 
Ananya Mohapatra April 5, 2017 at 9:30 pm #

def baseline_model():
# create model
model = Sequential()
model.add(Dense(10, input_dim=25, init=’normal’, activation=’softplus’))
model.add(Dense(3, init=’normal’, activation=’softmax’))
# Compile model
model.compile(loss=’mean_squared_error’, optimizer=’adam’, metrics=[‘accuracy’])
return model

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 67/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

sir here mean_square_error has been used for loss calculation. Is it the same as LMS algorithm. If not,
can we use LMS , NLMS or RLS to calculate the loss?

REPLY 
Ahmad Hijazi April 5, 2017 at 10:19 pm #

Hello Jason, thank you a lot for this example.

My question is, after I trained the model and an accuracy of 79.2% for example is obtained successfully,
how can I test this model on new data?

for example if a new patient with new records appear, I want to guess the result (0 or 1) for him, how can
I do that in the code?

Start Machine Learning ×

You can master applied Machine Learning REPLY 
Jason Brownlee April 9, 2017 at 2:36 pm #
without math or fancy degrees.
Find out how in this free and practical course.
You can fit your model on all available training data then make predictions on new data as
follows:
Email Address
1 yhat = model.predict(X)

START MY EMAIL COURSE

REPLY 
Perick Flaus April 6, 2017 at 12:16 am #

Thanks Jason, how can we test if new patient will be diabetic or no (0 or 1) ?

REPLY 
Jason Brownlee April 9, 2017 at 2:36 pm #

Fit the model on all training data and call:

1 yhat = model.predict(X)

REPLY 
Gangadhar April 12, 2017 at 1:28 am # Start Machine Learning

Dr Jason,

In compiling the model i got below error

TypeError: compile() got an unexpected keyword argument ‘metrics’

unable to resolve the below error

REPLY 
Jason Brownlee April 12, 2017 at 7:53 am #

Ensure you have the latest version of Keras, v2.0 or higher.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 68/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Omogbehin Azeez April 13, 2017 at 1:48 am #

Hello sir,
Thank you for the post. A quick question, my dataset has 24 input and 1 binary output( 170 instances,
100 epoch , hidden layer=6 and 10 batch, kernel_initializer=’normal’) . I adapted your code using Tensor
flow and keras. I am having an accuracy of 98 to 100 percent. I am scared of over-fitting in my model. I
need your candid advice. Kind regards sir

REPLY 
Jason Brownlee April 13, 2017 at 10:07 am #
Start Machine Learning ×
Yes, evaluate your model using k-fold cross-validation to ensure you are not tricking
yourself. You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Omogbehin Azeez April 14, 2017 at 1:08Email
am # Address

Thank you sir

START MY EMAIL COURSE

REPLY 
Sethu Baktha April 13, 2017 at 5:19 am #

Hi Jason,
If I want to use the diabetes dataset (NOT Pima) https://archive.ics.uci.edu/ml/datasets/Diabetes to
predict Blood Glucose which tutorials and e-books of yours would I need to start with…. Also, the data in
its current format with time, code and value is it usable as is or do I need to convert the data in another
format to be able to use it.

Thanks for your help

REPLY 
Jason Brownlee April 13, 2017 at 10:13 am #
Start Machine Learning
This process will help you frame and work through your dataset:
http://machinelearningmastery.com/start-here/#process

I hope that helps as a start.

REPLY 
Sethu Baktha April 13, 2017 at 10:25 am #

Dr. Jason,
The data is time series(time based data) with categorical(20) with two numbers one for insulin
level and another for blood sugar level… Each time series data does not have every categorical

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 69/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

data… For example one category is blood sugar before breakfast, another category is blood
sugar after breakfast, before lunch and after lunch… Some times some of these category data is
missing… I read through the above link, but does not talk about time series, categorical data
with some category of data missing what to do in those cases…. Please let me know if any of
your books will help clarify these points?

REPLY 
Jason Brownlee April 14, 2017 at 8:43 am #

Hi Sethu,

I have many posts on time series that will help. Get started here:
http://machinelearningmastery.com/start-here/#timeseries
Start Machine Learning
With categorical data, I would recommend an integer encoding perhaps followed by a one-
×
hot encoding. You can learn more about these encodings here:
You can master applied Machine Learning
http://machinelearningmastery.com/data-preparation-gradient-boosting-xgboost-python/
without math or fancy degrees.
I hope that helps. Find out how in this free and practical course.

Email Address

REPLY 
Omogbehin Azeez April 14, 2017 at 9:49 am #
START MY EMAIL COURSE
Hello sir,

Is it compulsory to normalize the data before using ANN model. I read it somewhere I which the author
insisted that each attribute be comparable on the scale of [0,1] for a meaningful model. What is your take
on that sir. Kind regards.

REPLY 
Jason Brownlee April 15, 2017 at 9:29 am #

Yes. You must scale your data to the bounds of the activation used.

REPLY 
shiva April 14, 2017 at 10:38 am #
Start Machine Learning
Hi Jason, You are simply awesome. I’m one of the many who got benefited from your book
“machine learning mastery with python”. I’m working with a medical image classification problem. I have
two classes of medical images (each class having 1000 images of 32*32) to be worked upon by the
convolutional neural networks. Could you guide me how to load this data to the keras dataset? Or how to
use my data while following your simple steps? kindly help.

REPLY 
Jason Brownlee April 15, 2017 at 9:30 am #

Load the data as numpy arrays and then you can use it with Keras.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 70/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Omogbehin Azeez April 18, 2017 at 12:09 am #

Hello sir,

I adapted your code with the cross validation pipelined with ANN (Keras) for my model. It gave me 100%
still. I got the data from UCI ( Chronic Kidney Disease). It was 400 instances, 24 input attributes and 1
binary attribute. When I removed the rows with missing data I was left with 170 instances. Is my dataset
too small for (24 input layer, 24 hidden layer and 1 output layer ANN, using adam and kernel initializer as
uniform )?

Jason Brownlee April 18, 2017 at 8:32 am #

Start Machine Learning ×
REPLY 

It is not too small.

You can master applied Machine Learning
withouton
Generally, the size of the training dataset really depends math
howoryou
fancy degrees.
intend to use the model.
Find out how in this free and practical course.

Email Address
REPLY 
Omogbehin Azeez April 18, 2017 at 11:10 pm #

Thank you sir for the response, I guessSTART

I haveMY
to contend with the over-fitting of my
EMAIL COURSE
model.

REPLY 
Padmanabhan Krishnamurthy April 19, 2017 at 6:26 pm #

Hi Jason,

Great tutorial. Love the site

Just a quick query : why have you used adam as an optimizer over sgd? Moreover, when do we use sgd
optimization, and what exactly does it involve?

Thanks

Start Machine Learning

REPLY 
Jason Brownlee April 20, 2017 at 9:23 am #

ADAM seems to consistently work well with little or no customization.

SGD requires configuration of at least the learning rate and momentum.

Try a few methods and use the one that works best for your problem.

REPLY 
Padmanabhan Krishnamurthy April 20, 2017 at 4:32 pm #

Thanks

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 71/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Omogbehin Azeez April 25, 2017 at 8:13 am #

Hello sir,

Good day sir, how can I get all the weights and biases of the keras ANN. Kind regards.

REPLY 
Jason Brownlee April 26, 2017 at 6:19 am #

You can save the network weights, see this post:

http://machinelearningmastery.com/save-load-keras-deep-learning-models/

Start
You can also use the API to access the weights directly. Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees.

Shiva April 27, 2017 at 5:43 am # Find out how in this free and practical course. REPLY 

Hi Jason, Email Address

I am currently working with the IMDB sentiment analysis problem as mentioned in your book. Am using
Anaconda 3 with Python 3.5.2. In an attempt to summarize the review length as you have mentioned in
your book, When i try to execute the command: START MY EMAIL COURSE

result = map(len, X)
print(“Mean %.2f words (%f)” % (numpy.mean(result), numpy.std(result)))

it returns the error: unsupported operand type(s) for /: ‘map’ and ‘int’

kindly help with the modified syntax. looking forward…

REPLY 
Jason Brownlee April 27, 2017 at 8:47 am #

I’m sorry to hear that. Perhaps comment out that line?

Or change it to remove the formatting and just print the raw mean and stdev values for you to
review?

Start Machine Learning

REPLY 
Elikplim May 1, 2017 at 1:58 am #

Hello, quite new to Python, Numpy and Keras(background in PHP, MYSQL etc). If there are 8
input variables and 1 output varable(9 total), and the Array indexing starts from zero(from what I’ve
gathered it’s a Numpy Array, which is built on Python lists) and the order is [rows, columns], then
shouldn’t our input variable(X) be X = dataset[:,0:7] (where we select from the 1st to 8th columns, ie. 0th
to 7th indices) and output variable(Y) be Y = dataset[:,8] (where we the 9th column, ie. 8th index)?

REPLY 
Jason Brownlee May 1, 2017 at 5:59 am #
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 72/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

You can learn more about array indexing in numpy here:

https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html

REPLY 
Jackie Lee May 1, 2017 at 12:47 pm #

I’m having troubles with the predictions part. It saves ValueError: Error when checking model
input: expected dense_1_input to have shape (None, 502) but got array with shape (170464, 502)

### MAKE PREDICTIONS ###

testset = numpy.loadtxt(“right_stim_FD1.csv”, delimiter=”,”)
A = testset[:,0:502]
B = testset[:,502]
Start Machine Learning
probabilities = model.predict(A, batch_size=10, verbose=1) ×
predictions = float(round(a) for a in probabilities)
accuracy = numpy.mean(predictions == B) You can master applied Machine Learning
#round predictions without math or fancy degrees.
#rounded = [round(x[0]) for x in predictions] Find out how in this free and practical course.

print(predictions)
print(“Prediction Accuracy: %.2f%%” % (accuracy*100))Email Address

START MY EMAIL COURSE

REPLY 
Jason Brownlee May 2, 2017 at 5:55 am #

It looks like you might be giving the entire dataset as the output (y) rather than just the
output variable.

REPLY 
Anastasios Selalmazidis May 2, 2017 at 12:27 am #

Hi there,

I have a question regarding deep learning. In this tutorial we build a MLP with Keras. Is this Deep
Learning or is it just a MLP Backpropagation ?

Start Machine Learning

REPLY 
Jason Brownlee May 2, 2017 at 5:59 am #

Deep learning is MLP backprop these days:

http://machinelearningmastery.com/what-is-deep-learning/

Generally, deep learning refers to MLPs with lots of layers.

REPLY 
Eric T May 2, 2017 at 8:59 pm #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 73/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Hi,
Would you mind if I use this code as an example of a simple network in a school project of mine?
Need to ask before using it, since I cannot find anywhere in this tutorial that you are OK with anyone
using the code, and the ethics moment of my course requires me to ask (and of course give credit where
credit is due).
Kind regards
Eric T

REPLY 
Jason Brownlee May 3, 2017 at 7:35 am #

Yes it’s fine but I take no responsibility and you must credit the source.

I answer this question in my FAQ: Start Machine Learning ×

http://machinelearningmastery.com/start-here/#faq
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
BinhLN May 7, 2017 at 3:11 am #
Email Address
Hi Jason
I have a problem
START MY
My Dataset have 500 record. But My teacher want my dataset EMAIL
have COURSE
100.000 record. I must have a new
algorithm for data generation. Please help me

REPLY 
Dp May 11, 2017 at 2:26 am #

Can you give a deep cnn code which includes 25 layers , in the first conv layer the filter sizs
should be 39×39 woth a total lf 64 filters , in the 2nd conv layer , 21 ×21 with 32 filters , in the 3rd conv
layer 11×11 with 64 filters , 4th Conv layer 7×7 with 32 layers . For a input size of image 256×256. Im
Competely new in this Deep learning Thing but if you can code that for me it would be a great help.
Thanks

REPLY 
Jason Brownlee May 11, 2017 at 8:33 am # Start Machine Learning

Consider using an off-the-shelf model like VGG:

https://keras.io/applications/

REPLY 
Maple May 13, 2017 at 12:58 pm #

I have to follow with the facebook metrics. But the result is very low. Help me.
I changed the input but did not improve
http://archive.ics.uci.edu/ml/datasets/Facebook+metrics

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 74/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee May 14, 2017 at 7:24 am #

I have a list of suggestions that may help as a start:

http://machinelearningmastery.com/improve-deep-learning-performance/

REPLY 
Alessandro May 14, 2017 at 1:01 am #

Hi Jason,

Great Tutorial and thanks for your effort.

I have a question, since I am beginner with keras and tensorflow.

I have installed both of them, keras and tensorflow, theStart Machine
latest version Learning
and I have run your example but I
×
get always the same error:
You can master applied Machine Learning
Traceback (most recent call last): without math or fancy degrees.
File “CNN.py”, line 18, in Find out how in this free and practical course.
model.compile(loss=’binary_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])
File “/Users/MacBookPro1/.virtualenvs/keras_tf/lib/python2.7/site-packages/keras/models.py”,
Email Address line 777,
in compile
**kwargs)
START MY EMAIL COURSE
File “/Users/MacBookPro1/.virtualenvs/keras_tf/lib/python2.7/site-packages/keras/engine/training.py”,
line 910, in compile
sample_weight, mask)
File “/Users/MacBookPro1/.virtualenvs/keras_tf/lib/python2.7/site-packages/keras/engine/training.py”,
line 436, in weighted
score_array = fn(y_true, y_pred)
File “/Users/MacBookPro1/.virtualenvs/keras_tf/lib/python2.7/site-packages/keras/losses.py”, line 51, in
binary_crossentropy
return K.mean(K.binary_crossentropy(y_pred, y_true), axis=-1)
File “/Users/MacBookPro1/.virtualenvs/keras_tf/lib/python2.7/site-
packages/keras/backend/tensorflow_backend.py”, line 2771, in binary_crossentropy
logits=output)
TypeError: sigmoid_cross_entropy_with_logits() got an unexpected keyword argument ‘labels’

Could you help? Thanks

Alessandro Start Machine Learning

REPLY 
Jason Brownlee May 14, 2017 at 7:30 am #

Ouch, I have not seen this error before.

Some ideas:
– Consider trying the theano backend and see if that makes a difference.
– Try searching/posting on the keras user group and slack channel.
– Try searching/posting on stackoverflow or cross validated.

Let me know how you go.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 75/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Alessandro May 14, 2017 at 9:44 am #

Hi Jason,

I found the issue. The tensorflow installation was outdated; so I have updated it and everything
is working nicely.

Good night,
Alessandro

Start
Jason Brownlee May 15, 2017 at 5:50 am # Machine Learning ×
REPLY 

I’m glad to hear it Alessandro. You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Sheikh Rafiul Islam May 25, 2017 at 3:36 pm # Email Address

Thank you Mr. Brownlee for your wonderful easy to understand explanation
START MY EMAIL COURSE

REPLY 
Jason Brownlee June 2, 2017 at 11:41 am #

Thnaks.

REPLY 
WAZED May 29, 2017 at 12:31 am #

Hi Jason,
Thank you very much for your wonderful tutorial. I have a question regarding the metrices.Is there
default way to declare metrices “Precision” and “Recall” in addtion with the “Accurace”.

Br
WAZED Start Machine Learning

REPLY 
Jason Brownlee June 2, 2017 at 12:15 pm #

Yes, see here:

https://keras.io/metrics/

REPLY 
chiranjib konwar May 29, 2017 at 4:30 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 76/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Hi Jason,

please send me a small note containing resources from where i can learn deep learning from scratch.
thanks for the wonderful read you had prepared.

Thanks in advance

yes, my email id is [email protected]

REPLY 
Jason Brownlee June 2, 2017 at 12:16 pm #

Here:
http://machinelearningmastery.com/start-here/#deeplearning
Start Machine Learning ×
You can master applied Machine Learning
REPLY 
Jeff June 1, 2017 at 11:48 am # without math or fancy degrees.
Find out how in this free and practical course.
Why the NN have mistakes many times?

Email Address

REPLY 
Jason Brownlee June 2, 2017 at 12:54 pm # START MY EMAIL COURSE

What do you mean exactly?

REPLY 
kevin June 2, 2017 at 5:53 pm #

Hi Jason,

I seem to be getting an error when applying the fit method:

ValueError: Error when checking input: expected dense_1_input to have shape (None, 12) but got array
with shape (767, 8)

I looked this up and the most prominent suggestion seemed to be upgrade keras and theno, which I did,
but that didn’t resolve the problem.
Start Machine Learning

REPLY 
Jason Brownlee June 3, 2017 at 7:24 am #

Ensure you have copied the code exactly from the post.

REPLY 
Hemanth Kumar K June 3, 2017 at 2:15 pm #

hi Jason,
I am stuck with an error

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 77/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

TypeError: sigmoid_cross_entropy_with_logits() got an unexpected keyword argument ‘labels’

my tensor flow and keras virsions are
keras: 2.0.4
Tensorflow: 0.12

REPLY 
Jason Brownlee June 4, 2017 at 7:46 am #

I’m sorry to hear that, I have not seen that error before. Perhaps you could post a question
to stackoverflow or the keras user group?

xena June 4, 2017 at 6:36 pm #

Start Machine Learning ×
REPLY 

You can master applied Machine Learning

can anyone tell me which neural network is being used here? Is it MLP??
without math or fancy degrees.
Find out how in this free and practical course.

Jason Brownlee June 5, 2017 at 7:40 am # Email Address REPLY 

Yes, it is a multilayer perceptron (MLP) feedforward neural network.

START MY EMAIL COURSE

REPLY 
Nirmesh Shah June 9, 2017 at 11:00 pm #

Hi Jason,

I have run this code successfully on PC with CPU.

If I have to run the same code n another PC which contains GPU, What line should I add to make it sure
that it runs on the GPU

REPLY 
Jason Brownlee June 10, 2017 at 8:24 am #

The code would stay the same, your configuration of the Learning
Start Machine Keras backend would change.

Please refer to TensorFlow or Theano documentation.

REPLY 
Prachi June 12, 2017 at 7:30 pm #

What if I want to train my neural which should detect whether the luggage is abandoned or not ?
How do i proceed for it ?

REPLY 
Jason Brownlee June 13, 2017 at 8:18 am #
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 78/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

This process will help you work through your predictive modeling problem end to end:
http://machinelearningmastery.com/start-here/#process

REPLY 
Ebtesam June 14, 2017 at 11:15 pm #

Hi
I was build neural machine translation model but the score i was get is 0 i am not sure why

REPLY 
Jason Brownlee June 15, 2017 at 8:45 am #

Here is a good list of things to try: Start Machine Learning ×

http://machinelearningmastery.com/improve-deep-learning-performance/
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Sarvottam Patel June 20, 2017 at 7:31 pm #
Email Address
HHey Jason , first of all thank you very much from the core of my heart to make me understand
this perfectly, I have an error after completing 150 iteration.
START MY EMAIL COURSE
File “keras_first_network.py”, line 53, in
print(“\n%s: %.2f” %(model.metrics_names[1]*100))
TypeError: not enough arguments for format string

REPLY 
Sarvottam Patel June 20, 2017 at 8:05 pm #

Sorry Sir my bad , actually I wrote it wrongly

REPLY 
Jason Brownlee June 21, 2017 at 8:12 am #

Confirm that you have copied the line exactly:

Start Machine Learning
1 print("\n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

REPLY 
Joydeep June 30, 2017 at 4:15 pm #

Hi Dr Jason,

Thanks for the tutorial to get started using Keras.

I used the below snippet to directly load the dataset from the URL rather than downloading and saving
as this makes the code more streamlined without having to navigate elsewhere.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 79/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

# load pima indians dataset

datasource = numpy.DataSource().open(“http://archive.ics.uci.edu/ml/machine-learning-databases/pima-
indians-diabetes/pima-indians-diabetes.data”)
dataset = numpy.loadtxt(datasource, delimiter=”,”)

REPLY 
Jason Brownlee July 1, 2017 at 6:28 am #

Thanks for the tip.

Yvette July 7, 2017 at 9:01 pm #

Start Machine Learning ×
REPLY 

Thanks for this helpful resource!

You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Jason Brownlee July 9, 2017 at 10:38 am #
Email Address
I’m glad it helped.

START MY EMAIL COURSE

REPLY 
Andeep July 10, 2017 at 1:14 am #

Hi Dr Brownlee,

thank you very much for this great tutorial!

I would be grateful, if you could answer some questions:

1. What does the 7 in “numpy.random.seed(7)” means?

2. In my case I have 3 input neurons and 2 output neurons. Is the correct notation:
X = dataset[:,0:3]
Y = dataset[:,3:4] ?

3. The batch size means how many training data are used in one epoch, am I right?
I have thought we have to use the whole training data set for the training. In this case I would determine
Start
the batch size as the number of training data pairs I have Machine
achieved Learning
through experiments etc.. In your
example, does the batch (sized 10) means that the computer always uses the same 10 training data in
every epoch or are the 10 training data randomly chosen among all training data before every epoch?

4. When evaluating the model what does the loss means (e.g. in loss: 0.5105 – acc: 0.7396)?
Is it the sum of values of the error function (e.g. mean_squared_error) of the output neurons?

REPLY 
Jason Brownlee July 11, 2017 at 10:19 am #

You can use any random seed you like, more here:
http://machinelearningmastery.com/reproducible-results-neural-networks-keras/

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 80/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

You are referring to the columns in your data. Your network will also need to be configured with the
correct number of inputs and outputs (e.g. input and output layers).

Batch size is the number of samples in the dataset to work through before updating network weights.
One epoch is comprised of one or more batches.

Loss is the term being optimized by the network. Here we use log loss:
https://en.wikipedia.org/wiki/Cross_entropy

REPLY 
Andeep July 16, 2017 at 7:43 am #

Thank you for your response, Dr Brownlee !!

Start Machine Learning ×

You can master applied Machine Learning REPLY 
Jason Brownlee July 16, 2017 at 8:00 am #
without math or fancy degrees.
Find out how in this free and practical course.
I hope it helps.

Email Address

REPLY 
Patrick Zawadzki July 11, 2017 at 5:35 am #
START MY EMAIL COURSE

Is there anyway to see the relationship between these inputs? Essentially understand which
inputs affect the output the most, or perhaps which pairs of inputs affect the output the most?

Maybe pairing this with unsupervised deep learning? I want to have less of a “black box” for the
developed network if at all possible. Thank you for your great content!

REPLY 
Jason Brownlee July 11, 2017 at 10:34 am #

Yes, try and RFE:

http://machinelearningmastery.com/feature-selection-machine-learning-python/

Start Machine Learning REPLY 

Bernt July 13, 2017 at 10:12 pm #

Hi Jason,
Thank you for sharing your skills and competence.

I want to study the change in weights and predictions between each epoch run.
Have tried to use the model.train_on_batch method and the model.fit method with epoch=1 and
batch_size equal all the samples.

But it seems like the model doesn’t save the new updated weights.
I print predictions before and after I dont see a change in the evaluation scores.

Parts of the code is printed below.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 81/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Any idea?
Thanks.

# Compile model
model.compile(loss=’binary_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])

# evaluate the model

scores = model.evaluate(X, Y)
print(“\n%s: %.2f%%” % (model.metrics_names[1], scores[1]*100))

# Run one update of the model trained run with X and compared with Y
model.train_on_batch(X, Y)

# Fit the model

model.fit(X, Y, epochs=1, batch_size=768)

scores = model.evaluate(X, Y) Start Machine Learning ×

print(“\n%s: %.2f%%” % (model.metrics_names[1], scores[1]*100))
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Jason Brownlee July 14, 2017 at 8:29 am #
Email Address
Sorry, I have not explored evaluating a Keras model this way.

Perhaps it is a fault, I would recommend preparing the smallest

START possible
MY EMAIL example that demonstrates
COURSE
the issue and post to the Keras GitHub issues.

REPLY 
iman July 18, 2017 at 11:18 pm #

Hi, I tried to apply this to the titanic data set, however the predictions were all 0.4. What do you
suggest for:
# create model
model = Sequential()
model.add(Dense(12, input_dim=4, activation=’relu’))
model.add(Dense(4, activation=’relu’))
model.add(Dense(1, activation=’sigmoid’))

model.compile(loss=’binary_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’]) #’sgd’

Start Machine Learning
model.fit(X, Y, epochs=15, batch_size=10)

REPLY 
Jason Brownlee July 19, 2017 at 8:26 am #

This post will give you some ideas to list the skill of your model:
http://machinelearningmastery.com/improve-deep-learning-performance/

REPLY 
Camus July 19, 2017 at 2:14 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 82/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Hi Dr Jason,
This is probably a stupid question but I cannot find out how to do it … and I am beginner on Neural
Network.
I have relatively same number of inputs (7) and one output. This output can take numbers between
-3000 and +3000.
I want to build a neural network model in python but I don’t know how to do it.
Do you have an example with outputs different from 0-1.
Tanks in advance

Camus

REPLY 
Jason Brownlee July 19, 2017 at 8:28 am #
Start Machine Learning ×
Ensure you scale your data then use the above tutorial to get started.
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Khalid Hussain July 21, 2017 at 11:28 pm #
Email Address
Hi Jason Brownlee

I am using the same data “pima-indians-diabetes.csv” but all predicted values are less then 1 and are in
START MY EMAIL COURSE
fraction which could not distinguish any class.

If I round off then all become 0.

I am using model.predict(x) function

You are requested to kindly guide me what I am doing wrong are how can I achieve correct predicted
value.

Thank you

REPLY 
Jason Brownlee July 22, 2017 at 8:36 am #

Consider you have copied all of the code exactly from the tutorial.

Start Machine Learning

REPLY 
Ludo July 25, 2017 at 6:59 pm #

Hello Jason,

Thanks you for your great example. I have some comments.

– Why you have choice “12” inputs hidden layers ? and not 24 / 32 .. it’s arbitary ?
– Same question about epochs and batch_size ?

This value are very sensible !! i have try with 32 inputs first layer , epchos=500 and batch_size=1000
and the result is very differents… i’am at 65% accurancy.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 83/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Thx for you help.

Regards.

REPLY 
Jason Brownlee July 26, 2017 at 7:50 am #

Yes, it is arbitrary. Tune the parameters of the model to your problem.

REPLY 
Almoutasem Bellah Rajab July 25, 2017 at 7:32 pm #

Wow, you’re still replying to comments more than a year later!!!… you’re great,, thanks..
Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees. REPLY 
Jason Brownlee July 26, 2017 at 7:50 am #
Find out how in this free and practical course.
Yep.
Email Address

START MY EMAIL COURSE REPLY 

Jane July 26, 2017 at 1:23 am #

Thanks for your tutorial, I found it very useful to get me started with Keras. I’ve previously tried
TensorFlow, but found it very difficult to work with. I do have a question for you though. I have both
Theano and TensorFlow installed, how do I know which back-end Keras is using? Thanks again

REPLY 
Jason Brownlee July 26, 2017 at 8:02 am #

Keras will print which backend it uses every time you run your code.

You can change the backend in the Keras configuration file (~/.keras/keras.json) which looks like:

1 {
2 "image_data_format": "channels_last",
3 "backend": "tensorflow",
4 "epsilon": 1e-07, Start Machine Learning
5 "floatx": "float32"
6 }

REPLY 
Masood Imran July 28, 2017 at 12:00 am #

Hello Jason,

My understanding of Machine Learning or evaluating deep learning models is almost 0. But, this article
gives me lot of information. It is explained in a simple and easy to understand language.

Thank you very much for this article. Would you suggest any good read to further explore Machine
Learning or deep learning models please?
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 84/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee July 28, 2017 at 8:31 am #

Thanks.

Yes, start right here:

http://machinelearningmastery.com/start-here/#deeplearning

REPLY 
Peggy August 3, 2017 at 7:14 pm #

If I have trained prediction models or neural network function scripts. How can I use them to
Start Machine Learning ×
make predictions in an application that will be used by end users? I want to use python but it seems I will
have to redo the training in Python again. Is there a way
YouI can rewriteapplied
can master the scripts in Python
Machine without
Learning
retraining and just call the function of predicting? without math or fancy degrees.
Find out how in this free and practical course.

Email Address REPLY 

Jason Brownlee August 4, 2017 at 6:58 am #

You need to train and save the final model then

STARTload
MY itEMAIL
to make predictions.
COURSE

This post will make it clear:

http://machinelearningmastery.com/train-final-machine-learning-model/

REPLY 
Shane August 8, 2017 at 2:38 pm #

Jason, I used your tutorial to install everything needed to run this tutorial. I followed your tutorial
and ran the resulting program successfully. Can you please describe what the output means? I would
like to thank you for your very informative tutorials.

REPLY 
Shane August 8, 2017 at 2:39 pm #
Start Machine Learning
768/768 [==============================] – 0s – loss: 0.4807 – acc: 0.7826
Epoch 148/150
768/768 [==============================] – 0s – loss: 0.4686 – acc: 0.7812
Epoch 149/150
768/768 [==============================] – 0s – loss: 0.4718 – acc: 0.7617
Epoch 150/150
768/768 [==============================] – 0s – loss: 0.4772 – acc: 0.7812
32/768 [>………………………..] – ETA: 0s
acc: 77.99%

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 85/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee August 8, 2017 at 5:12 pm #

It is summarizing the training of the model.

The final line evaluates the accuracy of the model’s predictions – really just to demonstrate how
to make predictions.

REPLY 
Jason Brownlee August 8, 2017 at 5:11 pm #

Well done Shane.

Which output?
Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees. REPLY 
Bene August 9, 2017 at 1:02 am #
Find out how in this free and practical course.
Hello Jason, i really liked your Work and it helped me a lot with my first steps.
Email Address
But i am not really familiar with the numpy stuff:

So here is my Question:
START MY EMAIL COURSE
dataset = numpy.loadtxt(“pima-indians-diabetes.csv”, delimiter=”,”)
# split into input (X) and output (Y) variables
X = dataset[:,0:8]
Y = dataset[:,8]

I get that the numpy.loadtxt is extracting the information from the cvs File

but what does the stuff in the Brackets mean like X = dataset[:,0:8]

why the “:” and why , 0:8

its probably pretty dumb but i can’t find a good explanation online

thanks really much!

Jason Brownlee August 9, 2017 at 6:37 am #Start Machine Learning REPLY 

Good question Bene, it’s called array slicing:

https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html

REPLY 
Bene August 9, 2017 at 10:59 pm #

That helped me out tank you Jason

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 86/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Chen August 12, 2017 at 5:43 pm #

Can I translate it to Chinese and put it to Internet in order to let other Chinese people can read
your article?

REPLY 
Jason Brownlee August 13, 2017 at 9:46 am #

No, please do not.

Deep Learning August 12, 2017 at 7:36 pm # Start Machine Learning ×

REPLY 

It seems that using this line: You can master applied Machine Learning
without math or fancy degrees.
np.random.seed(5) Find out how in this free and practical course.
…is redundant i.e. the Keras output in a loop running the same model with the same configuration will
yield a similar variety of results regardless if it’s set at all, or which
Email number it is set to. Or am I missing
Address
something?

START MY EMAIL COURSE

REPLY 
Jason Brownlee August 13, 2017 at 9:52 am #

Deep learning algorithms are stochastic (random within a range). That means that they will
make different predictions/learn different things when the same model is trained on the same data.
This is a feature:
http://machinelearningmastery.com/randomness-in-machine-learning/

You can fix the random seed to ensure you get the same result, and it is a good idea for tutorials to
help beginners out:
http://machinelearningmastery.com/reproducible-results-neural-networks-keras/

When evaluating the skill of a model, I would recommend repeating the experiment n times and
taking skill as the average of the runs. See here for the procedure:
http://machinelearningmastery.com/evaluate-skill-deep-learning-models/
Start Machine Learning
Does that help?

REPLY 
Deep Learning August 14, 2017 at 3:08 am #

Thanks Jason

I totally get what it should do, but as I had pointed out, it does not do it. If you run the codes you
have provided above in a loop for say 10 times. First 10 with random seed set and the other 10
times without that line of code all together. Then compare the result. At least the result I’m
getting, is suggesting the effect is not there i.e. both sets of 10 times will have similar variation in
the result.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 87/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee August 14, 2017 at 6:26 am #

It may suggest that the model is overprescribed and easily addresses the training
data.

REPLY 
Deep Learning August 14, 2017 at 3:12 am #

Nice post by the way > http://machinelearningmastery.com/evaluate-skill-deep-learning-models/

Thanks for sharing it. Been lately thinking about the aspect of accuracy a lot, it seems that at the
moment it’s a “hot mess” in terms of the way common Start
tools do Machine Learning
it out of the box. I think a lot of non PhD /
×
non expert crowd (most people) will at least initially be easily confused and make the kinds of mistakes
You can master applied Machine Learning
you point out in your post.
without math or fancy degrees.
Find
Thanks for all the amazing contributions you are making inout
thishow in this free and practical course.
field!

Email Address

REPLY 
Jason Brownlee August 14, 2017 at 6:26 am #
START MY EMAIL COURSE
I’m glad it helped.

REPLY 
Haneesh December 7, 2019 at 10:36 pm #

Hi Jason,

i’m actually trying to find “spam filter for quora questions” where i have a dataset with label-0’s
and 1’s and questions columns. please let me know the approach and path to build a model for
this.

Thanks

Start Machine Learning

REPLY 
Jason Brownlee December 8, 2019 at 6:10 am #

Sounds like a great project.

The tutorials here on text classification will help:

https://machinelearningmastery.com/start-here/#nlp

REPLY 
RATNA NITIN PATIL August 14, 2017 at 8:16 pm #

Hello Jason, Thanks for a wonderful tutorial.

Can I use Genetic Algorithm for feature selection??

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 88/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

If yes, Could you please provide the link for it???

Thanks in advance.

REPLY 
Jason Brownlee August 15, 2017 at 6:34 am #

Sure. Sorry, I don’t have any examples.

Generally, computers are so fast it might be easier to test all combinations in an exhaustive search.

REPLY 
sunny1304 August 15, 2017 at 3:44 pm #
Start Machine Learning ×
Hi Json,
Thank you for your awesome tutorial. You can master applied Machine Learning
I have a question for you. without math or fancy degrees.
Find out how in this free and practical course.
Is there any guideline on how to decide on neuron number for our network.
for example you used 12 for thr 1st layer and 8 for the second layer.
how do you decide on that ? Email Address

Thanks
START MY EMAIL COURSE

REPLY 
Jason Brownlee August 15, 2017 at 4:58 pm #

No, there is no way to analytically determine the configuration of the network.

I use trial and error. You can grid search, random search, or copy configurations from tutorials or
papers.

REPLY 
yihadad August 16, 2017 at 6:53 pm #

Hi Json,
Thanks for a wonderful tutorial.
Start Machine Learning
Run a model generated by a CNN it takes how much ram, cpu ?

Thanks

REPLY 
Jason Brownlee August 17, 2017 at 6:39 am #

It depends on the data you are using to fit the model and the size of the model.

Very large models could be 500MB of RAM or more.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 89/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Ankur September 1, 2017 at 3:15 am #

Hi ,
Please let me know , how can i visualise the complete neural network in Keras……………….

I am looking for the complete architecture – like number of neurons in the Input Layer, hidden layer ,
output layer with weights.

Please have a look at the link present below, here someone has created a beutiful
visualisation/architecture using neuralnet package in R.
Please let me know, can we create such type of model in KERAS

https://www.r-bloggers.com/fitting-a-neural-network-in-r-neuralnet-package/

Start Machine Learning ×

You# can master applied Machine Learning REPLY 
Jason Brownlee September 1, 2017 at 6:50 am
without math or fancy degrees.
Use the Keras visualization API: Find out how in this free and practical course.
https://keras.io/visualization/
Email Address

START MY EMAIL COURSE REPLY 

ASAD October 17, 2017 at 3:23 am #

Hello ANKUR,,,, how are you?

you have try visualization in keras which is suggested by Jason Brownlee?

if you have tried then please send me code i am also trying but didnot work..

please guide me

REPLY 
Adam September 3, 2017 at 1:45 am #

Thank you Dr. Brownlee for the great tutorial,

I have a question about your code:

is the argument metrics=[‘accuracy’] necessary in the code and does it change the results of the neural
Start
network or is it just for showing me the accuracy during Machine Learning
compiling?

thank you!!

REPLY 
Jason Brownlee September 3, 2017 at 5:48 am #

No, it just prints out the accuracy of the model at the end of each epoch. Learn more about
Keras metrics here:
https://machinelearningmastery.com/custom-metrics-deep-learning-keras-python/

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 90/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

PottOfGold September 5, 2017 at 12:14 am # REPLY 

Hi Jason,

your work here is really great. It helped me a lot.

I recently stumbled upon one thing I cannot understand:

For the pimas dataset you state:

<>
When I look at the table of the pimas dataset, the examples are in rows and the features in columns, so
your input dimension is the number of columns. As far as I can see, you don’t change the table.

For neural networks, isn’t the input normally: examples = columns, features=rows?
Is this different for Keras? Or can I use both shapes? An if yes, what’s the difference in the construction
of the net?
Start Machine Learning ×
Thank you!!
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Jason Brownlee September 7, 2017 at 12:36 pm #

Email
No, features are columns, rows are instances or Address
examples.

START MY EMAIL COURSE

REPLY 
PottOfGold September 7, 2017 at 3:35 pm #

Thanks!
I had a lot of discussions because of that.
In Andrew Ng new Coursera course it’s explained as examples = columns, features=rows, but
he doesn’t use Keras of course, but programms the neural networks from scratch.

REPLY 
Jason Brownlee September 9, 2017 at 11:38 am #

I doubt that, I think you may have mixed it up. Columns are never examples.

Start Machine Learning

PottOfGold October 6, 2017 at 6:26 pm #

Thats what I thought, but I looked it up in the notation for the new coursera
course (deeplearning.ai) and there it says: m is the numer of examples in the dataset
and n is the input size, where X superscript n x m is the input matrix …
But either way, you helped me! Thank you.

REPLY 
Lin Li September 16, 2017 at 1:50 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 91/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Hi Jason, thank you so much for your tutorial, it helps me a lot. I need your help for the question below:
I copy the code and run it. Although I got the classification results, there were some warning messages
in the process. As follows:

Warning (from warnings module):

File “C:\Users\llfor\AppData\Local\Programs\Python\Python35\lib\site-packages\keras\callbacks.py”, line
120
% delta_t_median)
UserWarning: Method on_batch_end() is slow compared to the batch update (0.386946). Check your
callbacks.

I don’t know why, and cannot find any answer to this question. I’m looking forward to your reply. Thanks
again!

Start Machine Learning ×

REPLY 
Jason Brownlee September 16, 2017 at 8:43You
am #can master applied Machine Learning

without math or fancy degrees.

Sorry, I have not seen this message before.
FindIt out
looks
howlike a warning,
in this free andyou mightcourse.
practical be able to
ignore it.

Email Address

REPLY 
Lin Li September 16, 2017 at 12:24 pm # START MY EMAIL COURSE

Thanks for your reply. I’m a start-learner on deep learning.I’d like to put it aside
temporarily.

REPLY 
Sagar September 22, 2017 at 2:51 pm #

Hi Jason,
Great article, thumbs up for that. I am getting this error when I try to run the file on the command prompt.
Any suggestions. Thanks for you response.

#######################################################################
C:\Work\ML>python keras_first_network.py
Using TensorFlow backend.
2017-09-22 10:11:11.189829: W C:\tf_jenkins\home\workspace\rel-win\M\windows\PY\
Start Machine Learning
36\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn
‘t compiled to use AVX instructions, but these are available on your machine and
could speed up CPU computations.
2017-09-22 10:11:11.190829: W C:\tf_jenkins\home\workspace\rel-win\M\windows\PY\
36\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn
‘t compiled to use AVX2 instructions, but these are available on your machine an
d could speed up CPU computations.
32/768 [>………………………..] – ETA: 0s
acc: 78.52%
#######################################################################

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 92/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee September 23, 2017 at 5:35 am #

Looks like warning messages that you can ignore.

REPLY 
Sagar September 24, 2017 at 3:52 am #

Thanks I got to know what the problem was. According to section 6 I had set verbose
argument to 0 while calling “model.fit()”. Now all the epochs are getting printed.

Start
Jason Brownlee September 24, 2017 at 5:17 Machine
am # Learning ×
REPLY 

Glad to hear it. You can master applied Machine Learning

without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Valentin September 26, 2017 at 6:35 pm #
Email Address

Hi Jason,
START MY EMAIL COURSE
Thanks for the amazing article . Clear and straightforward.
I had some problems installing Keras but was advised to prefix
with tf.contrib.keras
so I have code like

model=tf.contrib.keras.models.Sequential()
Dense=tf.contrib.keras.layers.Dense

Now I try to train Keras on some small datafile to see how things work out:
1,1,0,0,8
1,2,1,0,4
1,0,0,1,5
1,0,1,0,7
0,1,0,0,8
1,4,1,0,4
1,0,2,1,1
Start Machine Learning
1,0,1,0,7

The first 4 columns are inputs and the 5-th column is output.
I use the same code for training (adjust number of inputs) as in your article,
but the network only gets to 12.5% accuracy.
Any advise?

Thanks,
Valentin

REPLY 
Jason Brownlee September 27, 2017 at 5:40 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 93/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Thanks Valentin.

I have a good list of suggestions for improving model performance here:

http://machinelearningmastery.com/improve-deep-learning-performance/

REPLY 
Priya October 3, 2017 at 2:28 pm #

Hi Jason,

I tried replacing the pima data with random data as follows:

X_train = np.random.rand(18,61250)
X_test = np.random.rand(18,61250)
Y_train = np.array([0.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 0.0,Start
1.0, Machine Learning ×
0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0,])
Y_test = np.array([1.0, 0.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0, You
1.0, can master applied Machine Learning
1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0,]) without math or fancy degrees.
Find out how in this free and practical course.
_, input_size = X_train.shape #put this in input_dim in the first dense layer

I took the round() off of the predictions so I could see the full value
Email and then inserted my random test
Address
data in model.fit():

predictions = model.predict(X_test) START MY EMAIL COURSE

preds = [x[0] for x in predictions]
print(preds)

model.fit(X_train, Y_train, epochs=100, batch_size=10, verbose=2, validation_data=(X_test,Y_test))

I found something slightly odd; I expected the predicted values to be around 0.50, plus or minus some,
but instead, I got this:

[0.49525392, 0.49652839, 0.49729034, 0.49670222, 0.49342978, 0.49490061, 0.49570397, 0.4962129,

0.49774086, 0.49475089, 0.4958384, 0.49506786, 0.49696651, 0.49869373, 0.49537542, 0.49613148,
0.49636957, 0.49723724]

which is near 0.50 but always less than 0.50. I ran this a few times with different random seeds, so it’s
not coincidental. Would you have any explanation for why it does this?

Thanks,
Priya
Start Machine Learning

REPLY 
Jason Brownlee October 3, 2017 at 3:46 pm #

Perhaps calculate the mean of your training data and compare it to the predicted value. It
might be simple sampling error.

REPLY 
Priya October 4, 2017 at 1:02 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 94/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

I found out I was doing predictions before fitting the model. (I suppose that would mean the network
hadn’t adjusted to the data’s distribution yet.)

REPLY 
Saurabh October 7, 2017 at 5:59 am #

Hello Jason,

I tried to train this model on my laptop, it is working fine. But I tried to train this model on google-cloud
with the same instructions as in your example-5. But it is failing.
Can you just let me know, which changes are to required for the model, so that I can train this on cloud.

Start Machine Learning ×

REPLY 
Jason Brownlee October 7, 2017 at 7:37 am #
You can master applied Machine Learning
Sorry, I don’t know about google cloud. without math or fancy degrees.
Find out how in this free and practical course.
I have instructions here for running on AWS:
https://machinelearningmastery.com/develop-evaluate-large-deep-learning-models-keras-amazon-
web-services/ Email Address

START MY EMAIL COURSE

REPLY 
tobegit3hub October 12, 2017 at 6:40 pm #

Great post. Thanks for sharing.

REPLY 
Jason Brownlee October 13, 2017 at 5:45 am #

You’re welcome.

REPLY 
Manoj October 12, 2017 at 11:43 pm #

Hi Jason, Start Machine Learning

Is there a way to store the model, once it is created so that I can use it for different input data sets as
and when needed.

REPLY 
Jason Brownlee October 13, 2017 at 5:48 am #

Yes, you can save it to file. See this tutorial:

https://machinelearningmastery.com/save-load-machine-learning-models-python-scikit-learn/

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 95/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Cam October 23, 2017 at 6:11 pm #

I get a syntax error for the

model.fit() line in this example. Is it due to library conflicts with theano and tensorflow if i have both
installed?

REPLY 
Jason Brownlee October 24, 2017 at 5:28 am #

Perhaps ensure your environment is up to date and that you copied the code exactly.

This tutorial can help with setting up your environment:

Start Machine Learning
http://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-
×
anaconda/
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Cam October 24, 2017 at 2:11 pm #
Email Address
Thanks, fixed!

START MY EMAIL COURSE

REPLY 
Jason Brownlee October 24, 2017 at 4:01 pm #

Glad to hear it.

REPLY 
Diego Quintana October 25, 2017 at 7:37 am #

Hi Jason, thanks for the example.

How would you predict a single element from X? X[0] raises a ValueError

ValueError: Error when checking : expected dense_1_input to have shape (None, 8) but got array with
shape (8, 1)

Thanks! Start Machine Learning

REPLY 
Jason Brownlee October 25, 2017 at 3:56 pm #

You can reshape it to have 1 row and 8 columns:

1 X = X.reshape((1,8))

This post will give you further advice:

https://machinelearningmastery.com/index-slice-reshape-numpy-arrays-machine-learning-python/

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 96/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
harald April 10, 2019 at 8:26 pm #

Should it be: X[0].reshape((1,8)) ?

REPLY 
Jason Brownlee April 11, 2019 at 6:35 am #

Yep!

REPLY 
Shahbaz Wasti October 28, 2017 at 1:30 pm #
Start Machine Learning ×
Dear Sir ,
I have installed and configured the environment according to your
You can directions
master but whileLearning
applied Machine running the
program i have following error without math or fancy degrees.
Find out how in this free and practical course.
“from keras.utils import np_utils”

Email Address

REPLY 
Jason Brownlee October 29, 2017 at 5:50 am #START MY EMAIL COURSE

What is the error exactly?

REPLY 
Zhengping October 30, 2017 at 12:12 am #

Hi Jason, thanks for the great tutorials. I just learnt and repeated the program in your “Your First
Machine Learning Project in Python Step-By-Step” without problem. Now trying this one, getting stuck at
the line “model = Sequential()” when the Interactive window throws: NameError: name ‘Sequential’ is not
defined. tried to google, can’t find a solution. I did import Sequential from keras.models as in ur example
code. copy pasted as it is. Thanks in advance for your help.

Start Machine Learning REPLY 

Zhengping October 30, 2017 at 12:14 am #

I’m running ur examples in Anaconda 4.4.0 environment in visual studio community version.
relevant packages have been installed as in ur earlier tutorials instructed.

REPLY 
Zhengping October 30, 2017 at 12:18 am #

>> # create model

… model = Sequential()
…
Traceback (most recent call last):

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 97/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

File “”, line 2, in

NameError: name ‘Sequential’ is not defined
>>> model.add(Dense(12, input_dim=8, init=’uniform’, activation=’relu’))
Traceback (most recent call last):
File “”, line 1, in
AttributeError: ‘SVC’ object has no attribute ‘add’

REPLY 
Jason Brownlee October 30, 2017 at 5:39 am #

This does not look good. Perhaps post the error to stack exchange or other keras
support. I have a list of keras support sites here:
https://machinelearningmastery.com/get-help-with-keras/
Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees.
REPLY 
Jason Brownlee October 30, 2017 at 5:38 amFind
# out how in this free and practical course.

Looks like you need to install Keras. I have a tutorial here on how to do that:
Email Address
https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-
anaconda/
START MY EMAIL COURSE

REPLY 
Akhil October 30, 2017 at 5:04 pm #

Ho Jason,

Thanks a lot for this wonderful tutorial.

I have a question:

I want to use your code to predict the classification (1 or 0) of unknown samples. Should I create one
common csv file having the train (known) as well as the test (unknown) data. Whereas the ‘classification’
column for the known data will have a known value, 1 or 0, for the unknown data, should I leave the
column empty (and let the code decide the outcome)?

Thanks a lot

Start Machine Learning

REPLY 
Jason Brownlee October 31, 2017 at 5:29 am #

Great question.

No, you only need the inputs and the model can predict the outputs, call model.predict(X).

Also, this post will give a general idea on how to fit a final model:
https://machinelearningmastery.com/train-final-machine-learning-model/

REPLY 
Guilherme November 3, 2017 at 1:26 am #
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 98/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Hi Jason,

This is really cool! I am blown away! Thanks so much for making it so simple for a beginner to
have some hands on. I have a couple questions:

1) where are the weights, can I save and/or retrieve them?

2) if I want to train images with dogs and cats and later ask the neural network whether a new image has
a cat or a dog, how do I get my input image to pass as an array and my output result to be “cat” or
“dog”?

Thanks again and great job!

Jason Brownlee November 3, 2017 at 5:20 am #

Start Machine Learning ×
REPLY 

The weights are in the model, you can save them:

You can master applied Machine Learning
https://machinelearningmastery.com/save-load-keras-deep-learning-models/
without math or fancy degrees.
Find out how
Yes, you would save your model, then call model.predict() in this
on the newfree and practical course.
data.

Email Address

REPLY 
Michael November 5, 2017 at 8:33 am #
START MY EMAIL COURSE
Hi Jason,

Are you familiar with a python tool/package that can build neural network as in the tutorial, but suitable
for data stream mining?

Thanks,
Michael

REPLY 
Jason Brownlee November 6, 2017 at 4:46 am #

Not really, sorry.

Start Machine Learning

REPLY 
bea November 8, 2017 at 1:58 am #

Hi, there. Could you please clarify why exactly you’ve built your network with 12 neurons in the
first layer?

“The first layer has 12 neurons and expects 8 input variables. The second hidden layer has 8 neurons
and finally, the output layer has 1 neuron to predict the class (onset of diabetes or not)…”

Should’nt it have 8 neurons at the start?

Thanks

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 99/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee November 8, 2017 at 9:28 am #

The input layer has 8, the first hidden layer has 12. I chose 12 through a little trial and error.

REPLY 
Guilherme November 9, 2017 at 12:54 am #

Hi Jason,

Do you have or else could you recommend a beginner’s level image segmentation approach that uses
deep learning? For example, I want to train some neural net to automatically “find” a particular feature
out of an image.

Thanks! Start Machine Learning ×

You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course. REPLY 
Jason Brownlee November 9, 2017 at 10:00 am #

Sorry, I don’t have image segmentation examples, perhaps in the future.

Email Address

START MY EMAIL COURSE

REPLY 
Andy November 12, 2017 at 6:56 pm #

Hi Jason,

I just started my DL training a few weeks ago. According to what I learned in course, in order to train the
parameters for the NN, we need to run the Forward and Backward propagation; however, looking at your
Keras example, i don’t find any of these propagation processes. Does it mean that Keras has its own
mechanism to find the parameters instead of using Forward and Backward propagation?

Thanks!

REPLY 
Jason Brownlee November 13, 2017 at 10:13 am #

It is performing those operations under the covers

Start for you.
Machine Learning

REPLY 
Badr November 13, 2017 at 11:42 am #

Hi Jason,

Can you explain why I got the following output:

ValueError Traceback (most recent call last)

in ()
—-> 1 model.compile(loss=’binary_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])
2 model.fit(X, Y, epochs=150, batch_size=10)

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 100/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

3 scores = model.evaluate(X, Y)
4 print(“\n%s: %.2f%%” % (model.metrics_names[1], scores[1]*100))

/Users/badrshomrani/anaconda/lib/python3.5/site-packages/keras/models.py in compile(self, optimizer,

loss, metrics, sample_weight_mode, **kwargs)
545 metrics=metrics,
546 sample_weight_mode=sample_weight_mode,
–> 547 **kwargs)
548 self.optimizer = self.model.optimizer
549 self.loss = self.model.loss

/Users/badrshomrani/anaconda/lib/python3.5/site-packages/keras/engine/training.py in compile(self,
optimizer, loss, metrics, loss_weights, sample_weight_mode, **kwargs)
620 loss_weight = loss_weights_list[i]
621 output_loss = weighted_loss(y_true, y_pred, Start Machine Learning ×
–> 622 sample_weight, mask)
623 if len(self.outputs) > 1: You can master applied Machine Learning
624 self.metrics_tensors.append(output_loss) without math or fancy degrees.
Find out how in this free and practical course.
/Users/badrshomrani/anaconda/lib/python3.5/site-packages/keras/engine/training.py in weighted(y_true,
y_pred, weights, mask)
Email Address
322 def weighted(y_true, y_pred, weights, mask=None):
323 # score_array has ndim >= 2
–> 324 score_array = fn(y_true, y_pred) START MY EMAIL COURSE
325 if mask is not None:
326 # Cast the mask to floatX to avoid float64 upcasting in theano

/Users/badrshomrani/anaconda/lib/python3.5/site-packages/keras/objectives.py in
binary_crossentropy(y_true, y_pred)
46
47 def binary_crossentropy(y_true, y_pred):
—> 48 return K.mean(K.binary_crossentropy(y_pred, y_true), axis=-1)
49
50

/Users/badrshomrani/anaconda/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py in
binary_crossentropy(output, target, from_logits)
1418 output = tf.clip_by_value(output, epsilon, 1 – epsilon)
1419 output = tf.log(output / (1 – output))
-> 1420 return tf.nn.sigmoid_cross_entropy_with_logits(output, target)Learning
Start Machine
1421
1422

/Users/badrshomrani/anaconda/lib/python3.5/site-packages/tensorflow/python/ops/nn_impl.py in
sigmoid_cross_entropy_with_logits(_sentinel, labels, logits, name)
147 # pylint: disable=protected-access
148 nn_ops._ensure_xent_args(“sigmoid_cross_entropy_with_logits”, _sentinel,
–> 149 labels, logits)
150 # pylint: enable=protected-access
151

/Users/badrshomrani/anaconda/lib/python3.5/site-packages/tensorflow/python/ops/nn_ops.py in
_ensure_xent_args(name, sentinel, labels, logits)

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 101/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

1696 if sentinel is not None:

1697 raise ValueError(“Only call %s with ”
-> 1698 “named arguments (labels=…, logits=…, …)” % name)
1699 if labels is None or logits is None:
1700 raise ValueError(“Both labels and logits must be provided.”)

ValueError: Only call sigmoid_cross_entropy_with_logits with named arguments (labels=…,

logits=…, …)

REPLY 
Jason Brownlee November 14, 2017 at 10:05 am #

Perhaps double check you have the latest versions of the keras and tensorflow libraries
installed?! Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees.
Badr November 14, 2017 at 10:50 am # Find out how in this free and practical course. REPLY 

keras was outdated

Email Address

START MY EMAIL COURSE

REPLY 
Jason Brownlee November 15, 2017 at 9:44 am #

Glad to hear you fixed it.

REPLY 
Mikael November 22, 2017 at 8:20 am #

Hi Jason, thanks for your short tutorial, helps a lot to actually get your hands dirty with a simple
example.
I have tried 5 different parameters and got some interesting results to see what would happen.
Unfortunately, I didnt record running time.

Test 1 Test 2 Test 3 Test 4 Test 5 Test 6 Test 7

number of layers 3 3 3 3 3 3 4
Train set 768 768 768 768 768 768 768 Start Machine Learning
Iterations 150 100 1000 1000 1000 150 150
Rate of update 10 10 10 5 1 1 5
Errors 173 182 175 139 161 169 177
Values 768 768 768 768 768 768 768
% Error 23,0000% 23,6979% 22,7865% 18,0990% 20,9635% 22,0052% 23,0469%

I can’t seem to see a trend here.. That could put me on the right track to adjust my hyperparameters.

Do you have any advice on that?

REPLY 
Jason Brownlee November 22, 2017 at 11:17 am #
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 102/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Something is wrong. Here is a good list of things to try:

http://machinelearningmastery.com/improve-deep-learning-performance/

REPLY 
Nikolaos November 28, 2017 at 10:58 am #

Hi, I try to implement the above example with fer2013.csv but I receive an error, it is possible to
help me to implement this correctly?

1 keras.models import Sequential

2 from keras.layers import Dense
3 import numpy
4 import numpy as np
5
6
7
# fix Random seed for reproducibility
numpy.random.seed(7) Start Machine Learning ×
8 Y = []
9 X = [] You can master applied Machine Learning
10 #load dataset
without math or fancy degrees.
11 for line in open("fer2013.csv"):
12 row = line.split(',') Find out how in this free and practical course.
13 Y.append(int(row[0]))
14 X.append([int(p) for p in row[1].split()])
15 X, Y = np.array(X) / 255.0, np.array(Y) Email Address
16 print(Y.shape)
17 print(X.shape)
18
START MY EMAIL COURSE
19
20 #create model
21 model = Sequential()
22 model.add(Dense(12, input_dim=(35887, 2304), activation='tanh'))
23 model.add(Dense(8, activation='tanh'))
24 model.add(Dense(1, activation='sigmoid'))
25
26 #Compile Model
27 model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
28
29 #Fit Model
30 model.fit(X, Y, epochs=150, batch_size=1)
31
32 # evaluate the model
33 scores = model.evaluate(X, Y)
34 print("\n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))
35
36 # calculate predictions
37 predictions = model.predict(X)
38 # round predictions
39 rounded = [round(x[0]) for x in predictions]
Start Machine Learning
40 print(rounded)

REPLY 
Jason Brownlee November 29, 2017 at 8:10 am #

Sorry, I cannot debug your code.

What is the problem exactly?

REPLY 
Tanya December 2, 2017 at 12:06 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 103/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Hello,
i have a a bit general question.
I have to do a forecasting for restaurant sales (meaning that I have to predict 4 meals based on
a historical daily sales data), weather condition (such as temperature, rain, etc), official holiday and in-
off-season. I have to perform that forecasting using neuronal networks.
I am unfortunately not a very skilled in python. On my computer I have Python 2.7 and I have install
anaconda. I am trying to learn exercising with your codes, Mr. Brownlee. But somehow I can not run the
code at all (in Spyder). Can you tell me what kind of version of python and anaconda I have to install on
my computer and in which environment (jupiterlab,notebook,qtconsole, spyder, etc) I can run the code,
so to work and not to give error from the very beginning?
I will be very thankful for your response
KG
Tanya
Start Machine Learning ×
You can master applied Machine Learning
REPLY 
Jason Brownlee December 2, 2017 at 9:02 am
without
# math or fancy degrees.
Find out how in this free and practical course.
Perhaps this tutorial will help you setup and confirm your environment:
http://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-
Email Address
anaconda/

I would also recommend running code from the command like as IDEs and notebooks can introduce
START MY EMAIL COURSE
and hide errors.

REPLY 
Eliah December 3, 2017 at 10:53 am #

Hi Dr. Brownlee.

I looked over the tutorial and I had a question regarding reading the data from a binary file? For instance
I working on solving the sliding tiled n-puzzle using neural networks, but I seem to have trouble to getting
my data which is in a binary file and it generates the number of move required for the n-puzzle to be
solve in. Am not sure if you have dealt with this before, but any help would be appreciated.

REPLY 
Jason Brownlee December 4, 2017 at 7:43 am #
Start Machine Learning

Sorry, I don’t know about your binary file.

Perhaps after you load your data, you can convert it to a numpy array so that you can provide it to a
neural net?

REPLY 
Eliah December 4, 2017 at 9:28 am #

Thanks for the tip, I’ll try it.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 104/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Wafaa December 7, 2017 at 4:59 pm #

Thank you very very much for all your great tutorials.

If I wanted to add batch layer after the input layer, how should I do it?

Cuz I applied this tutorial on a different dataset and features and I think I need normalization or
standardization and I want to do it the easiest way.

Thank you,

REPLY 
Jason Brownlee December 8, 2017 at 5:35 am #

Start
I recommend preparing the data prior to fitting theMachine
model. Learning ×
You can master applied Machine Learning
without math or fancy degrees.
zaheer December 9, 2017 at 3:03 am # Find out how in this free and practical course. REPLY 

thanks for sharing such nice tutorials, it helpedEmail

me alot. i want to print the confusion matrix from
Address
the above example. and one more question.
if i have
20-input variable START MY EMAIL COURSE

1- class label (binary)

and 400 instances
how i would know , setting up the dense layer parameter in the first layer and hidden layer and output
layer. like above example you have placed. 12,8,1

REPLY 
Jason Brownlee December 9, 2017 at 5:44 am #

I recommend trial and error to configure the number of neurons in the hidden layer to see
what works best for your specific problem.

REPLY 
zaheer December 9, 2017 at 3:29 am # Start Machine Learning

C:\Users\zaheer\AppData\Local\Programs\Python\Python36\python.exe
C:/Users/zaheer/PycharmProjects/PythonBegin/Bin-CLNCL-Copy.py
Using TensorFlow backend.
Traceback (most recent call last):
File “C:/Users/zaheer/PycharmProjects/PythonBegin/Bin-CLNCL-Copy.py”, line 28, in
model.fit(x_train , y_train , epochs=100, batch_size=100)
File “C:\Users\zaheer\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\models.py”,
line 960, in fit
validation_steps=validation_steps)
File “C:\Users\zaheer\AppData\Local\Programs\Python\Python36\lib\site-
packages\keras\engine\training.py”, line 1574, in fit

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 105/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

batch_size=batch_size)
File “C:\Users\zaheer\AppData\Local\Programs\Python\Python36\lib\site-
packages\keras\engine\training.py”, line 1407, in _standardize_user_data
exception_prefix=’input’)
File “C:\Users\zaheer\AppData\Local\Programs\Python\Python36\lib\site-
packages\keras\engine\training.py”, line 153, in _standardize_input_data
str(array.shape))
ValueError: Error when checking input: expected dense_1_input to have shape (None, 20) but got array
with shape (362, 1)

REPLY 
Jason Brownlee December 9, 2017 at 5:45 am #
Start Machine Learning ×
Ensure the input shape matches your data.
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Anam Zahra December 10, 2017 at 7:40 pm #

Dear Jason! Great job a very simple guide. Email Address

I am trying to run the exact code but there is an eror
str(array.shape)) START MY EMAIL COURSE

ValueError: Error when checking target: expected dense_3 to have shape (None, 1) but got array with
shape (768, 8)

How can I resolve.

I have windows 10 and spyder.

REPLY 
Jason Brownlee December 11, 2017 at 5:24 am #

Sorry to hear that, perhaps confirm that you have the latest version of Numpy and Keras
installed?

Start Machine Learning

REPLY 
nazek hassouneh December 11, 2017 at 7:33 am #

after run this code , i will calculate the accuracy , how i did , i
i want to split the data set into test data , training data
and evaluate the model and calculate the accuracy
thank dr.

REPLY 
Suchith December 21, 2017 at 2:35 pm #

In the model how many hidden layers are there ?

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 106/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee December 21, 2017 at 3:35 pm #

There are 2 hidden layers, 1 input layer and 1 output layer.

REPLY 
Amare Mahtesenu December 22, 2017 at 9:55 am #

hi there. this blog is very awesome like the Adrian’s pyimagesearch blog. I have one question
and that is do you have or will you have a tutorial on keras frame work with SSD or Yolo architechtures?

Start Machine Learning ×

REPLY 
Jason Brownlee December 22, 2017 at 4:16 pm #
You can master applied Machine Learning
without
Thanks for the suggestion, I hope to cover math
them in the or fancy degrees.
future.
Find out how in this free and practical course.

Email Address
REPLY 
Kyujin Chae January 8, 2018 at 2:22 pm #

Thanks for your awesome article. START MY EMAIL COURSE

I am really enjoying
‘Machine Learning Mastery’!!

REPLY 
Jason Brownlee January 8, 2018 at 3:54 pm #

Thanks!

REPLY 
Luis Galdo January 9, 2018 at 8:41 am #

Hello Jason!

This is an awesome article! Start Machine Learning

I am writing a report for a subject in university and I have used your code during my implementation,
would it be possible to cite this post in bibtex?

Thank you!

REPLY 
Jason Brownlee January 9, 2018 at 3:17 pm #

Sure, you can cite the webpage directly.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 107/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Nikhil Gupta January 25, 2018 at 8:05 pm # REPLY 

My question is regarding predict. I used to get decimals in the prediction array. Suddenly, I
started seeing only Integers (0 or 1) in the run. Any idea what could be causing the change?

predictions = model.predict(X2)

predictions
Out[3]:
array([[ 0.],
[ 0.],
[ 0.],
…,
[ 0.],
[ 0.],
Start Machine Learning ×
[ 0.]], dtype=float32)
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Jason Brownlee January 26, 2018 at 5:39 am #

Email
Perhaps check the activation function on the Address
output layer?

START MY EMAIL COURSE

REPLY 
Nikhil Gupta January 28, 2018 at 3:30 am #

# create model. Fully connected layers are defined using the Dense class
model = Sequential()
model.add(Dense(12, input_dim=len(x_columns), activation=’relu’)) #12 neurons, 8 inputs
model.add(Dense(8, activation=’relu’)) #Hidden layer with 8 neurons
model.add(Dense(1, activation=’sigmoid’)) #1 output layer. Sigmoid give 0/1

REPLY 
joe January 27, 2018 at 1:25 am #

================== RESTART: /Users/apple/Documents/deep1.py ==================

Using TensorFlow backend.
Start Machine Learning
Traceback (most recent call last):
File “/Users/apple/Documents/deep1.py”, line 20, in
model.compile(loss=’binary_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/keras/models.py”,
line 826, in compile
**kwargs)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
packages/keras/engine/training.py”, line 827, in compile
sample_weight, mask)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
packages/keras/engine/training.py”, line 426, in weighted
score_array = fn(y_true, y_pred)

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 108/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/keras/losses.py”,
line 77, in binary_crossentropy
return K.mean(K.binary_crossentropy(y_true, y_pred), axis=-1)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
packages/keras/backend/tensorflow_backend.py”, line 3069, in binary_crossentropy
logits=output)
TypeError: sigmoid_cross_entropy_with_logits() got an unexpected keyword argument ‘labels’
>>>

REPLY 
Jason Brownlee January 27, 2018 at 5:58 am #

I have not seem this error, sorry. Perhaps try posting to stack overflow?
Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees.
REPLY 
Atefeh January 27, 2018 at 4:04 pm # Find out how in this free and practical course.

Hello Mr.Janson
Emailyour
After installing Anaconda and deep learning libraries, I read Address
Free mini-course and I tried to write the
code about the handwritten digit recognition.
I wrote the codes in jupyter notebook, am I right? START MY EMAIL COURSE
if not where should I write the codes ?
and if I want to use another dataset (my own data set) how can I use in the code?
and how can I see the result, for example the accuracy percentage?
I am really sorry for my simple questions! I have written a lot of code in “Matlab” but I am really a
beginner in Python and Anaconda, my teacher force me to use Python and keras for my project.

thank you very much for your help

REPLY 
Jason Brownlee January 28, 2018 at 8:22 am #

A notebook is fine.

You can write code in a Python script and then run the script directly.

Start Machine Learning

REPLY 
Atefeh January 28, 2018 at 12:01 am #

Hello Mr.Janson again

I wrote the code below from your Free mini course for hand written digit recognition, but after running I
faced the syntaxerror:

from keras.datasets import mnist

…
(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 1, 28, 28)

X_test = X_test.reshape(X_test.shape[0], 1, 28, 28)
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 109/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

from keras.utils import np_utils

…
y_train = np_utils.to_categorical(y_train)
y_test = np_utils.to_categorical(y_test)

model = Sequential()
model.add(Conv2D(32, (3, 3), padding=’valid’, input_shape=(1, 28, 28),
activation=’relu’))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(128, activation=’relu’))
model.add(Dense(num_classes, activation=’softmax’))
model.compile(loss=’categorical_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])

File “”, line 2

Start Machine Learning ×
2 model.add(Conv2D(32, (3, 3), padding=’valid’, input_shape=(1, 28, 28),
^ You can master applied Machine Learning
SyntaxError: invalid syntax without math or fancy degrees.
Find out how in this free and practical course.
would you please help me?!

thanks a lot Email Address

START MY EMAIL COURSE

REPLY 
Jason Brownlee January 28, 2018 at 8:25 am #

This:

1 model.add(Conv2D(32, (3, 3), padding=’valid’, input_shape=(1, 28, 28),

2 activation=’relu’))

should be:

1 model.add(Conv2D(32, (3, 3), padding=’valid’, input_shape=(1, 28, 28), activation=’r

REPLY 
Lila January 29, 2018 at 8:04 am #

Thank you for the awsome blog and explanations. I have just a question: How can we get
predicted values by the model. . Many thanks Start Machine Learning

REPLY 
Jason Brownlee January 29, 2018 at 8:21 am #

As follows:

1 X = ...
2 yhat = model.predict(X)

REPLY 
Lila January 30, 2018 at 1:22 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 110/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Thank you for your prompt answer. I am trying to learn how keras models work and I
used. I trained the model like this:

model.compile(loss=’mean_squared_error’, optimizer=’sgd’, metrics=[‘MSE’])

As output I have those lines

Epoch 10000/10000

10/200 [>………………………..] – ETA: 0s – loss: 0.2489 – mean_squared_error: 0.2489

200/200 [==============================] – 0s 56us/step – loss: 0.2652 –
mean_squared_error: 0.2652

and my question what the difference between the two lines (MSE values)

Start Machine Learning ×

REPLY 
Jason Brownlee January 30, 2018 at 9:53 am #
You can master applied Machine Learning
without
They should be the same thing. One maymath or fancy degrees.
be calculated at the end of each batch,
and one at the end of each epoch. Find out how in this free and practical course.

Email Address

REPLY 
Atefeh January 30, 2018 at 4:28 am #
START MY EMAIL COURSE
hello

after running again it show an error:

NameError Traceback (most recent call last)

in ()
—-> 1 model = Sequential()
2 model.add(Conv2D(32, (3, 3), padding=’valid’, input_shape=(1, 28, 28), activation=’relu’))
3 model.add(MaxPooling2D(pool_size=(2, 2)))
4 model.add(Flatten())
5 model.add(Dense(128, activation=’relu’))

NameError: name ‘Sequential’ is not defined

Start Machine Learning REPLY 

Jason Brownlee January 30, 2018 at 9:55 am #

You are missing the imports. Ensure you copy all code from the complete example at the
end.

REPLY 
Atefeh January 31, 2018 at 1:02 am #

from keras.datasets import mnist

…
(X_train, y_train), (X_test, y_test) = mnist.load_data()
X_train = X_train.reshape(X_train.shape[0], 1, 28, 28)
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 111/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

X_test = X_test.reshape(X_test.shape[0], 1, 28, 28)

from keras.utils import np_utils
…
y_train = np_utils.to_categorical(y_train)
y_test = np_utils.to_categorical(y_test)

model = Sequential()
2 model.add(Conv2D(32, (3, 3), padding=’valid’, input_shape=(1, 28, 28), activation=’relu’))
3 model.add(MaxPooling2D(pool_size=(2, 2)))
4 model.add(Flatten())
5 model.add(Dense(128, activation=’relu’))
6 model.add(Dense(num_classes, activation=’softmax’))
7 model.compile(loss=’categorical_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])

Start Machine Learning ×

You can master applied Machine Learning REPLY 
Atefeh February 2, 2018 at 5:01 am #
without math or fancy degrees.

hello Find out how in this free and practical course.

please tell me how can I find out that tensorflow and keras are correctly installed on my system.
maybe the problem is that, because no code runs in myEmail Address
jupyter. and no “import” acts well(for example
import pandas)
thank you
START MY EMAIL COURSE

REPLY 
Jason Brownlee February 2, 2018 at 8:23 am #

See this post:

https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-
anaconda/

REPLY 
Dan February 3, 2018 at 12:29 am #

Hi. I’m totally new to machine learning and I’m trying to wrap my head around it.
I have a problem I can’t quite solve yet. And don’t know where to start actually.
I have a dictionary with a few key:value pairs. The keyStart
is a random 4 digit
Machine number from 0000 to 9999.
Learning
And the value for each key is set as follows: if a digit in a number is either 0, 6 or 9 then its weight is 1, if
a digit is 8 then it’s weight is 2, any other digit has a weight of 0. All the weights are summarised then
and here you have the value for the key. (example: { ‘0000’: 4, ‘1234’: 0, ‘1692’: 2, ‘8800’: 6} – and so
on).

Now I’m trying to build a model that will predict the correct value of a given key. (i.e if I give it 2222 the
answer is 0, if I give it 9011 – it’s 2). What I did first is created a CSV file with 5 columns, first four is a
split (by a single digit) key from my dictionary, and the fifth column is the value for each key. Next I
created a dataset and defined a model (like this tutorial but with input_dim=4). Now when I train the
model the accuracy won’t go higher then ~30%. Also your model is based on binary output, whereas
mine should have an integer from 0 to 8. Where do I go from here?

Thank you for all your effort in advance!

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 112/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee February 3, 2018 at 8:42 am #

This post might help you nail down your problem as a predictive modeling problem:
http://machinelearningmastery.com/how-to-define-your-machine-learning-problem/

REPLY 
Alex February 5, 2018 at 5:22 am #

There is one thing I just dont get.

An example of row data is 6,148,72,35,0,33.6,0.627,50,1

Start Machine Learning ×
I guess the number at the end is if the person has diabetes (1) or does not (0) , but what I dont
You
understand is how I know the ‘prediction’is about that 0 or can masterare
1, tehere applied
a lot Machine
of other Learning
variables in the
data, and I dont see ‘diabetes’ being a label for any of without
that. math or fancy degrees.
Find out how in this free and practical course.
So, how do I know or how do I set wich variable (number) I want to predict?

Email Address

REPLY 
Jason Brownlee February 5, 2018 at 7:49 am #START MY EMAIL COURSE

You interpret the prediction in your application or usage.

The model does not care what the inputs and outputs are, it does the best it can. It does not
intrinsically care about diabetes.

REPLY 
blaisexen February 6, 2018 at 9:14 am #

hi,
@Jason Brownlee, Master of Keras Python.

I’m developing a face recognition testing, I successfully used Rprop, it was good for static images or face
pictures, I also have test svm results.

What do you think in your experienced that Keras is better

Startor powerfulLearning
Machine than Rprop?

because I was also thinking to used Keras(1:1) for final result of Rprop(1:many).

or which do you think is better system?

thanks in advance for the advices.

I also heard one of the leader of commercial face recognizers uses PNN(uses libopenblas), so I really
doubt which one to choose for my final thesis and application.

REPLY 
Jason Brownlee February 6, 2018 at 9:29 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 113/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

What do you mean by rprop? I believe it is just an optimization algorithm, whereas Keras is a deep
learning library.
https://en.wikipedia.org/wiki/Rprop

REPLY 
blaisexen February 17, 2018 at 10:46 am #

Ok, I think I understand you.

I used Accord.Net
Rprop testing was good
MLR testing was good
SVM testing was good
RBM testing was good Start Machine Learning ×
I used classification for face images You can master applied Machine Learning
They are only good for static face pictures 100×100
without math or fancy degrees.
but if I used another picture from them, Find out how in this free and practical course.

these 4 testing I have failed.

Email Address
Do you think if I used Keras in image face recognition will have a good result or good prediction?

because if Keras will have a good result then I’ll have to used cesarsouza keras c#
START MY EMAIL COURSE
https://github.com/cesarsouza/keras-sharp

thanks for the reply.

REPLY 
Jason Brownlee February 18, 2018 at 6:45 am #

Try it and see.

REPLY 
CHIRANJEEVI February 8, 2018 at 8:52 pm #

What is the difference between the accuracy we get when we fit the model and the
accuracy_score() of sklearn.metrics , what they mean exactly ?
Start Machine Learning

REPLY 
Jason Brownlee February 9, 2018 at 9:05 am #

Accuracy is a summary of the number of predictions that were made correctly out of all
predictions that were made.

It is used as an estimate of model skill on new out of sample data.

REPLY 
Shinan February 8, 2018 at 9:09 pm #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 114/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

is weather forecasting can done using RNN?

REPLY 
Jason Brownlee February 9, 2018 at 9:06 am #

No. Weather forecasting is done with ensembles of physics simulations on very large
computers.

REPLY 
CHIRANJEEVI February 9, 2018 at 3:56 pm #

we haven’t predicting anyting during the fit (its just a training , like mapping F(x)=Y)
but still getting acc , what is this acc? Start Machine Learning ×
Epoch 1/150 You can master applied Machine Learning
768/768 [==============================] – 1swithout
1ms/step – loss:
math 0.6771
or fancy – acc: 0.6510
degrees.
Find out how in this free and practical course.
Thank you in advance

Email Address

REPLY 
Jason Brownlee February 10, 2018 at 8:50 am #
START MY EMAIL COURSE

Predictions are made as part of back propagating error.

REPLY 
lcy1031 February 12, 2018 at 1:00 pm #

Hi Jason,

Many thanks to you for a great tutorial. I have couple questions to you as followings.
1). How can I get the score of Prediction?
2). How can I output the result of predict run to a file in which the output is listed by vertical?

I see you everywhere to answer questions and help people. Your time and patience were greatly
appreciated!

Charles
Start Machine Learning

REPLY 
Jason Brownlee February 12, 2018 at 2:50 pm #

You can make predictions with a model as follows:

yhat = model.predict(X)

You can then save the numpy array result to file.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 115/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Callum February 21, 2018 at 10:11 am #

Hi I’ve just finished this tutorial but the only problem is what are we actually finding in the results
as in what do accuracy and loss mean and what we are actually finding out.

I’m really new to the whole neural networks thing and don’t really understand them yet, I’d be very
grateful if you’re able to reply

Many Thanks

Callum

Jason Brownlee February 22, 2018 at 11:12 Start

am # Machine Learning ×
REPLY 

Accuracy is the model skill in terms of theYou

number of correct
can master predictions
applied divided by the total
Machine Learning
number of predictions. without math or fancy degrees.
Find out how in this free and practical course.
Loss the function that the network is optimising, something differentiable and relatable to the metric
of interest for the model, in this case logarithmic loss used for classification.
Email Address

START MY EMAIL COURSE REPLY 

Pedro Wenner February 23, 2018 at 1:27 am #

Hi Jason,

First of all congratulations for your awesome work, I finally got the hang of ML (hopefully, haha).
So, testing some changes in the number of neurons and batch size/epochs, I achieved 99.87% of
accuracy.

The parameters I used were:

# create model
model = Sequential()
model.add(Dense(240, input_dim=8, init=’uniform’, activation=’relu’))
model.add(Dense(160, init=’uniform’, activation=’relu’))
model.add(Dense(1, init=’uniform’, activation=’sigmoid’))
# Compile model
model.compile(loss=’binary_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])
Start Machine Learning
# Fit the model
model.fit(X, Y, epochs=1500, batch_size=100, verbose=2)

And when I run it, I always get 99,87% of accuracy, which I think it’s a good thing, right? Please tell me if
I did something wrong or if this is a false positive.

Thank you in advance and sorry for the bad english

REPLY 
Jason Brownlee February 23, 2018 at 12:00 pm #

that accuracy is great, there will always be some error.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 116/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Shiny March 2, 2018 at 12:56 am #

The above example is very good sir, I want to do price change prediction of electronics in online
shopping project. Can you give any suggestions about my project. You had any example of price
prediction using neural network please send a link sir.

REPLY 
Jason Brownlee March 2, 2018 at 5:33 am #

I would recommend following this process:

https://machinelearningmastery.com/start-here/#process
Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees.
REPLY 
awaludin March 6, 2018 at 12:38 am # Find out how in this free and practical course.

Hi, very helpful example. But I still don’t understand why you load
X = dataset[:,0:8] Email Address
Y = dataset[:,8]
If I do START MY EMAIL COURSE
X = dataset[:,0:7] it won’t work

REPLY 
Jason Brownlee March 6, 2018 at 6:16 am #

You can learn more about indexing and slicing numpy arrays here:
https://machinelearningmastery.com/index-slice-reshape-numpy-arrays-machine-learning-python/

REPLY 
Jeong Kim March 8, 2018 at 1:48 pm #

Thank you for the tutorial.

Perhaps, someone already told you this. The data set is no longer available.
Start Machine Learning

REPLY 
Jason Brownlee March 8, 2018 at 2:55 pm #

Thanks for the note, I’ll fix that up ASAP.

REPLY 
Wesley Campbell March 9, 2018 at 1:24 am #

Thanks very much for the concise example! As an “interested amateur” with more experience
coding for scientific data manipulation than for software development, a simple, high-level explanation

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 117/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

like this one is much appreciated. I find sometimes that documentation pages can be a bit low-level for
my liking, even with coding experience multiple languages. This article was all I needed to get started,
and was much more helpful than other “official tutorials.”

REPLY 
Jason Brownlee March 9, 2018 at 6:24 am #

Thanks, I’m glad to hear that Wesley.

REPLY 
Trung March 10, 2018 at 12:55 am #

Thank you for your tutorial, but the data set isStart Machine
not accessible. Could Learning
you please fix it.
×
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course. REPLY 
Jason Brownlee March 10, 2018 at 6:33 am #

Thanks, I’ll fix it. Email Address

START MY EMAIL COURSE

REPLY 
atefeh March 16, 2018 at 10:11 pm #

hello

I have found a code to converting my image data to mnist format . but I face to an error below.
would you please help me?

import os
from PIL import Image
from array import *
from random import shuffle

# Load from and save to

Names = [[‘./training-images’,’train’], [‘./test-images’,’test’]]

for name in Names:

Start Machine Learning
data_image = array(‘B’)
data_label = array(‘B’)

FileList = []
for dirname in os.listdir(name[0])[1:]: # [1:] Excludes .DS_Store from Mac OS
path = os.path.join(name[0],dirname)
for filename in os.listdir(path):
if filename.endswith(“.png”):
FileList.append(os.path.join(name[0],dirname,filename))

shuffle(FileList) # Usefull for further segmenting the validation set

for filename in FileList:

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 118/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

label = int(filename.split(‘/’)[2])

Im = Image.open(filename)

pixel = Im.load()

width, height = Im.size

for x in range(0,width):
for y in range(0,height):
data_image.append(pixel[y,x])

data_label.append(label) # labels start (one unsigned byte each)

hexval = “{0:#0{1}x}”.format(len(FileList),6) # number of files in HEX

# header for label array

Start Machine Learning ×
header = array(‘B’)
header.extend([0,0,8,1,0,0]) You can master applied Machine Learning
header.append(int(‘0x’+hexval[2:][:2],16)) without math or fancy degrees.
header.append(int(‘0x’+hexval[2:][2:],16)) Find out how in this free and practical course.

data_label = header + data_label

Email Address
# additional header for images array

if max([width,height]) <= 256:

START MY EMAIL COURSE
header.extend([0,0,0,width,0,0,0,height])
else:
raise ValueError('Image exceeds maximum size: 256×256 pixels');

header[3] = 3 # Changing MSB for image data (0x00000803)

data_image = header + data_image

output_file = open(name[1]+'-images-idx3-ubyte', 'wb')

data_image.tofile(output_file)
output_file.close()

output_file = open(name[1]+'-labels-idx1-ubyte', 'wb')

data_label.tofile(output_file)
output_file.close()

# gzip resulting files

Start Machine Learning
for name in Names:
os.system('gzip '+name[1]+'-images-idx3-ubyte')
os.system('gzip '+name[1]+'-labels-idx1-ubyte')

FileNotFoundError Traceback (most recent call last)

in ()
13
14 FileList = []
—> 15 for dirname in os.listdir(name[0])[1:]: # [1:] Excludes .DS_Store from Mac OS
16 path = os.path.join(name[0],dirname)
17 for filename in os.listdir(path):

FileNotFoundError: [WinError 3] The system cannot find the path specified: ‘./training-images’

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 119/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee March 17, 2018 at 8:37 am #

Looks like the code cannot find your images. Perhaps change the path in the code?

REPLY 
Sayan March 17, 2018 at 4:57 pm #

Thanks a lot sir, this was a very good and intuitive tutorial

Jason Brownlee March 18, 2018 at 6:01 am Start

#
Machine Learning ×
REPLY 

You can master applied Machine Learning

Thanks, I’m glad it helped.
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Nikhil Gupta March 19, 2018 at 11:12 pm # Email Address

I got a prediction model running successfully for fraud detection. My dataset is over 50 million
START MY EMAIL COURSE
and growing. I am seeing a peculiar issue.
When the loaded data is 10million or less, My prediction is OK.
As soon as I load 11 million data, My prediction saturates to a particular (say 0.48) and keeps on
repeating. That is all predictions will be 0.48, irrespective of the input.

I have tried will multiple combinations of the dense model.

# create model
model = Sequential()
model.add(Dense(32, input_dim=4, activation=’tanh’))
model.add(Dense(28, activation=’tanh’))
model.add(Dense(24, activation=’tanh’))
model.add(Dense(20, activation=’tanh’))
model.add(Dense(16, activation=’tanh’))
model.add(Dense(12, activation=’tanh’))
model.add(Dense(8, activation=’tanh’))
model.add(Dense(1, activation=’sigmoid’)) Start Machine Learning

REPLY 
Jason Brownlee March 20, 2018 at 6:21 am #

Perhaps check whether you need to train on all data, often a small sample is sufficient.

REPLY 
Nikhil Gupta March 22, 2018 at 2:45 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 120/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Oh. I believe that the machine learning accuracy will improve as we get more data over time.

REPLY 
Chandra Sutrisno Tjhong March 28, 2018 at 4:43 pm #

HI,

How do you define number of hidden layers and neurons per layer?

REPLY 
Jason Brownlee March 29, 2018 at 6:30 am #

There are no good heuristics, trial and error is a good approach. Discover what works best
Start Machine Learning ×
for your specific data.
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Aravind March 30, 2018 at 12:12 am #
Email Address
I executed the code and got the output, but how to use this prediction in the application.

START MY EMAIL COURSE

REPLY 
Jason Brownlee March 30, 2018 at 6:39 am #

Depends on the application.

REPLY 
Sabarish March 30, 2018 at 12:16 am #

What does the value 1.0 and 0..0 signifies??

REPLY 
Jason Brownlee March 30, 2018 at 6:39 am #

In what context? Start Machine Learning

REPLY 
Anand April 1, 2018 at 3:51 pm #

If number of inputs are 8 then why did you use 12 neurons in input layer ? Moreover why is
activation function used in input layer ?

REPLY 
Jason Brownlee April 2, 2018 at 5:19 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 121/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

The number of neurons in the first hidden layer can be different to the number of neurons in the input
layer (e.g. number of input features). They are only loosely related.

REPLY 
Lia April 1, 2018 at 11:49 pm #

Hello Sir,
Does the neural network use a standardized independent variable values, or should we feed it with
standardized ones in the fitting and predicting stages. Thanks

Jason Brownlee April 2, 2018 at 5:23 am #

Start Machine Learning ×
REPLY 

Try both and see what works best for your specific predictive modeling problem.
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
tareknahool April 4, 2018 at 5:17 am #
Email Address
you always fantastic, it’s a great lesson. But, frankly I don’t know what is the meaning of
“\n%s: %.2f%%” % and why you used the number(1)in that code(model.metrics_names[1],
START MY EMAIL COURSE
scores[1]*100))

REPLY 
Jason Brownlee April 4, 2018 at 6:19 am #

This is Python string formatting:

https://pyformat.info/

REPLY 
Abhilash Menon April 5, 2018 at 6:27 am #

Dr. Brownlee,

When we predict, is it possible to have the predictions for each row in the test data set right next to it in
Start Machine Learning
the same row. I thought of printing predictions and then copying it in excel but I am not sure if Keras
preserves order. Could you please help me out with this issue? Thanks so much for all your help!

REPLY 
Jason Brownlee April 5, 2018 at 3:05 pm #

Yes, the order of predictions matches the order of input values.

Does that help?

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 122/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Andrea Grandi April 9, 2018 at 6:37 am #

Is Deep Learning some kind of “black magic” ?

I had previously used scikit-learn and Machine Learning for the same dataset, trying to apply all the
techniques I did learn both here and on books, to get a 76% accuracy.

I tried this Keras tutorial, using TensorFlow as backend and I’m getting 80% accuracy at first try O_o

REPLY 
Jason Brownlee April 10, 2018 at 6:08 am #

No, not magic, just different.

Start Machine Learning ×
Well done though!
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Manny Corrao April 11, 2018 at 8:30 am #
Email Address
Can you tell us the column names? I think that is important because it helps us understand
what the network is evaluating and learning about.
START MY EMAIL COURSE
Thanks,

Manny

REPLY 
Jason Brownlee April 11, 2018 at 4:11 pm #

Yes, they are listed here:

https://github.com/jbrownlee/Datasets/blob/master/pima-indians-diabetes.names

REPLY 
rachit April 11, 2018 at 7:13 pm #

While Executing versions.py

Start Machine Learning
i am getting this error

Traceback (most recent call last):

File “versions.py”, line 2, in
import scipy
File “C:\Users\ATIT GARG\Anaconda3\lib\site-packages\scipy\__init__.py”, line 61, in
from numpy import show_config as show_numpy_config
File “C:\Users\ATIT GARG\Anaconda3\lib\site-packages\numpy\__init__.py”, line 142, in
from . import add_newdocs
File “C:\Users\ATIT GARG\Anaconda3\lib\site-packages\numpy\add_newdocs.py”, line 13, in
from numpy.lib import add_newdoc
File “C:\Users\ATIT GARG\Anaconda3\lib\site-packages\numpy\lib\__init__.py”, line 8, in

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 123/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

from .type_check import *

File “C:\Users\ATIT GARG\Anaconda3\lib\site-packages\numpy\lib\type_check.py”, line 11, in
import numpy.core.numeric as _nx
File “C:\Users\ATIT GARG\Anaconda3\lib\site-packages\numpy\core\__init__.py”, line 74, in
from numpy.testing import _numpy_tester
File “C:\Users\ATIT GARG\Anaconda3\lib\site-packages\numpy\testing\__init__.py”, line 12, in
from . import decorators as dec
File “C:\Users\ATIT GARG\Anaconda3\lib\site-packages\numpy\testing\decorators.py”, line 6, in
from .nose_tools.decorators import *
File “C:\Users\ATIT GARG\Anaconda3\lib\site-packages\numpy\testing\nose_tools\decorators.py”, line
20, in
from .utils import SkipTest, assert_warns
File “C:\Users\ATIT GARG\Anaconda3\lib\site-packages\numpy\testing\nose_tools\utils.py”, line 15, in
from tempfile import mkdtemp, mkstemp Start Machine Learning ×
File “C:\Users\ATIT GARG\Anaconda3\lib\tempfile.py”, line 45, in
from random import Random as _Random You can master applied Machine Learning
File “C:\Users\ATIT GARG\random.py”, line 7, in without math or fancy degrees.
from keras.models import Sequential Find out how in this free and practical course.
File “C:\Users\ATIT GARG\Anaconda3\lib\site-packages\keras\__init__.py”, line 3, in
from . import utils Email Address
File “C:\Users\ATIT GARG\Anaconda3\lib\site-packages\keras\utils\__init__.py”, line 4, in
from . import data_utils
START MY EMAIL COURSE
File “C:\Users\ATIT GARG\Anaconda3\lib\site-packages\keras\utils\data_utils.py”, line 23, in
from six.moves.urllib.error import HTTPError
ImportError: cannot import name ‘HTTPError’

REPLY 
Jason Brownlee April 12, 2018 at 8:35 am #

Perhaps you need to update your environment?

REPLY 
Gray April 14, 2018 at 4:25 am #

Jason – very impressive work! Even more impressive is your detailed answer to every question.
I went through them all and got a lot of useful information.
StartGreat job! Learning
Machine

REPLY 
Jason Brownlee April 14, 2018 at 6:50 am #

Thanks Gray!

REPLY 
octdes April 14, 2018 at 2:39 pm #

Hello Jason,
Thank’s for the good tuto !
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 124/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

How would you name/describe the structure of this neuronal network ?

The point is that i find strange that you can have a different nmber of input and of neurones in the input
layer. Most of the neuronal network diagramm i have seen, each input is directly connected with one
neurone of the input layer. I have never seen a neuronal network diagramm where the number of input is
different with the number of neurones in the input layer.
Do you have counterexample or do there is something i understand wrong ?
Thank you for your work and sharing your knowledge

REPLY 
Jason Brownlee April 15, 2018 at 6:24 am #

The type of neural network in this post is a multi-layer perceptron or an MLP for short.

Start
The first “layer” in the code actually defines both the Machine
input layer Learning
and the first hidden layer at the ×
same time.
You can master applied Machine Learning
The number of inputs must match the number of columns in theor
without math input data.
fancy The number of neurons
degrees.
in the first hidden layer can be anything you want. Find out how in this free and practical course.

Does that help?

Email Address

START MY EMAIL COURSE REPLY 

Ashley April 16, 2018 at 7:29 am #

Thank you VERY much for this tutorial, Jason! It is the best I have found on the internet. As a
political scientist pursuing complex outcomes like this one, I was looking for models that allow for more
complicated relationships. Your code and post are so clearly articulated; I was able to adapt it for my
purposes more easily than I thought would be possible. One possible extension of your work, and
possibly this tutorial, would be to map the layers and nodes onto a theory of the data generating
process.

REPLY 
Jason Brownlee April 16, 2018 at 2:54 pm #

Thanks Ashley, I’m glad it helped.

Thanks for the suggestion.

Start Machine Learning

REPLY 
Eric Miles April 20, 2018 at 1:22 am #

I’m just starting out working through your site – thanks for the great resource! I wanted to point
out what I think is a typo: in the code block just before Section 2 “Define Model” I believe we just want X
= dataset[:,0:7] so that we don’t include the output variables in our inputs.

REPLY 
Jason Brownlee April 20, 2018 at 6:00 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 125/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

No, it is correct Eric.

X will have 8 columns (0-7), the original dataset has 9.

You can learn more about array slicing and ranges in Python here:
https://machinelearningmastery.com/index-slice-reshape-numpy-arrays-machine-learning-python/

REPLY 
Rafa April 28, 2018 at 12:50 am #

Great tutorial, finally I have found a good web about deep learning (Y)

Jason Brownlee April 28, 2018 at 5:31 am # Start Machine Learning ×

REPLY 

You can master applied Machine Learning

Thanks.
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Vivek May 7, 2018 at 8:31 pm # Email Address

Great tutorial thank for help. I have one project in which i have to do CAD images(basically 3-d
START MY EMAIL COURSE
mechanical image classification). can you please give road map how can i proceed?
I am new and i dont have any idea

REPLY 
Jason Brownlee May 8, 2018 at 6:12 am #

This is my general roadmap for a predictive modeling problem:

https://machinelearningmastery.com/start-here/#process

REPLY 
Vivek May 9, 2018 at 10:03 pm #

Thanks a lot sir. This will help me to proceed

Start Machine Learning

REPLY 
Jason Brownlee May 10, 2018 at 6:31 am #

I’m glad to hear that.

REPLY 
Rahmad ars May 8, 2018 at 1:36 am #

Thanks sir for the tutorial.

Actually i still have some question:

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 126/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

1. Is this backpropagation neural network?

2. How to initialize nguyen-widrow random weights
3. I have my own dataset, each consist of 1×64 matrix, which is the correct one? I normalize each
column of it, or each row of it?

Thanks.
Im the one who asked u in backpropagation from scratch page

REPLY 
Jason Brownlee May 8, 2018 at 6:16 am #

Yes, it uses backpropgation to update the weights.

Sorry, I don’t know about that initialization method, you can see the supported methods here:
Start Machine Learning ×
https://keras.io/initializers/
Youworks
Try a suite of data preparation schemes to see what can master applied
best for your Machine Learningand chosen
specific dataset
model. without math or fancy degrees.
Find out how in this free and practical course.

Email Address
REPLY 
Hussein May 9, 2018 at 10:33 pm #

Hi Jason, START MY EMAIL COURSE

This is a very nice intro to a daunting but intriguing technology! I wanted to play around with your code
and see if I could come up with some simple dataset and see how the predictions will work out – one
idea that occurred to me is, can I make a model that predicts what country a telephone number belongs
to. So the training dataset looks like a 2 column CSV, phone number and country…that’s basically one
feature. Do you think this would be effective at all? What other features could be added here? I’ll still give
this a shot, but would appreciate any thoughts/ideas!

Thanks!

REPLY 
Jason Brownlee May 10, 2018 at 6:33 am #

The country code would make it too simple a problem – e.g. it can be solved with a look-up
table. Start Machine Learning

REPLY 
Hussein May 10, 2018 at 4:24 pm #

True, I just wanted to see if machine learning could be used to “figure out” the lookup
table as opposed to be provided with one by the user, given enough data..not a practical use-
case, but as a learning exercise. As it turns out, my data-set of about 700 phone numbers wasn’t
effective for this. But again, is this because the problem had too few features, i.e in my case, just
one? What if I increased the number of features, say phone number, country code, city the
phone number belongs to, maybe even the cellphone company the number is registered to, do
you think that would make the training more effective?

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 127/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee May 11, 2018 at 6:33 am #

If you can write an if statement or use a look-up table to solve the problem, then it
might be a bad fit for machine learning.

This post will help you frame your problem:

http://machinelearningmastery.com/how-to-define-your-machine-learning-problem/

Hussein May 11, 2018 at 5:15 pm #

Start
Thanks Jason for that resource. Machine
I’ll check Learning
it out. I also came across this ×
(https://elitedatascience.com/machine-learning-projects-for-beginners) that I’m reading
through, for anyone else that’s lookingYou
for can master
a small MLapplied
problemMachine Learning
to solve as a learning
without math or fancy degrees.
experience.
Find out how in this free and practical course.

Email Address
Jason Brownlee May 12, 2018 at 6:27 am #

Great. START MY EMAIL COURSE

REPLY 
Frank Lu May 14, 2018 at 7:44 pm #

Great tutorial very helpful ,then I have a question .Which accounted for the largest proportion in
8 inputs? We have 8 factors in the dataset like pregnancies, glucose, bloodpressure and the others. So ,
Which factor is most related to diabetes used? How do we know this proportion through MLP?
Thanks！

REPLY 
Jason Brownlee May 15, 2018 at 7:53 am #

We might not know. This is the differenceStart

between descriptive
Machine and predictive models.
Learning
This is really the issue of model interpretability, I write more about it here:
https://machinelearningmastery.com/faq/single-faq/how-do-i-interpret-the-predictions-from-my-model

REPLY 
Paolo May 16, 2018 at 7:59 pm #

Hi Jason,
thanks for your tutorials.

I have a question, do you use keras with pandas too? In this case, it is better to import data wih numpy
anyway? What do you suggest?

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 128/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Thank you again,

Paolo

REPLY 
Jason Brownlee May 17, 2018 at 6:31 am #

Yes, and yes.

REPLY 
Stefan November 10, 2018 at 1:06 am #

arrays?
How so? I usually see pandas.readcsv() to read files. Does keras only accept numpy
Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course. REPLY 
Jason Brownlee November 10, 2018 at 6:07 am #

Correct. Email Address

START MY EMAIL COURSE

REPLY 
zohreh May 20, 2018 at 9:14 am #

Thanks for your great tutorial. I have a credit card dataset and I want to do fraud detection on it.
it has 312 columns, So before doing DNN, I should do dimension reduction, then using DNN? and
another question is that Is it possible to do CNN on my dataset as well?

Thank you

REPLY 
Jason Brownlee May 21, 2018 at 6:24 am #

Yes, choose the features that best map to the output variable.

A CNN can be used if there is a spatial relationship in the data, such as a sequence of transactions
over space or time. Start Machine Learning

REPLY 
zohreh May 23, 2018 at 6:44 am #

Thanks for your answer, So I think CNN doesn’t make sense for my dataset,
Do you have any tutorial for active learning?
thanks for your time.

REPLY 
Jason Brownlee May 23, 2018 at 2:37 pm #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 129/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

I don’t know if it is appropriate, I was trying to provide enough information for you to
make that call.

I hope to cover active learning in the future.

zohreh May 24, 2018 at 3:13 am #

yes I understand, I said according to your provided information, thank you so

much for your answers and great tutorials.

Miguel García May 24, 2018 at 11:55 am # Start Machine Learning ×

REPLY 

Can you share a tutorial for first neural netowrk

Youwith
canmultilabel support?
master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Jason Brownlee May 24, 2018 at 1:51 pm #
Email Address

Thanks for the suggestion.

START MY EMAIL COURSE

REPLY 
Sathish May 24, 2018 at 12:57 pm #

how to create convolutional layers and visualize features in keras

REPLY 
Jason Brownlee May 24, 2018 at 1:51 pm #

Good question, sorry, I don’t have a worked example.

REPLY 
Anam May 28, 2018 at 3:52 am #
Start Machine Learning
Dear Jason,
I get an error”ValueError: could not convert string to float: “Kindly help to solve the issue.And I am using
my own dataset which consist of text not numbers(like the dataset you have used).
Thanks!

REPLY 
Jason Brownlee May 28, 2018 at 6:04 am #

This might give you some ideas:

https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-
me

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 130/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Anam May 29, 2018 at 7:26 am #

Dear Jason,
I am running your code example from section 6.But I get an error in the following code snippet:

Code Snippet:
dataset = numpy.loadtxt(“pima_indians.csv”, delimiter=”,”)
# split into input (X) and output (Y) variables
X = dataset[:,0:8]
Y = dataset[:,8]

Error:
ValueError: could not convert string to float: “6 Start Machine Learning ×
Kindly guide me to solve the issue. Thanks for your precious
You cantime.
master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Jason Brownlee May 29, 2018 at 2:49 pm #
Email Address
I’m sorry to hear that, I have some suggestions here:
https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-
START MY EMAIL COURSE
me

REPLY 
Gautam Sharma June 19, 2018 at 1:20 am #

Did you find any solution as I am getting the same error?

REPLY 
moti June 4, 2018 at 3:34 am #

Hi Doctor, in this python code where shall I get the “keras” package?

Start Machine Learning

REPLY 
Jason Brownlee June 4, 2018 at 6:34 am #

This tutorial shows you how to install Keras:

https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-
anaconda/

REPLY 
Ammara Habib June 5, 2018 at 5:13 am #

Hy jason, Thanks for an amazing post. I have a question here that can we use dense layer as
input for text classification(e.g : sentiment classification of movie reviews).If yes than how can we convert

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 131/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

the text dataset into numeric for dense layer.

REPLY 
Jason Brownlee June 5, 2018 at 6:47 am #

You can, although it is common to one hot encode the text or use an embedding layer.

I have examples of both on the blog.

REPLY 
Ammara Habib June 5, 2018 at 9:18 am #

Thanks for your precious time.Sir, you mean that first i use embedding layer as input layer and
Start Machine Learning ×
then i use dense layer as the hidden layer?
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Jason Brownlee June 5, 2018 at 3:05 pm #
Email Address
Yes.

START MY EMAIL COURSE

REPLY 
Lisa Xie June 15, 2018 at 1:12 pm #

Hi,thanks for your tutorial. I am wondering how you set the number neurons and activation
functions for each layer, eg. 12 neurons for the 1st layer and 8 for the second.

REPLY 
Jason Brownlee June 15, 2018 at 2:50 pm #

I used a little trial and error.

REPLY 
Marwa June 18, 2018 at 1:25 am #
Start Machine Learning
Hi jason,

I developped two neural networks using keras but I have this error:

line 1336, in _do_call

raise type(e)(node_def, op, message)

ResourceExhaustedError: OOM when allocating tensor with shape[7082368,50]

[[Node: training_1/Adam/Variable_14/Assign = Assign[T=DT_FLOAT, _class=
[“loc:@training_1/Adam/Variable_14″], use_locking=true, validate_shape=true,
_device=”/job:localhost/replica:0/task:0/device:GPU:0”](training_1/Adam/Variable_14,
training_1/Adam/zeros_14)]]

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 132/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Have you an idea?

Thanks.

REPLY 
Jason Brownlee June 18, 2018 at 6:42 am #

Sorry, I have not seen this error before. Perhaps try posting/searching on stackoverflow?

REPLY 
prateek bhadauria June 23, 2018 at 11:38 pm #

sir i have a regression related dataset which contains an array of 49999 rows and 20 coloumns
, i want to implement CNN on this dataset , Start Machine Learning ×
i put my code as per my perception kindly give me suggestion , to correct
You can master it i Machine
applied was stuck mainly by putting
Learning
my dense dimension specially without math or fancy degrees.
Find out how in this free and practical course.
from keras.models import Sequential
from keras.layers import Dense
import numpy as np Email Address
import tensorflow as tf
from matplotlib import pyplot START MY EMAIL COURSE
from sklearn.datasets import make_regression
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_squared_error
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.preprocessing import StandardScaler
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.optimizers import SGD

seed = 7
np.random.seed(seed)
from scipy.io import loadmat
dataset = loadmat(‘matlab2.mat’)
Bx=basantix[:, 50001:99999]
Bx=np.transpose(Bx)
Fx=fx[:, 50001:99999] Start Machine Learning
Fx=np.transpose(Fx)

from sklearn.cross_validation import train_test_split

Bx_train, Bx_test, Fx_train, Fx_test = train_test_split(Bx, Fx, test_size=0.2, random_state=0)

scaler = StandardScaler() # Class is create as Scaler

scaler.fit(Bx_train) # Then object is created or to fit the data into it
Bx_train = scaler.transform(Bx_train)
Bx_test = scaler.transform(Bx_test)

model = Sequential()
def base_model():

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 133/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

keras.layers.Dense(Dense(49999, input_shape=(20,), activation=’relu’))

model.add(Dense(20))
model.add(Dense(49998, init=’normal’, activation=’relu’))
model.add(Dense(49998, init=’normal’))
model.compile(loss=’mean_squared_error’, optimizer = ‘adam’)
return model

scale = StandardScaler()
Bx = scale.fit_transform(Bx)
Bx = scale.fit_transform(Bx)

clf = KerasRegressor(build_fn=base_model, nb_epoch=100, batch_size=5,verbose=0)

clf.fit(Bx,Fx)
res = clf.predict(Bx)
Start Machine Learning ×
## line below throws an error
clf.score(Fx,res) You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Jason Brownlee June 24, 2018 at 7:33 am # Email Address

Sorry, I cannot debug your code for you. Perhaps post your code and error to
stackoverflow? START MY EMAIL COURSE

REPLY 
Madhav Prakash June 24, 2018 at 3:01 am #

Hi Jason,
Looking at the dataset, I could find that there were many attributes with each of them differing in terms of
units. Why haven’t you rescaled/normalised the data? but still managed to get an accuracy of 75%?

REPLY 
Jason Brownlee June 24, 2018 at 7:35 am #

Ideally, we should rescale the data.

The relu activation function is more flexible with unscaled data. Learning
Start Machine

REPLY 
Madhav Prakash June 24, 2018 at 4:23 pm #

Ohkay, thanks.
Also, I’ve implemented a NN on a database similar to this, where the accuracy varies b/w 70-
75%. I’ve tried to increase the accuracy by tuning various parameters and functions (learning
rate, no. of layers, neurons per level, earlystopping, activation fn, initialization, optimizer etc…)
but it was not a success. My question is when do i come to know that i’ve reached the maximum
accuracy possible for my implementation? Do i stay content with the current accuracy?

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 134/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee June 25, 2018 at 6:19 am #

When we run out of time or ideas.

I list some more ideas here:

http://machinelearningmastery.com/machine-learning-performance-improvement-cheat-
sheet/

And here:
http://machinelearningmastery.com/improve-deep-learning-performance/

Aarron Wilson July 8, 2018 at 8:19 am #

Start Machine Learning ×
REPLY 

First of all thanks for the tutorial. Also I acknowledge that this network is more for educational
You can master applied Machine Learning
purposes. Yet this network can be improved to 83-84% accuracy with standard normalization alone. Also
without math or fancy degrees.
it can hit 93-95% accuracy by using a deeper model.
Find out how in this free and practical course.
#Standard normalization
X= StandardScaler().fit_transform(X) Email Address
#and a deeper model
model = Sequential() START MY EMAIL COURSE
model.add(Dense(12, input_dim=8, activation=’relu’))
model.add(Dense(12, activation=’relu’))
model.add(Dense(12, activation=’relu’))
model.add(Dense(12, activation=’relu’))
model.add(Dense(12, activation=’relu’))
model.add(Dense(8, activation=’relu’))
model.add(Dense(1, activation=’sigmoid’))

REPLY 
Jason Brownlee July 9, 2018 at 6:30 am #

Thanks, yes, normalization is a good idea in general when working with neural nets.

Start Machine Learning

REPLY 
Alex July 10, 2018 at 3:47 am #

Hi, thank you for this great article

Imagine that in my dataset instead of diabetes being a 0 or 1 I have 3 results, I mean, the data rows are
like this

data1, data2, sickness

123, 124, 0
142, 541, 0
156, 418, 1
142, 541, 1
156, 418, 2
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 135/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

So, I need to categorize for 3 values, If I use this same example you gave us how can I determine the
output?

REPLY 
Jason Brownlee July 10, 2018 at 6:51 am #

The output will be sickness Alex. Perhaps I don’t understand your question?

REPLY 
Alex July 10, 2018 at 7:11 am #

The output will be sickness yes

Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees. REPLY 
Alex July 10, 2018 at 10:17 am #
Find out how in this free and practical course.

Sorry for my English, it is not my natal tongue, I will re do my quesyion. What I mean is this, I
will be having a label with more than 2 results, 0 is one Email
sickness, 1 will be other and 2 will be other.
Address

How can I use the model you showed us to fit the 3 results?
START MY EMAIL COURSE

REPLY 
Jason Brownlee July 10, 2018 at 2:26 pm #

I see, this is called a multi-class classification problem.

This tutorial will help:

https://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/

REPLY 
adsad July 11, 2018 at 1:06 am #

is it possible to predict the lottery outcome. if so how?

Start Machine Learning

REPLY 
Jason Brownlee July 11, 2018 at 5:59 am #

No. I explain more here:

https://machinelearningmastery.com/faq/single-faq/can-i-use-machine-learning-to-predict-the-lottery

REPLY 
Tom July 14, 2018 at 2:32 am #

Hi Jason, I run your first example code in this tutorial. but what makes me confused is:

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 136/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Why the final training accuracy (0.7656) is different from the evaluated scores (78.26%) in the same
datasets (training set) ? I can’t figure it out. Can you tell me please? Thanks a lot!

Epoch 150/150
768/768 [==============================] – 0s – loss: 0.4827 – acc: 0.7656
32/768 [>………………………..] – ETA: 0s
acc: 78.26%

REPLY 
Jason Brownlee July 14, 2018 at 6:20 am #

One is the performance on the training set, the other on the validation set.

You can learn more about the difference here:

Start Machine Learning
https://machinelearningmastery.com/difference-test-validation-datasets/
×
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Tom July 14, 2018 at 9:09 pm #

Thanks for the rapid reply. But I noticed that inEmail Address
your code the training set and validation set are
exactly the same dataset. Please check it for confirmation. The code is in the part “6. Tie It All Together”.

# Fit the model START MY EMAIL COURSE

model.fit(X, Y, epochs=150, batch_size=10)
# evaluate the model
scores = model.evaluate(X, Y)

So, my problem is still the same: Why the final training accuracy (0.7656) is different from the evaluated
scores (78.26%) in the same datasets?
Thanks!

REPLY 
Jason Brownlee July 15, 2018 at 6:14 am #

Perhaps verbose output might be accumulated over each batch rather than summarizing
skill at the end of the training epoch.

Start Machine Learning

REPLY 
ami July 16, 2018 at 2:01 am #

Hello Jason,
Do you have some tutorial on signal processing using CNN ? I have csv files of some biomedical signals
like ECG and i want to classify normal and abnormal signals using deep learning.

With Regards

REPLY 
Jason Brownlee July 16, 2018 at 6:11 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 137/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Yes, I have a suite of tutorials scheduled on this topic. They should be out soon.

REPLY 
EL July 16, 2018 at 7:19 pm #

Hi, thank you so much for your tutorial. I am trying to make a neural network that will take a
dataset and return if it is suitable to be analyzed by another program i have. Is it possible to feed this
with acceptable datasets and unacceptable datasets and then call it on a new dataset and then return
whether this dataset is acceptable? Thank you for your help, I am very new to machine learning.

×
REPLY 
Jason Brownlee July 17, 2018 at 6:14 am #
Start Machine Learning
Try it and see how you go.
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
ami July 18, 2018 at 2:37 pm #
Email Address
Oh really ! Thank you so much. Can you please notify me when the tutorials will be out because
i am doing a project and i am stuck right now.
START MY EMAIL COURSE
With Regards

REPLY 
Diagrams July 30, 2018 at 2:45 pm #

It would be very very helpful for newcomers if you had a diagram of the network, showing
individual nodes and graph edges (and bias nodes and activation functions), and indicating on it which
parts were generated by which model.add commands/parameters. Similar to
https://zhu45.org/posts/2017/May/25/draw-a-neural-network-through-graphviz/

I’ve tried visualizing it with from keras.utils.plot_model and tensorboard, but neither produce a node-level
diagram.

Start Machine Learning

REPLY 
Jason Brownlee July 31, 2018 at 5:58 am #

Thanks for the suggestion.

REPLY 
Aravind July 30, 2018 at 7:57 pm #

can anyone tell a simple way to run my ann keras tensorflow backend in GPU. Thanks

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 138/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee July 31, 2018 at 6:00 am #

The simplest way I know how:

https://machinelearningmastery.com/develop-evaluate-large-deep-learning-models-keras-amazon-
web-services/

REPLY 
farli August 6, 2018 at 1:08 pm #

Did you use back propagation here?

Start Machine Learning ×

REPLY 
Jason Brownlee August 6, 2018 at 2:54 pm #
You can master applied Machine Learning
Yes. without math or fancy degrees.
Find out how in this free and practical course.

Email Address
REPLY 
farli August 13, 2018 at 9:40 am #

START MY neural
Can you please make a tutorial on convolutional EMAIL COURSE
net? That would be really
helpful ..:)

REPLY 
Jason Brownlee August 13, 2018 at 2:27 pm #

Yes, i have many on the blog already. Try the blog search.

REPLY 
Karim Gamal August 7, 2018 at 8:52 pm #

I have a problem where I get the result as shown below

Epoch 146/150 – 0s – loss: -1.2037e+03 – acc: 0.0000e +00

Start
Epoch 147/150 – 0s – loss: -1.2037e+03 – acc: 0.0000e +00Machine Learning
Epoch 148/150 – 0s – loss: -1.2037e+03 – acc: 0.0000e +00
Epoch 149/150 – 0s – loss: -1.2037e+03 – acc: 0.0000e +00
Epoch 150/150 – 0s – loss: -1.2037e+03 – acc: 0.0000e +00

where in my data set the output is a value between 0 to 500 not only 0 and 1
so how can I fix this in my code

REPLY 
Jason Brownlee August 8, 2018 at 6:18 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 139/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Sounds like a regression problem. Change the activation function in the output layer to linear and the
loss function to ‘mse’.

See this tutorial:

https://machinelearningmastery.com/regression-tutorial-keras-deep-learning-library-python/

REPLY 
Tim August 15, 2018 at 5:54 am #

AWESOME!!! Thanks so much for this.

Jason Brownlee August 15, 2018 at 6:11 amStart

# Machine Learning ×
REPLY 

You’re welcome, I’m happy it helped. You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
tania August 27, 2018 at 8:35 pm #
Email Address

Hi Jason,
START MY EMAIL COURSE
Thank you for the tutorial. I am relatively new to ML and I am currently working on a classification
problem that is non binary.

My dataset consists of a number of labeled samples – all measuring the same quantity/unit. The amount
typically ranges from 10 to 20 labeled samples/inputs. However, the feed forward or testing sample will
only contain 7 of those inputs (at random).

I’m struggling to find a solution to designing a system that accepts fewer inputs than what is typically
found in the training set.

REPLY 
Jason Brownlee August 28, 2018 at 5:59 am #

Perhaps try following this process:

https://machinelearningmastery.com/start-here/#process
Start Machine Learning

REPLY 
Vaibhav Jaiswal September 10, 2018 at 6:28 pm #

Great tutorial there! But the main aspect of the model is to predict on a sample. If i print the first
predicted value,it shows me some values for all the columns of categorical features. How to get the
predicted number from the sample?

REPLY 
Jason Brownlee September 11, 2018 at 6:26 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 140/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

The order of the predictions matches the order of the inputs.

REPLY 
Glen September 19, 2018 at 10:45 pm #

I think I must be doing something wrong, I keep getting the error:

File “C:\Users\glens\Anaconda3\lib\site-packages\tensorflow\python\framework\errors_impl.py”, line 519,
in __exit__
c_api.TF_GetCode(self.status.status))

InvalidArgumentError: Input to reshape is a tensor with 10 values, but the requested shape has 1
[[Node: training_19/Adam/gradients/loss_21/dense_64_loss/Mean_1_grad/Reshape =
Reshape[T=DT_FLOAT, Tshape=DT_INT32, _class=
Start Machine Learning
[“loc:@training_19/Adam/gradients/loss_21/dense_64_loss/Mean_1_grad/truediv”], ×
_device=”/job:localhost/replica:0/task:0/device:GPU:0″]
You can master applied Machine Learning
(training_19/Adam/gradients/loss_21/dense_64_loss/mul_grad/Sum,
without math or fancy degrees.
training_19/Adam/gradients/loss_21/dense_64_loss/Mean_1_grad/DynamicStitch/_1703)]]
Find out how in this free and practical course.
Are you able to shed any light on why I would get this error?

Thankyou Email Address

START MY EMAIL COURSE

REPLY 
Jason Brownlee September 20, 2018 at 7:59 am #

I have not seen this error, I have some suggestions here:

https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-
me

REPLY 
Snehasish September 19, 2018 at 11:15 pm #

Hi Jason, thanks for this awesome tutorial. I have one doubt – why did the evaluation not
produce 100% accuracy? After all, we used the same dataset for evaluation as the one used for training
itself.

Start Machine Learning

REPLY 
Jason Brownlee September 20, 2018 at 8:00 am #

Good question!

We are approximating a challenging mapping function, not memorizing examples. As such, there will
always be error.

I explain more here:

https://machinelearningmastery.com/faq/single-faq/why-cant-i-get-100-accuracy-or-zero-error-with-
my-model

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 141/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Mark C September 27, 2018 at 12:49 am #

How do you predict something you want to predict such as new data. for example I did a spam
detection but dont know how to predict whether a sentence i write is spam or not .

REPLY 
Jason Brownlee September 27, 2018 at 6:01 am #

You can call model.predict() with a finalized model. More here:

https://machinelearningmastery.com/faq/single-faq/how-do-i-make-predictions

Start Machine Learning ×

REPLY 
Vivek October 1, 2018 at 3:17 am #
You can master applied Machine Learning
Hello Sir,
without math or fancy degrees.
I am new and understood some part of your code. I have Findquestion
out how in
in this free andmodel
prediction practical course.we
basically
divide our data into training and test set. In the example above the entire dataset is used as training
dataset. How can we train the model on training set useEmail
it for Address
the prediction on test set?

START MY EMAIL COURSE

REPLY 
Jason Brownlee October 1, 2018 at 6:28 am #

Great question, yes, train the model on all available data and then use it to start making
predictions.

More here:
https://machinelearningmastery.com/train-final-machine-learning-model/

REPLY 
Vivek35 October 1, 2018 at 7:11 am #

Hello Sir,
It’s great tutorial to understand. However, I am new and want to understand something out of it. In the
above code we have treated entire dataset as training set. Can we divide this into training set and test
Start Machinecan
set, apply model to training set and use it for test set prediction.How Learning
we achieve with the above
code?

REPLY 
Jason Brownlee October 1, 2018 at 2:39 pm #

Thanks.

Yes, you can split the dataset manually or use scikit-learn to make the split for you. I explain more
here:
https://machinelearningmastery.com/faq/single-faq/how-do-i-evaluate-a-machine-learning-algorithm

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 142/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Lipi October 5, 2018 at 6:26 am #

Hi Jason,

I am trying to predict using my neural network. I have used MinMaxScaler in the features while training
the data. I don’t get a good prediction if I don’t use the same transform function on the prediction data
set which I used on the features while training the data. Could you suggest me the correct approach in
this situation?

REPLY 
Jason Brownlee October 5, 2018 at 2:29 pm #

You must use the same transform to bothStart

prepareMachine
training data and to make predictions on
Learning ×
new data.
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Lipi October 5, 2018 at 10:12 pm #
Email Address
Thank you!

START MY EMAIL COURSE

REPLY 
neenu October 6, 2018 at 3:57 pm #

hi i am new to this i writew following code in spyder

from keras.models import Sequential
from keras.layers import Dense
import numpy
# fix random seed for reproducibility
numpy.random.seed(7)
# load pima indians dataset
dataset = numpy.loadtxt(“pima-indians-diabetes.txt”,encoding=”UTF8″, delimiter=”,”)
# split into input (X) and output (Y) variables
X = dataset[:,0:8]
Y = dataset[:,8]

# create model Start Machine Learning

model = Sequential()
model.add(Dense(12, input_dim=8, activation=’relu’))
model.add(Dense(8, activation=’relu’))
model.add(Dense(1, activation=’sigmoid’))
# Compile model
model.compile(loss=’binary_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])
# Fit the model
model.fit(X, Y, epochs=150, batch_size=10)
# evaluate the model
scores = model.evaluate(X, Y)
print(“\n%s: %.2f%%” % (model.metrics_names[1], scores[1]*100))

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 143/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

And i got this as output

runfile(‘C:/Users/DELL/Anaconda3/Scripts/temp.py’, wdir=’C:/Users/DELL/Anaconda3/Scripts’)
Using TensorFlow backend.
Traceback (most recent call last):

File “”, line 1, in

runfile(‘C:/Users/DELL/Anaconda3/Scripts/temp.py’, wdir=’C:/Users/DELL/Anaconda3/Scripts’)

File “C:\Users\DELL\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py”, line

668, in runfile
execfile(filename, namespace)

File “C:\Users\DELL\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py”, line

108, in execfile
exec(compile(f.read(), filename, ‘exec’), namespace)
Start Machine Learning ×
File “C:/Users/DELL/Anaconda3/Scripts/temp.py”, line 1, in
You can master applied Machine Learning
from keras.models import Sequential
without math or fancy degrees.
File “C:\Users\DELL\Anaconda3\lib\site-packages\keras\__init__.py”, linefree
Find out how in this 3, in
and practical course.
from . import utils

Email Address
File “C:\Users\DELL\Anaconda3\lib\site-packages\keras\utils\__init__.py”, line 6, in
from . import conv_utils

START MY EMAIL COURSE

File “C:\Users\DELL\Anaconda3\lib\site-packages\keras\utils\conv_utils.py”, line 9, in
from .. import backend as K

File “C:\Users\DELL\Anaconda3\lib\site-packages\keras\backend\__init__.py”, line 89, in

from .tensorflow_backend import *

File “C:\Users\DELL\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py”, line 5, in

import tensorflow as tf

File “C:\Users\DELL\Anaconda3\lib\site-packages\tensorflow\__init__.py”, line 22, in

from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import

File “C:\Users\DELL\Anaconda3\lib\site-packages\tensorflow\python\__init__.py”, line 49, in

from tensorflow.python import pywrap_tensorflow

File “C:\Users\DELL\Anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow.py”, line 74, in

raise ImportError(msg)

ImportError: Traceback (most recent call last): Start Machine Learning

File “C:\Users\DELL\Anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py”, line
14, in swig_import_helper
return importlib.import_module(mname)
File “C:\Users\DELL\Anaconda3\lib\importlib\__init__.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 994, in _gcd_import
File “”, line 971, in _find_and_load
File “”, line 955, in _find_and_load_unlocked
File “”, line 658, in _load_unlocked
File “”, line 571, in module_from_spec
File “”, line 922, in create_module

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 144/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

File “”, line 219, in _call_with_frames_removed

ImportError: DLL load failed with error code -1073741795

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File “C:\Users\DELL\Anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow.py”, line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File “C:\Users\DELL\Anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py”, line
17, in
_pywrap_tensorflow_internal = swig_import_helper()
File “C:\Users\DELL\Anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py”, line
16, in swig_import_helper
return importlib.import_module(‘_pywrap_tensorflow_internal’)
File “C:\Users\DELL\Anaconda3\lib\importlib\__init__.py”, line 126, in import_module
Start Machine Learning ×
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named ‘_pywrap_tensorflow_internal’
You can master applied Machine Learning
without math or fancy degrees.
Failed to load the native TensorFlow runtime.
Find out how in this free and practical course.
See https://www.tensorflow.org/install/install_sources#common_installation_problems
Emailstack
for some common reasons and solutions. Include the entire Address
trace
above this error message when asking for help.

START MY EMAIL COURSE

REPLY 
Jason Brownlee October 7, 2018 at 7:24 am #

I recommend this tutorial to help you setup your environment:

https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-
anaconda/

I recommend that you don’t use am IDE or notebook:

https://machinelearningmastery.com/faq/single-faq/why-dont-use-or-recommend-notebooks

Instead, I recommend you save code to a .py file and run from the command line:
https://machinelearningmastery.com/faq/single-faq/how-do-i-run-a-script-from-the-command-line

Start Machine Learning REPLY 

kamal October 15, 2018 at 1:08 am #

sir please provide the python code for adaptive neuro fuzzy classifier

REPLY 
Jason Brownlee October 15, 2018 at 7:31 am #

Thanks for the suggestion.

REPLY 
Shahbaz October 24, 2018 at 4:44 am #
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 145/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

blessed on u sir,
can u give me idea about OCR system, for my final year project, plz give me back-end stratigy
for OCR , r u have any code on OCR

REPLY 
Jason Brownlee October 24, 2018 at 6:32 am #

Perhaps start here:

http://machinelearningmastery.com/handwritten-digit-recognition-using-convolutional-neural-
networks-python-keras/

Andrew Agib October 29, 2018 at 10:39 pm # Start Machine Learning ×

REPLY 

You can master applied Machine Learning

model.compile(loss=’binary_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])
without math or fancy degrees.
show a syntax error on that sentence what could be the reason
Find out how in this free and practical course.

Email Address
REPLY 
Jason Brownlee October 30, 2018 at 6:02 am #
START MY EMAIL COURSE
I have some suggestions here:
https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-
me

REPLY 
VASUDEV K P November 3, 2018 at 10:13 pm #

Hello Jason,

I have the theano back end installed. I am using Windows OS and during execution I am getting an error
“No module named TensorFlow”. Please help

REPLY 
Start
Jason Brownlee November 4, 2018 at 6:27 am # Machine Learning

You may have to change the configuration of Keras to use Theano instead.

More details here:

https://keras.io/backend/

REPLY 
Imen Drs November 4, 2018 at 7:09 am #

Hi Jason,
Please,how can we calculate the precision and recall of this example?
And thanks.
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 146/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee November 5, 2018 at 6:06 am #

You can use scikit-learn metrics:

http://scikit-learn.org/stable/modules/classes.html#sklearn-metrics-metrics

REPLY 
Stefan November 10, 2018 at 2:59 am #

I thought sigmoid and softmax were quite similar activation functions. But when trying the same
model with softmax as activation for the last layer instead of sigmoid, my accuracy is much much worse.
Start Machine Learning ×
Does that make sense to you? If so why? I feel like I see softmax more often in other code than sigmoid.
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Jason Brownlee November 10, 2018 at 6:09 am #

Nope. Email Address

Sigmoid for 2 classes.

Softmax for >2 classes START MY EMAIL COURSE

REPLY 
Amuda Kamorudeen November 10, 2018 at 4:46 pm #

I’m working on model that will predict propensity of customer that are likely to terminate their
service with company. I have dataset of 70000 rows and 500 columns, Please how can I pass numeric
data as an input to a convolutional neural network (CNN) .

REPLY 
Jason Brownlee November 11, 2018 at 5:59 am #

CNNs are only appropriate for data with a spatial relationship, such as images, time series
and text. Start Machine Learning

REPLY 
irfan November 18, 2018 at 3:22 pm #

hi jason,

i am using tensor flow as backend.

from keras.models import Sequential
from keras.layers import Dense
import sys
from keras import layers
from keras.utils import plot_model
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 147/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

print (model.layer())

erro.

—————————————————————————
AttributeError Traceback (most recent call last)
in
9 model.add(Dense(512, activation=’relu’))
10 model.add(Dense(10, activation=’sigmoid’))
—> 11 print (model.layer())
12 # Compile model
13 model.compile(loss=’binary_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])

AttributeError: ‘Sequential’ object has no attribute ‘layer’

Start Machine Learning ×

You can master applied Machine Learning REPLY 
Jason Brownlee November 19, 2018 at 6:44 am #
without math or fancy degrees.
Why are you trying to print model.layer()?Find out how in this free and practical course.

Email Address

REPLY 
Mario December 2, 2018 at 5:30 am #
START MY EMAIL COURSE
Hi Jason
First thanks for amazing tutorial , since your scripts are using list of values while my inputs are list of
24×20 matrices which are filled out by values in especial order how they measured for 3 parameters in
3000 cycles , how can I feed this type matrice-data or let’s say how can I feed stream of images for 3
different parameters I already extracted from raw dataset and after preprocessing I convert them to
24*20 matrices or .png images ? How should I change this script so that I can use my dataset?

REPLY 
Jason Brownlee December 2, 2018 at 6:26 am #

When using an MLP with images, you must flatten each matrix of pixel data to a single row
vector.

Start Machine Learning

REPLY 
Evangelos Argyropoulos December 18, 2018 at 6:15 am #

Hi Jason,
Thank for tutorial. 1 questions.
I use the algorithm for time series prediction 0=buy 1=sell. Does this model overfit?

REPLY 
Jason Brownlee December 18, 2018 at 6:27 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 148/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

You can only know if you try fitting it and evaluating learning curves on train and validation datasets.

REPLY 
SOURAV MONDAL December 28, 2018 at 7:42 am #

Great tutorial Sir.

Is there a way to visualize different layers with their nodes and interconnections among them, of a model
created in keras (i mean the basic structure of a neural network with layers of nodes and
interconnections among them).

×
REPLY 
Jason Brownlee December 29, 2018 at 5:46 am #
Start Machine Learning
Yes, check out this tutorial:
You can master applied Machine Learning
https://machinelearningmastery.com/visualize-deep-learning-neural-network-model-keras/
without math or fancy degrees.
Find out how in this free and practical course.

Email Address REPLY 

Imen Drs December 28, 2018 at 11:29 pm #

Thanks for this tutorial.

START MY EMAIL COURSE
I have a problem when i try to compile and fit my model. It return value error : ValueError: could not
convert string to float: ’24, 26, 99, 31, 623, 863, 77, 32, 362, 998, 1315, 33, 291, 14123, 39, 8, 335,
2308, 349, 403, 409, 1250, 417, 47, 1945, 50, 188, 51, 4493, 3343, 13419, 6107, 84, 18292, 339, 9655,
22498, 1871, 782, 1276, 2328, 56, 17633, 24004, 24236, 1901, 6112, 22506, 26397, 816, 502, 352,
24238, 18330, 7285, 2160, 220, 511, 17680, 68, 5137, 26398, 875, 542, 354, 2045, 555, 2145, 93, 327,
26399, 3158, 7501, 26400, 8215′ .

Can you help me please.

REPLY 
Jason Brownlee December 29, 2018 at 5:52 am #

Perhaps your data contains a string?

Start Machine Learning

REPLY 
Imen Drs December 29, 2018 at 7:59 am #

The data contains ” user, number_of_followers, list_of_followers, number_of_followee,

list_of_followee, number_of_mentions, list_of_user_mentioned…”
the values in the list are separated by commas.
For example: “36 ; 3 ; 52,3,87 ; 5 ; 63,785,22,11,6 ; 0 ; “

REPLY 
Somashekhar January 2, 2019 at 4:39 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 149/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Hi, Is there a solution posted for solving pima-indians-diabetes.csv for prediction using LSTM?

REPLY 
Jason Brownlee January 2, 2019 at 6:42 am #

No. LSTMs are for sequential data only, and the pima indians dataset is not a sequence
prediction problem.

REPLY 
Imen Drs January 4, 2019 at 9:56 pm #

And thanks.
Is there a way to use specific fields in the dataset instead of the entire uploaded dataset.
Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course. REPLY 
Jason Brownlee January 5, 2019 at 6:56 am #

Yes, fields are columns in the dataset matrix andAddress

Email you can remove those columns that you
do not want to use as inputs to your model.

START MY EMAIL COURSE

REPLY 
Kahina January 5, 2019 at 12:43 am #

Thank you so much ! It’s helpful

REPLY 
Jason Brownlee January 5, 2019 at 6:58 am #

I’m happy to hear that it was helpful.

REPLY 
Khemmarut January 12, 2019 at 11:35 pm #
Start Machine Learning
Traceback (most recent call last):
File “C:/Users/Admin/PycharmProjects/NN/nnt.py”, line 119, in
rounded = [round(X[:1]) for x in predictions]
File “C:/Users/Admin/PycharmProjects/NN/nnt.py”, line 119, in
rounded = [round(X[:1]) for x in predictions]
TypeError: type numpy.ndarray doesn’t define __round__ method

Help me please

Thank you.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 150/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee January 13, 2019 at 5:41 am #

Perhaps ensure that your libraries are up to date?

This might help:

https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-
anaconda/

REPLY 
Priti Pachpande January 31, 2019 at 2:50 am #

Hi Jason,
Thank you for the amazing tutorial. I am trying to buildStart
an autoencoder
Machine modelLearning
in keras using backend ×
tensorflow.
I need to use tensorflow(like tf.ifft,tf.fft) functions in theYou
model. Can you
can master guideMachine
applied me towards how can I do
Learning
it? I tried using lambda layer but the accuracy decreases whenmath
without I useorit.fancy degrees.
Find out how in this free and practical course.
Also, I m using model.predict() function to check the values between the intermediate layers. Am I doing
it right?
Email Address
Also, can you guide me towards how to use reshape function in keras?

Thanks for your help START MY EMAIL COURSE

REPLY 
Jason Brownlee January 31, 2019 at 5:36 am #

Sorry, I don’t know about the functions you are using. Perhaps post on stackoverflow?

REPLY 
Crawford January 31, 2019 at 9:34 pm #

Hi Jason,
Your tutorials are brilliant, thanks for putting all this together.
In this tutorial the result is either a 1 or 0, but what if you have data with more than two possible results,
e.g. 0, 1, 2, or similar?
Can I do something with the code you have presentedStart
here,Machine Learning
or is a whole other approach required?
I have somewhat achieved what I’m trying to do using your “first machine learning project” using a knn
model, but I had to simplify my data by stripping out some variables. I believe there is value in these
extra variables, so thought the neural network might be useful, but like I said I have three classifications
not two.
Thanks.

REPLY 
Jason Brownlee February 1, 2019 at 5:37 am #

Yes, here is an example of a multi-class classification with a neural net:

https://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 151/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Crawford February 1, 2019 at 10:11 pm #

Brilliant, thanks.

REPLY 
Sergio February 1, 2019 at 10:18 am #

Hi, Im trying to construct a neural network using complex number as inputs, I followed your
recommendatins but i get the following warning:
`
Start
ComplexWarning: Casting complex values to real discards the Machine
imaginary partLearning
return array(a, dtype, ×
copy=False, order=order)
You can master applied Machine Learning
The code run without problems, but the predictions is 25 % exact.
without math or fancy degrees.
Find out how in this free and practical course.
Is possible to use complex number in neural networks..?

Do u have some advices?

Email Address

START MY EMAIL COURSE

REPLY 
Jason Brownlee February 1, 2019 at 11:06 am #

I don’t think the Keras API supports complex numbers as input.

REPLY 
Sergio February 1, 2019 at 2:17 pm #

Do u have any suggestion to deal with complex numbers?

REPLY 
Jason Brownlee February 2, 2019 at 6:06 am #

Not off hand, sorry.

Start Machine Learning
Perhaps post to the Keras users group to see if anyone has tried this before:
https://machinelearningmastery.com/get-help-with-keras/

REPLY 
Arnab Kumar Mishra February 1, 2019 at 9:47 pm #

Hi Jason,

I am trying to run the code in the tutorial with some minor modifications, but I am facing a problem with
the training.

The training loss and accuracy both are staying the same across epochs (Please take a look at the code
snippet and the output below). This is for a different dataset, not the diabetes dataset.
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 152/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

I have tried to solve this problem using the suggestions given in

https://stackoverflow.com/questions/37213388/keras-accuracy-does-not-change

But the problem is still there.

Can you please take a look at this and help me solve this problem? Thanks.

CODE and OUTPUT Snippets:

# create model
model = Sequential()
model.add(Dense(15, input_dim=9, activation=’relu’))
model.add(Dense(10, activation=’relu’))
model.add(Dense(5, activation=’relu’))
model.add(Dense(1, activation=’sigmoid’))

# compile model Start Machine Learning ×

model.compile(loss=’binary_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])
You can master applied Machine Learning
# Fit the model without math or fancy degrees.
model.fit(xTrain, yTrain, epochs=500, batch_size=10) Find out how in this free and practical course.

Epoch 1/200
Email Address
81/81 [==============================] – 0s 177us/step – loss: -8.4632 – acc: 0.4691
Epoch 2/200
81/81 [==============================] – 0s 148us/step – loss: -8.4632 – acc: 0.4691
START MY EMAIL COURSE
Epoch 3/200
81/81 [==============================] – 0s 95us/step – loss: -8.4632 – acc: 0.4691
Epoch 4/200
81/81 [==============================] – 0s 116us/step – loss: -8.4632 – acc: 0.4691
Epoch 5/200
81/81 [==============================] – 0s 106us/step – loss: -8.4632 – acc: 0.4691
Epoch 6/200
81/81 [==============================] – 0s 98us/step – loss: -8.4632 – acc: 0.4691
Epoch 7/200
81/81 [==============================] – 0s 145us/step – loss: -8.4632 – acc: 0.4691
Epoch 8/200
81/81 [==============================] – 0s 138us/step – loss: -8.4632 – acc: 0.4691
Epoch 9/200
81/81 [==============================] – 0s 105us/step – loss: -8.4632 – acc: 0.4691
Epoch 10/200 Start Machine Learning
81/81 [==============================] – 0s 128us/step – loss: -8.4632 – acc: 0.4691
Epoch 11/200
81/81 [==============================] – 0s 129us/step – loss: -8.4632 – acc: 0.4691
Epoch 12/200
81/81 [==============================] – 0s 111us/step – loss: -8.4632 – acc: 0.4691
Epoch 13/200
81/81 [==============================] – 0s 106us/step – loss: -8.4632 – acc: 0.4691
Epoch 14/200
81/81 [==============================] – 0s 144us/step – loss: -8.4632 – acc: 0.4691
Epoch 15/200
81/81 [==============================] – 0s 106us/step – loss: -8.4632 – acc: 0.4691
Epoch 16/200

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 153/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

81/81 [==============================] – 0s 180us/step – loss: -8.4632 – acc: 0.4691

Epoch 17/200
81/81 [==============================] – 0s 125us/step – loss: -8.4632 – acc: 0.4691
Epoch 18/200
81/81 [==============================] – 0s 183us/step – loss: -8.4632 – acc: 0.4691
Epoch 19/200
81/81 [==============================] – 0s 149us/step – loss: -8.4632 – acc: 0.4691
Epoch 20/200
81/81 [==============================] – 0s 146us/step – loss: -8.4632 – acc: 0.4691
Epoch 21/200
81/81 [==============================] – 0s 206us/step – loss: -8.4632 – acc: 0.4691
Epoch 22/200
81/81 [==============================] – 0s 135us/step – loss: -8.4632 – acc: 0.4691
Epoch 23/200 Start Machine Learning ×
81/81 [==============================] – 0s 116us/step – loss: -8.4632 – acc: 0.4691
Epoch 24/200 You can master applied Machine Learning
without math
81/81 [==============================] – 0s 135us/step or fancy
– loss: degrees.
-8.4632 – acc: 0.4691
Epoch 25/200 Find out how in this free and practical course.
81/81 [==============================] – 0s 121us/step – loss: -8.4632 – acc: 0.4691
Epoch 26/200 Email Address
81/81 [==============================] – 0s 110us/step – loss: -8.4632 – acc: 0.4691
Epoch 27/200
START MY EMAIL COURSE
81/81 [==============================] – 0s 104us/step – loss: -8.4632 – acc: 0.4691
Epoch 28/200
81/81 [==============================] – 0s 122us/step – loss: -8.4632 – acc: 0.4691
Epoch 29/200
81/81 [==============================] – 0s 117us/step – loss: -8.4632 – acc: 0.4691
Epoch 30/200
81/81 [==============================] – 0s 111us/step – loss: -8.4632 – acc: 0.4691
Epoch 31/200
81/81 [==============================] – 0s 123us/step – loss: -8.4632 – acc: 0.4691
Epoch 32/200
81/81 [==============================] – 0s 116us/step – loss: -8.4632 – acc: 0.4691
Epoch 33/200
81/81 [==============================] – 0s 120us/step – loss: -8.4632 – acc: 0.4691
Epoch 34/200
81/81 [==============================] – 0s 156us/step – loss: -8.4632 – acc: 0.4691
Start Machine Learning
Epoch 35/200
81/81 [==============================] – 0s 131us/step – loss: -8.4632 – acc: 0.4691
Epoch 36/200
81/81 [==============================] – 0s 122us/step – loss: -8.4632 – acc: 0.4691
Epoch 37/200
81/81 [==============================] – 0s 110us/step – loss: -8.4632 – acc: 0.4691
Epoch 38/200
81/81 [==============================] – 0s 121us/step – loss: -8.4632 – acc: 0.4691
Epoch 39/200
81/81 [==============================] – 0s 123us/step – loss: -8.4632 – acc: 0.4691
Epoch 40/200
81/81 [==============================] – 0s 111us/step – loss: -8.4632 – acc: 0.4691

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 154/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Epoch 41/200
81/81 [==============================] – 0s 115us/step – loss: -8.4632 – acc: 0.4691
Epoch 42/200
81/81 [==============================] – 0s 119us/step – loss: -8.4632 – acc: 0.4691
Epoch 43/200
81/81 [==============================] – 0s 115us/step – loss: -8.4632 – acc: 0.4691
Epoch 44/200
81/81 [==============================] – 0s 133us/step – loss: -8.4632 – acc: 0.4691
Epoch 45/200
81/81 [==============================] – 0s 114us/step – loss: -8.4632 – acc: 0.4691
Epoch 46/200
81/81 [==============================] – 0s 112us/step – loss: -8.4632 – acc: 0.4691
Epoch 47/200
Start Machine
81/81 [==============================] – 0s 143us/step Learning
– loss: -8.4632 – acc: 0.4691 ×
Epoch 48/200
81/81 [==============================] – 0s 124us/step – loss:
You can master -8.4632
applied – acc:Learning
Machine 0.4691
Epoch 49/200 without math or fancy degrees.
Find out how
81/81 [==============================] – 0s 129us/step in this-8.4632
– loss: free and–practical course.
acc: 0.4691
Epoch 50/200

The same goes on for the rest of the epochs as well. Email Address

START MY EMAIL COURSE

REPLY 
Jason Brownlee February 2, 2019 at 6:14 am #

I have some suggestions here that might help:

http://machinelearningmastery.com/improve-deep-learning-performance/

REPLY 
Nagesh February 4, 2019 at 1:50 am #

Hi Jason,

Can you please update me, whether we can plot a graph(epoch vs acc)?
If yes then how.

Start Machine Learning

REPLY 
Jason Brownlee February 4, 2019 at 5:49 am #

I show how here:

https://machinelearningmastery.com/display-deep-learning-model-training-history-in-keras/

REPLY 
Nils February 5, 2019 at 1:28 am #

Great stuff, thanks!

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 155/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

I just wondered that in chapter 2 there is a description of the “init” parameter, but in all sources it was
missing.
I added it like:

model.add(Dense(12, input_dim=8, init=’uniform’ ,activation=’relu’))

Then I got this warning:

pima_diabetes.py:25: UserWarning: Update your Dense call to the Keras 2 API: Dense(12,
input_dim=8, activation="relu"
, kernel_initializer="uniform")
model.add(Dense(12, input_dim=8, init=’uniform’ ,activation=’relu’))

Solution for me was to use the “kernel_initializer” instead:

model.add(Dense(12, input_dim=8, activation=”relu”, kernel_initializer=”uniform”))

Regarding the same line I got one question: Is it correct, that it adds one input layer with 8 neurons AND
Start Machine Learning ×
another hidden layer with 12 neurons?
So, would it result in the same ANN to do this? You can master applied Machine Learning
model.add(Dense(8, input_dim=8, kernel_initializer=’uniform’))
without math or fancy degrees.
model.add(Dense(8, activation=”relu”, kernel_initializer=’uniform’))
Find out how in this free and practical course.

Email Address

REPLY 
Jason Brownlee February 5, 2019 at 8:29 am #
START MY EMAIL COURSE
Yes, perhaps your version of the book is out of date, email me to get the latest version?

Yes, the definition of the first hidden layer also defines the input layer via an argument.

REPLY 
Shuja February 8, 2019 at 12:00 am #

Hi Jason
I am getting the following error
(env) shuja@latitude:~$ python keras_test.py
Using TensorFlow backend.
Traceback (most recent call last):
File “keras_test.py”, line 8, in
dataset = numpy.loadtxt(“pima-indians-diabetes.csv”, delimiter=”,”)
Start Machine Learning
File “/home/shuja/env/lib/python3.6/site-packages/numpy/lib/npyio.py”, line 955, in loadtxt
fh = np.lib._datasource.open(fname, ‘rt’, encoding=encoding)
File “/home/shuja/env/lib/python3.6/site-packages/numpy/lib/_datasource.py”, line 266, in open
return ds.open(path, mode, encoding=encoding, newline=newline)
File “/home/shuja/env/lib/python3.6/site-packages/numpy/lib/_datasource.py”, line 624, in open
raise IOError(“%s not found.” % path)
OSError: pima-indians-diabetes.csv not found.

REPLY 
Jason Brownlee February 8, 2019 at 7:52 am #

Looks like the dataset was not downloaded and place in the same directory as your script.
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 156/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Shubham February 12, 2019 at 4:55 am #

Hi, Jason

Thanks for the tutorial.

Do you have some good reference or an example where I can learn about setting up “Adversarial Neural
Networks”.

Shubham

Start
Jason Brownlee February 12, 2019 at 8:08 am # Machine Learning ×
REPLY 

Not at this stage, I hope to cover the topicYou can future.

in the master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Daniel March 13, 2019 at 8:14 am # Email Address

Hey Jason,
START MY EMAIL COURSE
I’ve been reading your tutorials for a while now on a variety of ML topics, and I think that you write very
cleanly and concisely. Thank you for making almost every topic I’ve encountered understandable.

However, one thing I have noticed is that the comment sections on your pages sometimes cover the bulk
of the webpage. The first couple times I saw this site, I saw how tiny my scroll bar was and I assumed
that the tutorial would be 15 pages long, only to find that your introductions were in fact “gentle” as
promised and everything but the first sliver of the page were people’s responses and your responses
back. I think it would be very useful if you could somehow condense the responses (maybe a “show
responses” button?) to only show the actual content. Not only would everything look better, but I think it
would also prevent people from initially thinking your blog was exceptionally long, like I did a few times.

REPLY 
Jason Brownlee March 13, 2019 at 8:26 am #

Great feedback, thanks Daniel. I’ll see if there

Start are some good
Machine wordpress plugins for this.
Learning

REPLY 
ismael March 22, 2019 at 5:22 am #

do not work why

REPLY 
Jason Brownlee March 22, 2019 at 8:39 am #

Sorry to hear that you’re having trouble, what is the problem exactly?

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 157/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Felix Daniel March 30, 2019 at 7:09 am #

Awesome work on machine learning… I was just thinking on how to start my journey into
Machine Learning, I randomly searched for people in Machine Learning on LinkedIn that’s how I find
myself here… I’m delighted to see this… Here is my final bus stop to start building up in ML. Thanks for
accepting my connection on LinkedIn.

I have a project that am about to start but I don’t know how and the road Map. Please I need your
detailed guideline.

Here is the topic

Human Activity Recognition System that Controls overweight in Children and Adults.
Start Machine Learning ×
You can master applied Machine Learning
REPLY 
Jason Brownlee March 31, 2019 at 9:22 am without
# math or fancy degrees.
Find out how in this free and practical course.
Sounds like a great project, you can get started here:
https://machinelearningmastery.com/start-here/#deep_learning_time_series
Email Address

START MY EMAIL COURSE

REPLY 
Akshaya E April 13, 2019 at 11:38 pm #

can you please explain me why we use 12 neurons in the first layer ? 8 are inputs and are the
rest 4 biases ?

REPLY 
Jason Brownlee April 14, 2019 at 5:49 am #

No, the 12 refers to the 12 nodes in the first hidden layer, not the input layer.

The input layer is defined by a input_dim argument on the first hidden layer.

I explain more here:

https://machinelearningmastery.com/faq/single-faq/how-do-you-define-the-input-layer-in-keras

Start Machine Learning

REPLY 
Akshaya E April 14, 2019 at 8:09 pm #

thank you for the immediate response. my doubt has been cleared.

REPLY 
Jason Brownlee April 15, 2019 at 7:52 am #

Happy to hear that.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 158/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Abhiram April 19, 2019 at 11:50 pm #

hii Jason, above predictions are between 0 to 1,My labels are 1,1,1,2,2,2,3,3,3……..36,36,36.
Now i want to predict class 36 then what should i do??

REPLY 
Jason Brownlee April 20, 2019 at 7:39 am #

What problem are you having exactly?

Akash April 22, 2019 at 12:56 am # Start Machine Learning ×

REPLY 

Hi Jason, You can master applied Machine Learning

without math or fancy degrees.
I am learning NLP and facing difficulties with understanding NLP with Deep Learning.
Find out how in this free and practical course.
Please, can you help with converting the following N:N to N:1 model?
I want to change my vec_y from max_input_words_amount length to 1.
How should I define the layers and use LSTM or RNN or Email
…? Address
Thank You.

x=df1[‘Question’].tolist() START MY EMAIL COURSE

y=df1[‘Answer’].tolist()

max_input_words_amount = 0
tok_x = []
for i in range(len(x)) :
tokenized_q = nltk.word_tokenize(re.sub(r”[^a-z0-9]+”, ” “, x[i].lower()))
max_input_words_amount = max(len(tokenized_q), max_input_words_amount)
tok_x.append(tokenized_q)

vec_x=[]
for sent in tok_x:
sentvec = [ft_cbow_model[w] for w in sent]
vec_x.append(sentvec)

vec_y=[]
for sent in y:
Start Machine Learning
sentvec = [ft_cbow_model[sent]]
vec_y.append(sentvec)

for tok_sent in vec_x:

tok_sent[max_input_words_amount-1:]=[]
tok_sent.append(ft_cbow_model[‘_E_’])

for tok_sent in vec_x:

if len(tok_sent)<max_input_words_amount:
for i in range(max_input_words_amount-len(tok_sent)):
tok_sent.append(ft_cbow_model['_E_'])

for tok_sent in vec_y:

tok_sent[max_input_words_amount-1:]=[]
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 159/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

tok_sent.append(ft_cbow_model['_E_'])

for tok_sent in vec_y:

if len(tok_sent)<max_input_words_amount:
for i in range(max_input_words_amount-len(tok_sent)):
tok_sent.append(ft_cbow_model['_E_'])

vec_x=np.array(vec_x,dtype=np.float64)
vec_y=np.array(vec_y,dtype=np.float64)

x_train,x_test, y_train,y_test = train_test_split(vec_x, vec_y, test_size=0.2, random_state=1)

model=Sequential()
model.add(LSTM(output_dim=100,input_shape=x_train.shape[1:],return_sequences=True,
init='glorot_normal', inner_init='glorot_normal', activation='sigmoid'))
model.add(LSTM(output_dim=100,input_shape=x_train.shape[1:],return_sequences=True,
Start Machine Learning ×
init='glorot_normal', inner_init='glorot_normal', activation='sigmoid'))
model.add(LSTM(output_dim=100,input_shape=x_train.shape[1:],return_sequences=True,
You can master applied Machine Learning
init='glorot_normal', inner_init='glorot_normal', activation='sigmoid'))
without math or fancy degrees.
model.add(LSTM(output_dim=100,input_shape=x_train.shape[1:],return_sequences=False,
Find out how in this free and practical course.
init='glorot_normal', inner_init='glorot_normal', activation='sigmoid'))
model.compile(loss='cosine_proximity', optimizer='adam', metrics=['accuracy'])
Email Address
model.fit(x_train, y_train, nb_epoch=100,validation_data=(x_test, y_test),verbose=0)
START MY EMAIL COURSE

REPLY 
Jason Brownlee April 22, 2019 at 6:25 am #

I’m happy to answer questions, but I don’t have the capacity to review your code, sorry.

REPLY 
Charlie April 22, 2019 at 8:41 am #

Jason – I think you are honestly the best teacher of these concepts on the web. Would you do a
graph convolutions post? Maybe working through the concepts in Kipf and Welling 2016 GCN
(https://arxiv.org/abs/1609.02907) paper, and/or (ideally) a worked example applying to a graph network
problem in Keras, maybe using Spektral, the recent graph convolutions Keras library
(https://github.com/danielegrattarola/spektral ) – would HUGELY appreciate it, and with the rise of graph
Start Machine
ML eg per this DeepMind paper (https://arxiv.org/abs/1806.01261) I’mLearning
sure there will be lots of great
applications and interest for people but there’s not much online that’s easy to follow. Thanks so much in
hope.

REPLY 
Jason Brownlee April 22, 2019 at 2:26 pm #

Thanks.

Thanks for the suggestion.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 160/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Kuda April 23, 2019 at 10:01 pm # REPLY 

Hi Jason

Thank you so much for your examples they are crystal clear. Do you have the implementation of RBF
neural network in python?

REPLY 
Jason Brownlee April 24, 2019 at 8:00 am #

Not at this stage, sorry.

Start Machine Learning ×

REPLY 
Tom Cole April 25, 2019 at 5:18 am #
You can master applied Machine Learning
Do you have updated python code for this model on github?
without math orI’m enjoying
fancy degrees.working through the
model but having some difference in the library loads required to dointhe
Find out how thisdata splitting
free and and course.
practical the model
fitting steps.
Thanks
Email Address

START MY EMAIL COURSE

REPLY 
Jason Brownlee April 25, 2019 at 8:26 am #

What problem are you having exactly?

REPLY 
Mridul April 26, 2019 at 3:20 pm #

Hi! Jeson Brownlee,

I try to implement the model in Jupyter notebook.
But when i try to run,an error message show me that “module ‘tensorflow’ has no attribute
‘get_default_graph'” for compiling model = Sequential().I have try lot to overcome it.But couldn’t solve it.
well you please help on this.

Start Machine Learning

REPLY 
Jason Brownlee April 27, 2019 at 6:27 am #

I recommend running code from the command line and not from a notebook, here’s how:
https://machinelearningmastery.com/faq/single-faq/how-do-i-run-a-script-from-the-command-line

REPLY 
Royal May 5, 2019 at 10:18 pm #

Hi Jason,
Super tutorials!

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 161/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

If I run Your First Neural Network once and then repeat several times (without resetting the seed, during
the same python session) using only this code:

model.fit(X, Y, epochs=150, batch_size=10, verbose=0)

scores = model.evaluate(X, Y)
print(“\n%s: %.2f%%” % (model.metrics_names[1], scores[1]*100))

then I get on average a ca. 3% improvement in accuracy (range 77.85% – 83.07%). Apparently the
initialization values are benefitting from the previous runs.
Does it make sense to use a model based on the best fit found after running several times? That would
provide an almost 5% greater accuracy!
Or are we overfitting?

Start Machine Learning ×

REPLY 
Jason Brownlee May 6, 2019 at 6:48 am #
You can master applied Machine Learning
Yes, see this post: without math or fancy degrees.
https://machinelearningmastery.com/faq/single-faq/why-do-i-get-different-results-each-time-i-run-the-
Find out how in this free and practical course.
code

Email Address

REPLY 
Roger May 12, 2019 at 1:53 am # START MY EMAIL COURSE

(base) C:\Users\Roger\Documents\Python Scripts>python firstnn.py

Using Theano backend.
Traceback (most recent call last):
File “firstnn.py”, line 14, in
model.add(Dense(12, input_dim=8, activation=’relu’))
File “C:\Users\Roger\Anaconda3\lib\site-packages\keras\engine\sequential.py”, line 165, in add
layer(x)
File “C:\Users\Roger\Anaconda3\lib\site-packages\keras\engine\base_layer.py”, line 431, in __call__
self.build(unpack_singleton(input_shapes))
File “C:\Users\Roger\Anaconda3\lib\site-packages\keras\layers\core.py”, line 866, in build
constraint=self.kernel_constraint)
File “C:\Users\Roger\Anaconda3\lib\site-packages\keras\legacy\interfaces.py”, line 91, in wrapper
return func(*args, **kwargs)
File “C:\Users\Roger\Anaconda3\lib\site-packages\keras\engine\base_layer.py”,
Start Machine Learning line 249, in add_weight
weight = K.variable(initializer(shape),
File “C:\Users\Roger\Anaconda3\lib\site-packages\keras\initializers.py”, line 218, in __call__
dtype=dtype, seed=self.seed)
File “C:\Users\Roger\Anaconda3\lib\site-packages\keras\backend\theano_backend.py”, line 2600, in
random_uniform
return rng.uniform(shape, low=minval, high=maxval, dtype=dtype)
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\sandbox\rng_mrg.py”, line 872, in uniform
rstates = self.get_substream_rstates(nstreams, dtype)
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\configparser.py”, line 117, in res
return f(*args, **kwargs)
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\sandbox\rng_mrg.py”, line 779, in
get_substream_rstates

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 162/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

multMatVect(rval[0], A1p72, M1, A2p72, M2)

File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\sandbox\rng_mrg.py”, line 62, in multMatVect
[A_sym, s_sym, m_sym, A2_sym, s2_sym, m2_sym], o, profile=False)
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\compile\function.py”, line 317, in function
output_keys=output_keys)
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\compile\pfunc.py”, line 486, in pfunc
output_keys=output_keys)
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\compile\function_module.py”, line 1841, in
orig_function
fn = m.create(defaults)
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\compile\function_module.py”, line 1715, in
create
input_storage=input_storage_lists, storage_map=storage_map)
Start Machine
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\link.py”, Learning
line 699, in make_thunk ×
storage_map=storage_map)[:3]
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\vm.py”, line 1091,
You can master applied in make_all
Machine Learning
impl=impl)) without math or fancy degrees.
Find out how in this
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\op.py”, linefree and
955, in practical
make_thunkcourse.
no_recycling)
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\op.py”,
Email Address line 858, in make_c_thunk
output_storage=node_output_storage)
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\cc.py”, line 1217, in make_thunk
START MY EMAIL COURSE
keep_lock=keep_lock)
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\cc.py”, line 1157, in __compile__
keep_lock=keep_lock)
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\cc.py”, line 1609, in cthunk_factory
key = self.cmodule_key()
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\cc.py”, line 1300, in cmodule_key
c_compiler=self.c_compiler(),
File “C:\Users\Roger\Anaconda3\lib\site-packages\theano\gof\cc.py”, line 1379, in cmodule_key_
np.core.multiarray._get_ndarray_c_version())
AttributeError: (‘The following error happened while compiling the node’, DotModulo(A, s, m, A2, s2, m2),
‘\n’, “module ‘numpy.core.multiarray’ has no attribute ‘_get_ndarray_c_version'”)

REPLY 
Roger May 12, 2019 at 1:58 am #
Start Machine Learning
I followed all the steps to set up the environment but when I ran the code I got an attribute
error ‘module ‘numpy.core.multiarray’ has no attribute ‘_get_ndarray_c_version”

REPLY 
Jason Brownlee May 12, 2019 at 6:45 am #

Perhaps try searching/posting on stackoverflow?

REPLY 
Jason Brownlee May 12, 2019 at 6:45 am #
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 163/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Ouch, perhaps numpy is not installed correctly?

REPLY 
Roger May 12, 2019 at 8:34 pm #

No numpy 1.16.2 does not work with theano 1.0.3 as served up currently by Anaconda. I
downgraded to numpy 1.13.0.

REPLY 
Jason Brownlee May 13, 2019 at 6:46 am #

Thanks Roger.
Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees. REPLY 
Aditya May 21, 2019 at 5:02 pm #
Find out how in this free and practical course.
Hi Jason,
Thanks for this amazing example! Email Address
What I observe in the example is the database used is purely numeric.
My doubt is:
START MY EMAIL COURSE
How can the example be modified to handle categorical input?
Will it work if the inputs are One Hot Encoded?

REPLY 
Jason Brownlee May 22, 2019 at 7:38 am #

Yes, you can use a one hot encoding for our input categorical variables.

REPLY 
Aditya May 31, 2019 at 3:41 pm #

Can you please provide a good reference point for OHE in python?
Thanks in advance!
Start Machine Learning

REPLY 
Jason Brownlee June 1, 2019 at 6:09 am #

Sure:
https://machinelearningmastery.com/why-one-hot-encode-data-in-machine-learning/

Aditya June 2, 2019 at 3:36 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 164/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

I read the link and it was helpful. Now, I have a doubt specific to my network.
I have 3 categorical input which have different sizes. One has around 15 ‘categories’
while the other two have 5. So after I One Hot encode each of them, do I have to make
their sizes same by padding? Or it’ll work as it it?

Jason Brownlee June 2, 2019 at 6:42 am #

You can encode each variable and concatenate them together into one vector.

Or you can have a model with one input for each variable and let the model concatenate
them.

Start Machine Learning ×

REPLY 
Sri June 17, 2019 at 7:29 pm # You can master applied Machine Learning
without math or fancy degrees.
Hi, Find out how in this free and practical course.

If there is one independent variable (say country) with more than 100 labels, how to resolve it.
Email Address
I think only one hot encoding will not work including scaling.

Is there any alternative for it

START MY EMAIL COURSE

REPLY 
Jason Brownlee June 18, 2019 at 6:37 am #

You can try:

– integer encoding
– one hot encoding
– embedding

Test each and see what works best for your specific dataset.

REPLY 
MK June 21, 2019 at 7:05 pm #
Start Machine Learning
Hi jason,

thanks a lot for your posts, helped me a lot.

1. How can I add confusion matrix?

2. How can I change learning rate?

Cheers Martin

REPLY 
Jason Brownlee June 22, 2019 at 6:35 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 165/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Add a confusion matrix:

https://machinelearningmastery.com/custom-metrics-deep-learning-keras-python/

Tune learning rate:

https://machinelearningmastery.com/understand-the-dynamics-of-learning-rate-on-deep-learning-
neural-networks/

REPLY 
Guhan palanivel July 1, 2019 at 10:35 pm #

hi jason,
I have trained a neural network model with 6 months data and deployed at a remote site ,
when receiving the new data for upcoming months ,
Start
is there any way to automatically update the model with Machine
addition Learning
of new training data ? ×
You can master applied Machine Learning
without math or fancy degrees.
Jason Brownlee July 2, 2019 at 7:31 am # Find out how in this free and practical course. REPLY 

Yes, perhaps the easiest way is to refit theEmail

modelAddress
on the new data or on all available data.

START MY EMAIL COURSE

REPLY 
Shubham July 5, 2019 at 8:46 pm #

Hi jason,

I want to print the neural network score as a function of one of the variable., how do i do that?

Regards
Shubham

REPLY 
Jason Brownlee July 6, 2019 at 8:35 am #

Perhaps try a linear activation unit and a mse loss function?

Start Machine Learning

REPLY 
Maha Lakshmi July 17, 2019 at 7:37 pm #

Sir, I am working with sklearn.neural_network.MLPClassifier in Python. now I want to give my

own Initial Weights to Classifier.how to do that? please help me. Thanks in Advance

REPLY 
Jason Brownlee July 18, 2019 at 8:25 am #

Sorry, I don’t have an example of this.

Perhaps try posting on stackoverflow?

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 166/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Maha Lakshmi July 18, 2019 at 4:09 pm #

Thank you for your response

REPLY 
Ron July 24, 2019 at 8:39 am #

Normalization of the data increases the accuracy in the 90’s.

https://stackoverflow.com/questions/39525358/neural-network-accuracy-optimization

Start Machine Learning ×

REPLY 
Jason Brownlee July 24, 2019 at 2:19 pm #
You can master applied Machine Learning
without math or fancy degrees.
Thanks for sharing.
Find out how in this free and practical course.

Email Address
REPLY 
Hammad July 29, 2019 at 6:12 pm #
START MY EMAIL COURSE
Dear sir,

I would like to apply above shared example on arrays produced by “train_test_split” but it does not work,
as these arrays are not in the form of numpy.

Let me give you the details, I have “XYZ” dataset. The dataset has the following specifications:

Total Images = 630

2500 features has been extracted from each image. Each feature has float type.
Total Classes = 7

Now, after processing the feature file, I have got results in the following variables:

XData: contains features data in two dimensional array form (rows: 630, columns: 2500)
YData: contain original labels of classes in one dimensional array form (rows: 630, column: 1)

So, by using the following code, I split the data set into train and testing data:

from sklearn.model_selection import train_test_split Start Machine Learning

x_train, x_test, y_train, y_test = train_test_split(XData, YData, stratify=YData, test_size=0.25)

Now, I would like to apply the deep-learning examples shared on this blog on my dataset which is now in
the form arrays, and generate output as prediction of testing data and accuracy.

Can you please let me know about it, which can work on the above arrays?

REPLY 
Jason Brownlee July 30, 2019 at 6:05 am #

Yes, the Keras model can operate on numpy arrays directly.

Perhaps I don’t follow the problem that you’re having exactly?

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 167/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Hammad July 30, 2019 at 6:01 pm #

Dear sir,

Thanks, I converted my arrays into numpy format.

Now, I have followed your tutorial on multi-classification problem

(https://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-
library/) and use the following code:

############################################################
import pandas
from keras.models import Sequential
Start Machine Learning ×
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasClassifier
You can master applied Machine Learning
from keras.utils import np_utils without math or fancy degrees.
from sklearn.model_selection import cross_val_score
Find out how in this free and practical course.
from sklearn.model_selection import KFold
from sklearn.preprocessing import LabelEncoder
Email Address
from sklearn.pipeline import Pipeline
from sklearn.metrics import accuracy_score
START MY EMAIL COURSE
seed=5
totalclasses=7 # Class Labels are: ‘p1’, ‘p2’, ‘p3’, ‘p4’, ‘p5’, ‘p6’, ‘p7′
totalimages=630
totalfeatures=2500 #features generated from images

# Data has been imported from feature file, which results two arrays XData and YData
# XData contains features dataset without numpy array form
# YData contains labels without numpy array form

# encode class values as integers

encoder = LabelEncoder()
encoder.fit(YData)
encoded_Y = encoder.transform(YData)
# convert integers to dummy variables (i.e. one hot encoded)
dummy_y = np_utils.to_categorical(encoded_Y)

# define baseline model Start Machine Learning

def baseline_model():
# create model
model = Sequential()
model.add(Dense(8, input_dim=totalfeatures+1, activation=’relu’))
model.add(Dense(totalclasses, activation=’softmax’))
# Compile model
model.compile(loss=’categorical_crossentropy’, optimizer=’adam’, metrics=
‘accuracy’])
return model

estimator = KerasClassifier(build_fn=baseline_model, nb_epoch=200, batch_size=5, verbose=0)

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 168/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

x_train, x_test, y_train, y_test = train_test_split(XData, dummy_y, test_size=0.25,

random_state=seed)

x_train = np.array(x_train)
x_test = np.array(x_test)
y_train = np.array(y_train)
y_test = np.array(y_test)

estimator.fit(x_train, y_train)
predictions = estimator.predict(x_test)

print(predictions)
print(encoder.inverse_transform(predictions))

########################################################

The code generates no syntax error. Start Machine Learning ×

Now, I would like to ask: You can master applied Machine Learning
without math or fancy degrees.
1. Does I have applied the deep learning (Neural Network Model) in a right way?
Find out how in this free and practical course.
2. How could I calculate the accuracy, confusion matrix, and classification_report?
3. Can you please suggest what other type of deep learning algorithms could I apply on this type
of problem? Email Address

After applying different deep learning algorithm, I would like to compare their accuracies such as,
START MY EMAIL COURSE
you did in tutorial https://machinelearningmastery.com/machine-learning-in-python-step-by-step/,
by plotting graphs.

REPLY 
Jason Brownlee July 31, 2019 at 6:46 am #

Sorry, I don’t have the capacity to review your code.

This post shows how to calculate metrics:

https://machinelearningmastery.com/how-to-calculate-precision-recall-f1-and-more-for-deep-
learning-models/

I recommend testing a suite of methods in order to discover what works best for your
specific dataset:
https://machinelearningmastery.com/faq/single-faq/what-algorithm-config-should-i-use
Start Machine Learning

REPLY 
Tyson September 3, 2019 at 10:07 pm #

Hi Jason,
Great tutorial. I am now trying new data sets from the UCI archive. However I am running into problems
when the data is incomplete. Rather than a number there is a ‘?’ indicating that the data is missing or
unknown. So I am getting
ValueError: could not convert string to float: ‘?’

Is there a way to ignore that data? I am sure many data sets have this issue where pieces are missing.

Thanks in advance!

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 169/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee September 4, 2019 at 5:58 am #

Yes, you can replace missing data with the mean or median of the variable – at least as a
starting point.

REPLY 
Srinu September 10, 2019 at 9:07 pm #

Can you provide GUI code for the same data like calling the ANN model from a website or from
android application.

Start Machine Learning ×

You can master applied Machine Learning
REPLY 
Jason Brownlee September 11, 2019 at 5:33without
am #
math or fancy degrees.
Find out how in this free and practical course.
I don’t see why not.

Email Address

REPLY 
Hemanth Kumar September 20, 2019 at 12:58 pm #
START MY EMAIL COURSE

dear sir
ValueError: Error when checking input: expected conv2d_5_input to have 4 dimensions, but got array
with shape (250, 250, 3)
I am getting this error

what steps I did

original_image->resized to same resolution->converted to numpy array ->saved and loaded to x_train ->
fed into network model ->modal.fit(x_train .. getting this error

REPLY 
Jason Brownlee September 20, 2019 at 1:42 pm #

Perhaps start with this tutorial for image classification:

https://machinelearningmastery.com/how-to-develop-a-convolutional-neural-network-to-classify-
Start Machine Learning
photos-of-dogs-and-cats/

REPLY 
Hemanth Kumar September 20, 2019 at 3:14 pm #

thanks for response sir

after that I am getting list index out of range error at model.fit

REPLY 
Jason Brownlee September 21, 2019 at 6:43 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 170/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

I’m sorry to hear that, I have some suggestions here that may help:
https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-
me

REPLY 
Anthony The Koala September 26, 2019 at 2:58 am #

Dear Dr Jason,
Thank you for this tutorial.
I have been playing around with the number of layers and the number of neurons.
In the current code

1 model = Sequential()
2
3
model.add(Dense(12, input_dim=8, activation='relu'))
model.add(Dense(8, activation='relu')) Start Machine Learning ×
4 model.add(Dense(1, activation='sigmoid'))
You can master applied Machine Learning
I have played around with increasing the numbers in the first layer:
without math or fancy degrees.
Find out how in this free and practical course.
1 model = Sequential()
2 model.add(Dense(100, input_dim=8, activation='relu'))
3 model.add(Dense(8, activation='relu'))
4 model.add(Dense(1, activation='sigmoid')) Email Address

The result is that the accuracy didn’t improve much.

There was an improvement in the addition of layers. START MY EMAIL COURSE
When each layer had say a large number of neurons, the accuracy improved.
This is not the only example, but playing around with the following code:

1 model = Sequential()
2 model.add(Dense(200, input_dim=8, activation='relu'))
3 model.add(Dense(800, activation='relu'))
4 model.add(Dense(200, activation='relu'))
5 model.add(Dense(400, activation='relu'))
6 model.add(Dense(200, activation='relu'))
7 model.add(Dense(1, activation='sigmoid'))

The accuracy achieved was 91.1%

I added two more layers

1 model = Sequential()
2 model.add(Dense(200, input_dim=8, activation='relu'))
3 model.add(Dense(800, activation='relu'))
4 model.add(Dense(200, activation='relu'))
5 model.add(Dense(400, activation='relu')) Start Machine Learning
6 model.add(Dense(200, activation='relu'))
7 model.add(Dense(400, activation='relu'))
8 model.add(Dense(800, activation='relu'))
9 model.add(Dense(1, activation='sigmoid'))

The accuracy dropped slightly to 88%

From these brief experiments, increasing the number of neurons as in your first example did not increase
accuracy.
However adding more layers especially with a large number of neurons did increase the accuracy to
about 91%
BUT if there are too many layers there is a slight drop in accuracy to 88%.

My question is there a way to increase the accuracy any further than 91%?

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 171/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Thank you,
Anthony of Sydney

REPLY 
Jason Brownlee September 26, 2019 at 6:45 am #

If this is the pima indians dataset, then the best accuracy is about 78% via 10-fold cross
validation, anything more is probably overfitting.

Yes, I have tons of tutorials on diagnosing issues with models and lifting performance, you can start
here:
https://machinelearningmastery.com/start-here/#better

Start Machine Learning ×

REPLY 
Anthony The Koala September 26, 2019 at 6:05 You
am #can master applied Machine Learning
without math or fancy degrees.
Dear Dr Jason, Find out how in this free and practical course.
Further experimentation, I played with the following code

1 model = Sequential() Email Address

2 model.add(Dense(25, input_dim=8, activation='relu'))
3 model.add(Dense(89, activation='relu'))
4 model.add(Dense(377, activation='relu')) START MY EMAIL COURSE
5 model.add(Dense(233, activation='relu'))
6 model.add(Dense(55, activation='relu'))
7 model.add(Dense(1, activation='sigmoid'))

I obtained an accuracy of 95% by playing around with the number of neurons increasing then
decreasing.
I cannot work out a systematic way of improving the accuracy.

Thank you,
Anthony of Sydney

REPLY 
Jason Brownlee September 26, 2019 at 6:46 am #

Haha, yes. That is the great open problem with neural nets (no good theories for how to
configure them) and why we must use empirical methods.
Start Machine Learning

REPLY 
Anthony The Koala September 26, 2019 at 1:57 pm #

Dear Dr Jason,
thank you for those replies.

Yes, it was the Pima Indian dataset that is covered in this tutorial.

Before I indulge in further readings on 10-fold cross validation, please briefly answer:
* what is the meaning of overfit.
* why is an accuracy of 96% regarded as overfit.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 172/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

To do:
Play around with simple functions and play around with this tutorial and then look at overfitting:
For example suppose we have x = 0, 1, 2, 3, 4, 5 and f(x) = x^2

1 x : 0, 1, 2 , 3, 4, 5
2 f(x) : 0, 1, 4, 9, 16, 25

The aim:
* to see if there is an accurate mapping of the function of x and f(x) for x = 0..5
* to see what happens when we predict for x = 6, 7, 8. Will it be 36, 49, 64?
* we ask if there is such a thing as overfitting the model exists.

Thank you,
Anthony of Sydney

Start Machine Learning ×

REPLY 
Jason Brownlee September 27, 2019 at 7:43You
am #can master applied Machine Learning
without math or fancy degrees.
Overfit means better performance on the Find
training set at
out how in the
this cost of performing
free and worse on the
practical course.
test set.

Emailset
It can also mean better performance on a test/validation Address
at the cost of worse performance on
new data.

I know from experience that the limit on that datasetSTART MY EMAIL

is 77-78% COURSE
after having worked with it in tutorials
for about 20 years.

REPLY 
Andrey September 29, 2019 at 8:32 pm #

Hi Jason,

I see the data is not divided for that of training and for the test. Why is that? What does prediction mean
in this case?

Andrey

REPLY 
Jason Brownlee September 30, 2019 at 6:07Start
am # Machine Learning

It might mean that the result is a little optimistic.

I did that to keep this example very simple and easy to follow.

REPLY 
Anthony The Koala September 29, 2019 at 9:00 pm #

Dear Dr Jason,
I tried to do the same for a deterministic model of x and fx where x = [0,1,2,3,4,5] and fx = x**2
I want to see how machine learning operates with a deterministic function.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 173/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

However I am only getting 16.67% accuracy.

Here is the code based on the this tutorial

1 from keras.models import Sequential

2 from keras.layers import Dense
3 import numpy as np
4
5 #Aim is to see how a deterministic function will operate using machine learning
6 #In year 7 algebra we have x and y. y is known as f(x).
7
8 #So here we aim to have a structore of [indep var, dep var]
9 #that is [x, fx]
10
11 #Making a 2D (like) list
12 x = [i for i in range(6)]; # have a list of x = [0,1,2,3,4,5]
13 #x = np.array(x)
14
15
16
fx = [x**2 for x in x]; # have a list of fx = [0,1,4,9,16,25]
#fx = np.array(fx) Start Machine Learning ×
17
18 model = Sequential()
You can master applied Machine Learning
19 model.add(Dense(100, input_dim=1,activation='relu'))
20 model.add(Dense(200, activation='relu')) without math or fancy degrees.
21 model.add(Dense(1, activation='relu')) Find out how in this free and practical course.
22
23
24 model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
Email Address
25 model.fit(x,fx,epochs=150, batch_size=2,verbose=0)
26 _,accuracy = model.evaluate(x,fx)
27 print('Accuracy: %.2f' % (accuracy*100))
START MY EMAIL COURSE
We know that fx = x**2 is predictable. What do I need to do.

Thank you,
Anthony of Sydney

REPLY 
Jason Brownlee September 30, 2019 at 6:10 am #

Perhaps you need hundreds of thousands of examples?

And perhaps the model will need to be tuned for your problem, e.g. perhaps using mse loss and a
linear activation function in the output layer because it is a regression problem.

Start Machine Learning REPLY 

Anthony The Koala October 1, 2019 at 5:15 am #

Dear Dr Jason,
I tried with mse-loss and linear activation function and still only obtained 1% accuracy.

1 from keras.models import Sequential

2 from keras.layers import Dense
3 import numpy as np
4
5 #Aim is to see how a deterministic function will operate using machine learning
6 #In year 7 algebra we have x and y. y is known as f(x).
7
8 x = [i for i in range(100)]; # have a list of x = [0,1,2,3,4,5]
9 x = np.array(x)
10
11 fx = [x**2 for x in x]; # have a list of fx = [0,1,4,9,16,25]

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 174/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

12 fx = np.array(fx)
13
14 model = Sequential()
15 model.add(Dense(12, input_dim=1,activation='linear'))
16 model.add(Dense(33, activation='linear'))
17 model.add(Dense(1, activation='linear'))
18
19
20 #model.compile(loss='mean_squared_error', optimizer='softmax', metrics=['accuracy'])
21 model.compile(loss='mean_squared_error', optimizer='sgd')
22 #model.compile(loss='mean_squared_error')
23
24 model.fit(x,fx,epochs=10, batch_size=1,verbose=0)
25 _,accuracy = model.evaluate(x,fx)
26 print('Accuracy: %.2f' % (accuracy*100))

However I get this:

1 32/100 [========>.....................] - ETA: 0s

2 100/100 [==============================] - 0s 312us/step Start Machine Learning ×
3 Traceback (most recent call last):
4 File "C:\Python36\deterministicII.py", line
You25, in
can master applied Machine Learning
5 _,accuracy = model.evaluate(x,fx) without math or fancy degrees.
6 TypeError: 'float' object is not iterable
Find out how in this free and practical course.
I want to map a deterministic function to see if machine learning will work out f(x) without the formula.
Email Address

REPLY 
Jason Brownlee October 1, 2019 at 7:00 am # START MY EMAIL COURSE

Accuracy is not a valid metric for regression problems:

https://machinelearningmastery.com/faq/single-faq/how-do-i-calculate-accuracy-for-regression

You are very close!

Also, try a much larger dataset of examples. Hundreds or thousands.

REPLY 
Anthony The Koala October 1, 2019 at 10:18 am #

Dear Dr Jason,
I removed the model.evaluate from the program. BUT still I have not got a satisfactory match of the
expected and actual values.

1 from keras.models import Sequential Start Machine Learning

2 from keras.layers import Dense
3 import numpy as np
4
5 #Aim is to see how a deterministic function will operate using machine learning
6 #In year 7 algebra we have x and y. y is known as f(x).
7
8 x = [i for i in range(100)]; # have a list of x = [0,1,2,3,4,5]
9 x = np.array(x)
10
11 fx = [x**2 for x in x]; # have a list of fx = [0,1,4,9,16,25]
12 fx = np.array(fx)
13
14 model = Sequential()
15 model.add(Dense(100, input_dim=1,activation='linear'))
16 model.add(Dense(100, activation='linear'))
17 model.add(Dense(1, activation='linear'))
18 model.compile(loss='mean_squared_error', optimizer='adam')
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 175/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step
19
20 model.fit(x,fx,epochs=1000, batch_size=1000,verbose=0)
21
22 #Removing the model.evaluate code
23
24 predictions = model.predict_classes(x)
25 for i in range(6):
26 print('%s => %d (expected %d)' % (x[i],predictions[i],fx[i]))

Output

1 0 => 0 (expected 0)
2 1 => 0 (expected 1)
3 2 => 0 (expected 4)
4 3 => 0 (expected 9)
5 4 => 1 (expected 16)
6 5 => 1 (expected 25)

Not yet getting a match of the expected and the actual values
Start Machine Learning ×
Thank you,
You can master applied Machine Learning
Anthony of Sydney
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Jason Brownlee October 1, 2019 at 2:17 pm #Email Address

Perhaps the model architecture (layers and nodes) needs tuning?

START MY EMAIL COURSE
Perhaps the learning rate needs tuning?
Perhaps you need more training examples?
Perhaps you need more or fewer epochs?
…

More ideas here:

https://machinelearningmastery.com/start-here/#better

REPLY 
Anthony The Koala October 2, 2019 at 7:41 am #

Dear Dr Jason,
I cannot find a systematic way to find a way for a machine learning algorithm to use it to compute a
deterministic equation such as y = f(x) where f(x) = x**2.

StartEssentially
I am still having trouble. I will be posting this on the page. Machine Learning
is (i) adding/dropping layers, (ii)
adjusting the number of epochs, (iii) adjusting the batch_size. But I haven’t come close yet.

Also using the function model.predict rather than model.predict_classes.

Here is the program with most of the commented out lines deleted.

1 from keras.models import Sequential

2 from keras.layers import Dense
3 import numpy as np
4
5 #Aim is to see how a deterministic function will operate using machine learning
6 #In year 7 algebra we have x and y. y is known as f(x). Here y = f(x) = x**2
7
8 x = [i for i in range(100)]; # have a list of x = [0,1,2,3,4,5,.....,99]
9 x = np.array(x)
10

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 176/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step
11 fx = [x**2 for x in x]; # have a list of fx = x**2 = [0,1,4,9,16,25,...,9801]
12 fx = np.array(fx)
13
14 model = Sequential()
15 model.add(Dense(55, input_dim=1,activation='linear'))
16 model.add(Dense(34, activation='linear'))
17 model.add(Dense(21, activation='linear'))
18 model.add(Dense(13, activation='linear'))
19 model.add(Dense(1, activation='linear'))
20
21 model.compile(loss='mean_squared_error', optimizer='adam')
22
23 model.fit(x,fx,epochs=89, batch_size=144,verbose=0)
24
25 predictions = model.predict(x); #This seems to work instead of model.predict_classes
26
27 print("x, predicted, expected")
28 for i in range(6):
29 print('%s => %d (expected %d)' % (x[i],predictions[i],fx[i]))
Start Machine Learning ×
The output is:
You can master applied Machine Learning
1 x, predicted, expected
without math or fancy degrees.
2 0 => 29 (expected 0)
3 1 => 110 (expected 1) Find out how in this free and practical course.
4 2 => 191 (expected 4)
5 3 => 272 (expected 9)
6 4 => 353 (expected 16) Email Address
7 5 => 434 (expected 25)

No matter how much I adjust the number of neurons perSTARTlayer, MY

the EMAIL
number of layers, the no of epochs
COURSE
and the batch size, the “predicted” appears like an arithmetic progression, not a geometric progression.

Note the terms tn+1 – tn is 81 for all the predicted values in the machine learning model.

BUT we know that the difference between successive terms in y = f(x) is not the same.

For example, in non linear relation such as f(x) = x**2, f(x) = 0, 1, 2, 4, 9, 16, 25, 36, the difference
between the terms is: 1, 1, 2, 5, 7, 9, 11, that is tn+1 – tn != tn+2 – tn+1.

So still having trouble working out how to get a machine learning algorithm evaluate f(x) without the
formula.

REPLY 
Jason Brownlee October 2, 2019 at 8:15 am #

Here is the solution, hope it helps

Start Machine Learning
1 # fit an mlp on x vs x^2
2 from sklearn.preprocessing import MinMaxScaler
3 from keras.models import Sequential
4 from keras.layers import Dense
5 from numpy import asarray
6 from matplotlib import pyplot
7 # define data
8 x = asarray([i for i in range(1000)])
9 y = asarray([a**2 for a in x])
10 # reshape into rows and cols
11 x = x.reshape((len(x), 1))
12 y = y.reshape((len(y), 1))
13 # scale data
14 x_s = MinMaxScaler()
15 x = x_s.fit_transform(x)
16 y_s = MinMaxScaler()
17 y = y_s.fit_transform(y)
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 177/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

18 # fit a model
19 model = Sequential()
20 model.add(Dense(10, input_dim=1, activation='relu'))
21 model.add(Dense(1))
22 model.compile(loss='mse', optimizer='adam')
23 model.fit(x, y, epochs=150, batch_size=10, verbose=0)
24 mse = model.evaluate(x, y, verbose=0)
25 print(mse)
26 # predict
27 yhat = model.predict(x)
28 # plot real vs predicted
29 pyplot.plot(x,y,label='y')
30 pyplot.plot(x,yhat,label='yhat')
31 pyplot.legend()
32 pyplot.show()

I guess you could also do an inverse_transform() on the predicted values to get back to original
units.
Start Machine Learning ×
You can master applied Machine Learning

Anthony The Koala October 2, 2019 at 9:05 am without

#
math or fancy degrees. REPLY 

Find out how in this free and practical course.

Dear Dr Jason,
Thank you very much for your reply. I got an mse in theEmail
order Address
of 3 x 10**-6.

Despite this, I will be studying the program and learn myself about (i) the MinMaxScaler and why we use
it, (ii) fit_transform(y) and (iii) one hidden layer of 10 neurons,
STARTand (iii) I will
MY EMAIL still have to learn about the
COURSE
choice of activation function and loss functions. The keras website has a section on loss functions at
https://keras.io/losses/ but having a look at the Python “IDLE” program, a look at from keras import
losses, there are many more loss functions which are necessary to compile a model.

In addition, the predicted values will have to be re-computed to its unscaled values. So I will also look up
‘rescaling’.

Thank you again,

Anthony, Sydney NSW

REPLY 
Jason Brownlee October 2, 2019 at 10:10 am #

Yes, you can use inverse_transform to unscale the predictions, as I mentioned.

Start Machine Learning

REPLY 
Anthony The Koala October 3, 2019 at 6:26 am #

Dear Dr Jason,
I know how to use the inverse_transform function:
First apply the MinMaxScaler to scale to 0 to 1

1 x_s = MinMaxScaler()
2 x = x_s.fit_transform(x)
3 y_s = MinMaxScaler()
4 y = y_s.fit_transform(y)

If we want to reconstitute x and y, it is simple to:

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 178/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step
1 x_original = x_s.inverse_transform(x); # where x was transformed/scaled
2 y_original = y_s.inverse_transform(y);# where y was transformed/scaled

x_s and y_s has the min and max values stored of the original pre-transformed data.

BUT how do you transform yhat to its original scale when it was not subject to the inverse_transform
function.

If I relied on the y_s.inverse_transform(yhat), where you get this:

1 yhat_restored = y_s.inverse_transform(yhat); #using the values of ymin and ymax of ori

2 yhat_restored[0:10]
3 array([[6838.43],
4 [6838.43],
5 [6838.43],
6 [6838.43],
7 [6838.43],
8
9
[6838.43],
[6838.43], Start Machine Learning ×
10 [6838.43],
11 [6838.43],
You can master applied Machine Learning
12 [6838.43]], dtype=float32)
without math or fancy degrees.
I was ‘hoping’ for something close to the original: Find out how in this free and practical course.

1 >>> y_restored = y_s.inverse_transform(y)

2 >>> y_restored[0:10] Email Address
3 array([[ 0.],
4 [ 1.],
5 [ 4.],
6 [ 9.], START MY EMAIL COURSE
7 [16.],
8 [25.],
9 [36.],
10 [49.],
11 [64.],
12 [81.]])

BUT yhat does not use the MinMaxScaler at the start.

Do I have to rewrite my own function?

Thanks,
Anthony of Sydney NSW

REPLY 
Jason Brownlee October 3, 2019 at 6:54 am #
Start Machine Learning
The model predicts scaled values, apply the inverse transform on yhat directly.

REPLY 
Anthony The Koala October 3, 2019 at 2:39 pm #

Dear Dr Jason,
I did that apply the inverse transform of yhat directly, BUT GOT these
Cut down version of code

1 y_s = MinMaxScaler()
2 y = y_s.fit_transform(y); #y_s stores the min and max values according to the sklearn
3 #Note the above is for y. WE DON'T KNOW yhat(min
4

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 179/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step
5
6 yhat = model.predict(x); #we have the scaled estimate.
7
8
9 x_original = x_s.inverse_transform(x); #this printed okay
10
11 #Printout of yhat transformed
12 #Calculate yhat scaled using the min and max values of f(x) = y
13
14 yhat_restored = y_s.inverse_transform(yhat)
15
16 #Print yhat
17 print(yhat_restored[0:10])
18 array([[6838.43],
19 [6838.43],
20 [6838.43],
21 [6838.43],
22 [6838.43],
23
24
[6838.43],
[6838.43], Start Machine Learning ×
25 [6838.43],
26 [6838.43], You can master applied Machine Learning
27 [6838.43]], dtype=float32)
without math or fancy degrees.
Findwhen
Don’t understand how to get an inverse transform of yhat out how in this
I don’t free the
know and‘untransformed’
practical course. value
because I have not estimated it.

Thank you, Email Address

Anthony of Sydney
START MY EMAIL COURSE

REPLY 
Jason Brownlee October 4, 2019 at 5:39 am #

You can inverse transform y and yhat and plot both.

REPLY 
Anthony The Koala October 4, 2019 at 3:20 am #

Dear Dr Jason,
I tried it again to illustrate that despite the predicted fitting a parabola for scaled predicted and expected
values of f(x) the resulting values when ‘unscaled’ back to the original does seems quite absurd.
Code – relevant

1 # plot real vs predicted Start Machine Learning

2 pyplot.plot(x,y,label='y')
3 pyplot.plot(x,yhat,label='yhat')
4 pyplot.legend()
5 print("The graph of the (x, predicted f(x) and (x, f(x) is on a separate window")
6 pyplot.show()
7 y_predicted = y_s.inverse_transform(yhat)
8 y_expected = y_s.inverse_transform(y)
9
10 x_original = x_s.inverse_transform(x)
11 #print(y_predicted[0:10,].tolist(), x_original[0:10,].tolist())
12 print("Printing the first 10, predicted, expected, and x")
13 for i in range(10):
14 print(y_predicted[i], y_expected[i], x_original[i])
15
16
17 print("let's try some other arbitrary section, say 10:20")
18 #print(y_predicted[9:21,].tolist(),y_predicted[9:21,].tolist(), x_original[9:21,].toli

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 180/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

19 print("printing 10th to 20th, predicted, expected, and x")

20 for i in range(10):
21 print(y_predicted[i+10],y_expected[i+10], x_original[i+10])

The resulting output:

1 5.472537487406726e-06
2 Printing the first 10, predicted, expected, and x
3 [1030.0833] [0.] [0.]
4 [1030.0833] [1.] [1.]
5 [1030.0833] [4.] [2.]
6 [1030.0833] [9.] [3.]
7 [1030.0833] [16.] [4.]
8 [1030.0833] [25.] [5.]
9 [1030.0833] [36.] [6.]
10 [1030.0833] [49.] [7.]
11 [1030.0833] [64.] [8.]
12 [1030.0833] [81.] [9.]
13
14
let's try some other arbitrary section, say 10:20
printing 10th to 20th, predicted, expected, and x Start Machine Learning ×
15 [1030.0833] [100.] [10.]
16 [1030.0833] [121.] [11.] You can master applied Machine Learning
17 [1030.0833] [144.] [12.] without math or fancy degrees.
18 [1030.0833] [169.] [13.]
19 [1030.0833] [196.] [14.] Find out how in this free and practical course.
20 [1030.0833] [225.] [15.]
21 [1030.0833] [256.] [16.]
22 [1030.0833] [289.] [17.] Email Address
23 [1030.0833] [324.] [18.]
24 [1030.0833] [361.] [19.]
START MY EMAIL COURSE
When I plotted (x, yhat) and (x,f(x)), the plot was as expected. BUT when I rescaled the yhat back, all the
values of unscaled yhat were 1030.0833 which is quite odd.

Why?

Thank you,
Anthony of Sydney NSW

REPLY 
Anthony The Koala October 4, 2019 at 3:31 am #

Dear Dr Jason,
I printed the yhat, and they were all the same.

This is despite that the plot of the scaled values (x, yhat) looked like a parabola
Note: this is prior to scaling.
Start Machine Learning
1 # plot real vs predicted
2 pyplot.plot(x,y,label='y')
3 pyplot.plot(x,yhat,label='yhat')
4 pyplot.legend()
5 print("the graph is printed on another window")
6 pyplot.show()
7
8 print("Printing the output of the scaled values of yhat, f(x) and x")
9 print("printing the first 10")
10 for i in range(10):
11 print(yhat[i],y[i],x[i])
12 print("printing the 10th to 20th")
13 for i in range(10):
14 print(yhat[i+10],y[i+10],x[i+10])

Yet despite the expected plots of scaled values (x,yhat), and (x, y), yhat’s values are the same
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 181/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

1 Printing the output of the scaled values

2 printing the first 10
3 [0.00117336] [0.] [0.]
4 [0.00117336] [1.002003e-06] [0.001001]
5 [0.00117336] [4.00801202e-06] [0.002002]
6 [0.00117336] [9.01802704e-06] [0.003003]
7 [0.00117336] [1.60320481e-05] [0.004004]
8 [0.00117336] [2.50500751e-05] [0.00500501]
9 [0.00117336] [3.60721081e-05] [0.00600601]
10 [0.00117336] [4.90981472e-05] [0.00700701]
11 [0.00117336] [6.41281923e-05] [0.00800801]
12 [0.00117336] [8.11622433e-05] [0.00900901]
13 printing the 10th to 20th
14 [0.00117336] [0.0001002] [0.01001001]
15 [0.00117336] [0.00012124] [0.01101101]
16 [0.00117336] [0.00014429] [0.01201201]
17 [0.00117336] [0.00016934] [0.01301301]
18 [0.00117336] [0.00019639] [0.01401401]
19
20
[0.00117336]
[0.00117336]
[0.00022545] [0.01501502]
[0.00025651] [0.01601602] Start Machine Learning ×
21 [0.00117336] [0.00028958] [0.01701702]
22 [0.00117336] [0.00032465] [0.01801802] You can master applied Machine Learning
23 [0.00117336] [0.00036172] [0.01901902]
without math or fancy degrees.
Find out how in this free and practical course.
I don’t get it.You would expect a similarity of yhat and f(x).

I would appreciate a response

Email Address
Thank you,
Anthony of Sydney
START MY EMAIL COURSE

REPLY 
Jason Brownlee October 4, 2019 at 5:49 am #

Sorry, I don’t have the capacity to debug your examples further. I hope that you can
understand.

REPLY 
Anthony The Koala October 4, 2019 at 6:36 am #

Dear Dr Jason,
I asked the question at https://datascience.stackexchange.com/questions/61223/reconstituting-
estimated-predicted-values-to-original-scale-from-minmaxscaler and hope that there is an answer.
Thanks Start Machine Learning
Anthony Of Sydney

REPLY 
Jason Brownlee October 4, 2019 at 8:35 am #

Here is the solution

1 # fit an mlp on x vs x^2

2 from sklearn.preprocessing import MinMaxScaler
3 from keras.models import Sequential
4 from keras.layers import Dense
5 from numpy import asarray
6 from matplotlib import pyplot
7 # define data
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 182/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step
8 x = asarray([i for i in range(1000)])
9 y = asarray([a**2 for a in x])
10 # reshape into rows and cols
11 x = x.reshape((len(x), 1))
12 y = y.reshape((len(y), 1))
13 # scale data
14 x_s = MinMaxScaler()
15 x = x_s.fit_transform(x)
16 y_s = MinMaxScaler()
17 y = y_s.fit_transform(y)
18 # fit a model
19 model = Sequential()
20 model.add(Dense(10, input_dim=1, activation='relu'))
21 model.add(Dense(1))
22 model.compile(loss='mse', optimizer='adam')
23 model.fit(x, y, epochs=150, batch_size=10, verbose=0)
24 mse = model.evaluate(x, y, verbose=0)
25 print(mse)
26
27
# predict
yhat = model.predict(x) Start Machine Learning ×
28 # inverse transforms
29 x = x_s.inverse_transform(x) You can master applied Machine Learning
30 y = y_s.inverse_transform(y)
without math or fancy degrees.
31 yhat = y_s.inverse_transform(yhat)
32 # plot real vs predicted Find out how in this free and practical course.
33 pyplot.plot(x,y,label='y')
34 pyplot.plot(x,yhat,label='yhat')
35 pyplot.legend() Email Address
36 pyplot.show()

The three missing lines were:

START MY EMAIL COURSE
1 # inverse transforms
2 x = x_s.inverse_transform(x)
3 y = y_s.inverse_transform(y)
4 yhat = y_s.inverse_transform(yhat)

REPLY 
Anthony The Koala October 4, 2019 at 9:59 am #

Dear Dr Jason,
I am coming to the conclusion that there must be a bug NOT in your solution and neither in my solution. I
think it is coming from a bug in the lower implementation of the language.

I printed the scaled version of yhat, f(x) actual and x and got this.
NOTE the values are the same for the scaled version of yhat.
That is:
Start Machine Learning
1 model.fit(x, y, epochs=277, batch_size=200, verbose=0)
2 mse = model.evaluate(x, y, verbose=0)
3 print("the value of the mse")
4 print(mse)
5 # predict
6 yhat = model.predict(x)

DESPITE the successful plot of (x, yhat) and (x, f(x),

the resulting output of the first 10 of the scaled output of yhat is the same,

That is we would get a FLAT LINE if we plotted (x, yhat), BUT THE PLOT WAS A PARABOLA.

1 [0.00161531] [0.] [0.]

2 [0.00161531] [1.002003e-06] [0.001001]
3 [0.00161531] [4.00801202e-06] [0.002002]
4 [0.00161531] [9.01802704e-06] [0.003003]
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 183/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

5 [0.00161531] [1.60320481e-05] [0.004004]

6 [0.00161531] [2.50500751e-05] [0.00500501]
7 [0.00161531] [3.60721081e-05] [0.00600601]
8 [0.00161531] [4.90981472e-05] [0.00700701]
9 [0.00161531] [6.41281923e-05] [0.00800801]
10 [0.00161531] [8.11622433e-05] [0.00900901]

When we did the following transforms:

1 x = x_s.inverse_transform(x)
2 y = y_s.inverse_transform(y)
3 yhat = y_s.inverse_transform(yhat)

WE STILL GOT THE SAME FAULT FOR THE UNSCALED VALUES of yhat. The 2nd column is f(x) and
third column is x.

1 [1612.0857] [0.] [0.]

2
3
[1612.0857]
[1612.0857]
[1.] [1.]
[4.] [2.] Start Machine Learning ×
4 [1612.0857] [9.] [3.]
5 [1612.0857] [16.] [4.]
You can master applied Machine Learning
6 [1612.0857] [25.] [5.]
7 [1612.0857] [36.] [6.] without math or fancy degrees.
8 [1612.0857] [49.] [7.] Find out how in this free and practical course.
9 [1612.0857] [64.] [8.]
10 [1612.0857] [81.] [9.]
Email Address
Conclusion: It is not a programmatical bug in either your solution or my solution. I believe it may be a
lower implementation problem.
START MY EMAIL COURSE
Why am I ‘persistent’ in this matter: because in case I have more complex models I want to see the
predicted/yhat values that are re-scaled.

I don’t know if there are people at stackexchange who may have an insight.

I appreciate your time, many blessings to you,

Anthony of Sydney

REPLY 
Jason Brownlee October 6, 2019 at 8:05 am #

I believe is correct, given that it is an exponential, the model has decided that it can give up
correctness at the low end for correctness at the high end – given the reduction in MSE.

Consider changing the number of examples from 1K to 100, then review all 100 values manually –
you’ll see what I mean. Start Machine Learning

All of this is a good exercise, well done.

REPLY 
Anthony The Koala October 13, 2019 at 10:57 pm #

Dear Dr Jason,
I did this problem again and got very good results!
I cannot explain why I got accurate results, when I expected to get accurate results, BUT they
are certainly an improvement.

The rescaled original and fitted values produced an RMS of 0.0.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 184/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Here is the code with variable names changed slightly.

1 from sklearn.preprocessing import MinMaxScaler

2 from keras.models import Sequential
3 from keras.layers import Dense
4 from numpy import sqrt
5 from matplotlib import pyplot
6
7 x = asarray([i for i in range(100)])
8 y = asarray([i**2 for i in x])
9 x = x.reshape((len(x),1))
10 y = y.reshape((len(y),1))
11
12 x_s = MinMaxScaler()
13 xscaled = x_s.fit_transform(x)
14 y_s = MinMaxScaler()
15 yscaled = y_s.fit_transform(y)
16
17
18
model = Sequential()
model.add(Dense(100, input_dim=1, activation = 'relu')
Start Machine Learning ×
19 model.add(Dense(1))
20 You can master applied Machine Learning
model.compile(loss='mse',optimizer='adam')
21 model.fit(xscaled,yscaled,epochs=150, batch_size=10,verbose=0)
without math or fancy degrees.
22
23 mse= model.evaluate(xscaled,yscaled,verbose=0)
Find out how in this free and practical course.
24 mse
25 2.9744908551947447e-05
26 Email Address
27 yhat = model.predict(x)
28 yhat_original = y_s.inverse_transform(yscaled)
29 START MY EMAIL COURSE
30 #First five elements of predicted values
31 yhat_original[:5].T
32 array([[ 0., 1., 4., 9., 16.]])
33 #First five elements of original y
34 y[:5].T
35 array([[ 0, 1, 4, 9, 16]])
36
37 #Last five elements of the original series.
38 y[-5:].T
39 array([[9025, 9216, 9409, 9604, 9801]])
40 #Last five elements of predicted values
41 yhat_original[-5:].T
42 array([[9025., 9216., 9409., 9604., 9801.]])
43
44 #Now determining the RMS of the predicted and original values of y
45 cum_sum = 0
46 for i in range(len(yhat_original)):
47 cum_sum += (yoriginal[i] - yhat_original[i])**2/len(yhat_original)
48 rms = sqrt(cum_sum)
49 rms[0]
50 0.0 Start Machine Learning
51
52 #Plotting the rescaled original and rescaled yhat
53 pyplot.plot(xoriginal,yoriginal,label='y')
54 pyplot.plot(xoriginal,yhat_original,label='fitted')
55 pyplot.legend()
56 pyplot.show()

It works, the rescaled yhat is as expected but cannot explain why it was “cuckoo”, in the
previous. More experimentation on this.

Nevertheless, my next project is k-folds sampling on a deterministic function to see if the gaps in
the resampled data fold will give us an accurate prediction despite the random sampling in each
fold.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 185/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Thank you,
Anthony of Sydney

REPLY 
Anthony The Koala October 13, 2019 at 11:44 pm #

Dear Dr Jason,
Apologies, I thought the RMS was ‘unrealistic’. I had a programming error.
Nevertheless, I did it again, and still produced results which looked pleasing.

1 x = x.reshape((len(x),1))
2 y = y.reshape((len(y),1))
3 x_s = MinMaxScaler()
4 y_s = MinMaxScaler()
5 x_scaled = x_s.fit_transform(x) Start Machine Learning ×
6 y_scaled = y_s.fit_transform(y)
7 model = Sequential()
8 You can master applied Machine Learning
model.add(Dense(100,input_dim=1,activation='relu'))
9 model.add(Dense(1)) without math or fancy degrees.
10
Find out how in this free and practical course.
11 model.compile(loss='mse',optimizer='adam')
12 model.fit(x_scaled,y_scaled, epochs=100, batch_size=10,verbose=0)
13
14 Email Address
mse = model.evaluate(x_scaled, y_scaled,verbose=0)
15 mse
16 1.0475558547113905e-05
17 START MY EMAIL COURSE
18 yhat = model.predict(x_scaled)
19 yhat_original = y_s.inverse_transform(yhat)
20
21 #First five of yhat_original (yhat rescaled)
22 yhat_original[:5].T
23 array([[11.835742, 11.835742, 11.835742, 11.835742, 11.835742]]
24 #compared to first original 5 elements of y = 0,1,4,9,16
25
26 #Last five of yhat_original (yhat rescaled)
27 yhat_original[-5:].T
28 array([[8985.839, 9154.454, 9323.067, 9491.684, 9660.3 ]
29 #compared to last original 5 elements of y = 9025, 9216, 9409, 9604, 9801
30
31 #Now determine the RMS of the predicted and original values
32 cum_sum = 0
33 for i in range(len(yhat_original)):
34 cum_sum+= (y[i]-yhat_original[i])**2/len(yhat_original)
35 mse = sqrt(cum_sum)
36 mse
37 array([31.72189417])
38 pyplot.plot(x,y,label='y') Start Machine Learning
39 pyplot.plot(x,yhat_original,label='estimated')
40 pyplot.legend()
41 pyplot.show()

In sum, the rescaled yhat produced results closer to the original values. The lower values of
yhat rescaled appear to be odd.

Despite that the values need to be more realistic at the bottom end even though the plot of
the rescaled x & rescaled y, and rescaled x and rescaled yhat look close.

More investigations needed on the batch size, epochs and optimizers.

Next, to do k-folds sampling on a deterministic function to see if the gaps in the resampled
data fold will give us an accurate prediction despite the random sampling in each fold.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 186/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Again apologies for the mistake in the previous post.

Anthony of Sydney

REPLY 
Jason Brownlee October 14, 2019 at 8:08 am #

Well done.

Anthony The Koala November 21, 2019 at 5:03 am #

Dear Dr Jason,
Start Machine Learning ×
A person ‘Serali’ a particle physicist relied to me at “StackExchange” replied and
suggested that I shuffle the original data.
You The shuffling
can master of data
applied in this Learning
Machine context has nothing
to do with the shuffling in k-folds. According
withouttomath
the contributor, the results should
or fancy degrees.
improve. Source https://datascience.stackexchange.com/questions/61223/reconstituting-
Find out how in this free and practical course.
estimated-predicted-values-to-original-scale-from-minmaxscaler

The code is exactly the same as what IEmail Address

was experimenting with. So I will show the
necessary code to shuffe at the start and de-shuffle at the end.

Shuffling code at the beginning: START MY EMAIL COURSE

1 from sklearn.preprocessing import MinMaxScaler

2 from keras.models import Sequential
3 from keras.layers import Dense
4 from numpy import asarray
5 from numpy import sqrt
6 from matplotlib import pyplot
7
8 from numpy.random import seed
9 from numpy.random import shuffle
10 from numpy.random import sample
11
12 import numpy as np
13
14 #We will want to reshuffle the data
15 x = [i for i in range(100)]
16 y = [i**2 for i in x]
17
18 xfx = np.vstack((x,y)).T
19 xy = xfx
20 shuffle(xy) Start Machine Learning
21
22 #x = asarray([i for i in range(100)])
23 #y = asarray([i**2 for i in x])
24
25 #x = asarray(xy[:,0]);#x.reshape((len(x),1))
26 x = np.reshape(xy[:,0], (100,1))
27 #print('debug, size x = %d ' + str(np.shape(x)))
28 y = np.reshape(xy[:,1], (100,1))
29 #y = asarray(xy[:,1]);#y.reshape((len(y),1))
30 #print('debug, size y = %d ' + str(np.shape(y)))
31 x_s = MinMaxScaler()
32 y_s = MinMaxScaler()
33 x_scaled = x_s.fit_transform(x)
34 y_scaled = y_s.fit_transform(y)
35 #The rest is fed into model
36 ......

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 187/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

37 .......
38 yhat = model.predict(x_scaled)
39 yhat_original = y_s.inverse_transform(yhat)

The end code was ‘unshuffled’/sorted in order to display the difference between the
actual and predicted.

1 #Plotting dots instead of lineplot otherwise we get a zig-zag plot

2 pyplot.plot(x,y,'r.',label='y')
3 pyplot.plot(x,yhat_original,'b.', label='estimated')
4
5 #printing the first values - we have to sort the values in order to see
6 #their proper context.
7 xy = np.vstack((x[:,0],y[:,0])).T
8 xyhat = np.vstack((x[:,0],yhat_original[:,0])).T
9
10 xyy = np.sort(xy,axis=0)
11
12
xyhatt = np.sort(xyhat,axis=0)
Start Machine Learning ×
13 print("printing x, y, yhat")
14 for loop in range(10):
15 You can master applied Machine Learning
print(xyy[loop,0],xyy[loop,1],xyhatt[loop,1])
16 without math or fancy degrees.
17 Find out how in this free and practical course.
18 pyplot.legend()
19 pyplot.show()
Email Address
Here is a listing of x, f(x) and yhat

1 printing x, y, yhat
2 0 0 1.4915295839309692 START MY EMAIL COURSE
3 1 1 2.66086745262146
4 2 4 4.75526237487793
5 3 9 9.125076293945312
6 4 16 15.723174095153809
7 5 25 24.287418365478516
8 6 36 35.04938507080078
9 7 49 47.73912811279297
10 8 64 62.95930480957031
11 9 81 80.16889190673828

Things to improve:
* adjusting the number of layers.
* adjusting how many neurons in each layer
* adjusting the batch size
* adjusting the epoch size
In addition
* look at k-folds for further model refinement.
Start Machine Learning
Thank you
Anthony of Sydney

Anthony The Koala November 24, 2019 at 4:12 pm #

Dear Dr Jason,
Here is an even improved version with very close results.
Instead of MinMaxScaler, I took the logs (to the base e) of the inputs x and f(x) applied
my model, then retransformed my model to its original values.

Snippets of code transforming the data

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 188/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

1 #We will want to reshuffle the data

2 x = [i for i in range(100)]
3 y = [i**2 for i in x]
4
5 xfx = np.vstack((x,y)).T
6 xy = xfx
7 seed(1)
8 shuffle(xy)
9
10 x = np.reshape(xy[:,0], (100,1))
11 print('debug, size x = %d ' + str(np.shape(x))) #shape is (100,1)
12 y = np.reshape(xy[:,1], (100,1))
13
14 print('debug, size y = %d ' + str(np.shape(y)))
15 #x_s = MinMaxScaler()
16 #x_s = MinMaxScaler(feature_range =(0,200))
17 #y_s = MinMaxScaler()
18 #x_scaled = x_s.fit_transform(x)
19
20
#y_scaled = y_s.fit_transform(y)
Start Machine Learning
x_scaled = np.log(x+1); # we add 1 so as not to have an error as log(0)
×
21 y_scaled = np.log(y+1); # we add 1 so as not to have an error as log(0)
22 You can master applied Machine Learning
23 model = Sequential()
without math or fancy degrees.
24 ...# the model is applied on the transformed data
Find out how in this free and practical course.
The

1 Email Address
#We need to resort the numbers
2 #in order to print the first 10 values
3 xy = np.vstack((x[:,0],y[:,0])).T
4 xyhat = np.vstack((x[:,0],yhat_original[:,0])).T
START MY EMAIL COURSE
5
6 xyy = np.sort(xy,axis=0)
7 xyhatt = np.sort(xyhat,axis=0)
8 print("printing x, y, yhat")
9 for loop in range(10):
10 print(xyy[loop,0],xyy[loop,1],xyhatt[loop,1])
11
12 #want to predict for the values 100 and 200
13 Xnew = np.reshape([100,200],(2,1))
14
15 print("let's predict for values 100 and 200")
16 print("the values of x = Xnew before transform %s, %s " % (Xnew[0],Xnew[
17
18 Xnew = np.log(Xnew+1)
19 print("values of scaled xnew to put into the model %s, %s " % (Xnew[0],X
20 ynew = model.predict(Xnew)
21
22 #Re-transform the original values
23 ynew = np.exp(ynew) - 1
24 print("The values of Xnew and its predicted yhat")
25 for loop in range(len(Xnew)):
Start Machine Learning
26 print("Xnew[%s] = %s, ynew[%s] = %s " % (loop,Xnew[loop],loop,yn

The resulting output: Note how close the actual f(x) is to the predicted f(x)

1 printing x, y, yhat
2 0 0 0.00208890438079834
3 1 1 0.9818048477172852
4 2 4 4.111057281494141
5 3 9 9.025933265686035
6 4 16 15.918327331542969
7 5 25 24.944564819335938
8 6 36 36.00426483154297
9 7 49 49.05435562133789
10 8 64 63.969764709472656
11 9 81 80.93276977539062
12
13 let's predict for values 100 and 200
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 189/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step
14 the values of x = Xnew before transform [100], [200]
15 values of scaled xnew to put into the model [4.61512052], [5.30330491]
16
17 The values of Xnew and its predicted yhat
18 Xnew[0] = [100.], ynew[0] = [10008.037]
19 Xnew[1] = [200.], ynew[1] = [40082.062]

Jason Brownlee November 25, 2019 at 6:21 am #

Nice work.

kamu October 6, 2019 at 7:51 pm # Start Machine Learning ×

REPLY 

Hi Jason, You can master applied Machine Learning

Thank you very much for “Your First Deep Learning Project
withoutin math
Python
or with
fancyKeras Step-By-Step” tutorial.
degrees.
It is very useful for me. I want to ask you: Find out how in this free and practical course.
Can I code:

model.add(Dense(8)) # input layer Email Address

model.add(Dense(12, activation=’relu’)) # first hidden layer

Instead of: START MY EMAIL COURSE

model.add(Dense(12, input_dim=8, activation=’relu’)) # input layer and first hidden layer

Sincerely.

REPLY 
Jason Brownlee October 7, 2019 at 8:29 am #

No.

The input_dim argument defines the input layer.

REPLY 
keryums October 17, 2019 at 1:22 am #
Start Machine Learning
Hi Jason, is it not necessary to use the keras utilility ‘to_categorical’ to convert your y vector into
a matrix before fitting the model?

REPLY 
Jason Brownlee October 17, 2019 at 6:37 am #

You can, or you can use the sklearn tools to do the same thing.

REPLY 
Aquilla Setiawan Kanadi October 17, 2019 at 6:35 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 190/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Hi Jason,

Thanks a lot for your tutorial about deep learning project, it really help me a lot in my journey to
learn machine learning.

I have a question about the data splitting in code above, how is the splitting work between data for
training and the data for validate the training data? I’ve tried to read your tutorial about the data splitting
but i have no ideas about the data splitting work above.

Thankyou,

Aquilla

Jason Brownlee October 17, 2019 at 6:47 am #

Start Machine Learning ×
REPLY 

We did not split the data, we fit and evaluated on one set. We did this for brevity.
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Love your work! October 17, 2019 at 11:43 am #
Email Address
Hi Jason,

I just wanted to thank you. This tutorial is incredibly clearSTART

and well presented.
MY EMAIL Unlike many other online
COURSE
tutorials you explain very eloquently the intuition behind the lines of code and what is being
accomplished which is very useful. As someone just starting out with Keras I had been finding some of
the coding, as well as how Keras and Tensorflow interact, confusing. After your explanations Keras
seems incredibly basic. I’ve been looking over some of my recent code from other Keras tutorials and I
now understand how everything works.

Thanks again!

REPLY 
Jason Brownlee October 17, 2019 at 1:50 pm #

Well done on your progress and thanks for your support!

Start Machine Learning

REPLY 
Ahmed October 19, 2019 at 6:16 am #

Dear Jason. I am deeply grateful to this amazing work. Everything works well so far. King
Regards

REPLY 
Jason Brownlee October 19, 2019 at 6:55 am #

Thanks, well done on your progress!

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 191/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
JAMES JONAH October 28, 2019 at 10:56 am #

Please i need help, which algorithms is the best in cyber threat detection and how to implement
it. thanks

REPLY 
Jason Brownlee October 28, 2019 at 1:18 pm #

This is a common question that I answer here:

https://machinelearningmastery.com/faq/single-faq/what-algorithm-config-should-i-use

Start Machine Learning ×

REPLY 
shivan October 29, 2019 at 7:01 am #
You can master applied Machine Learning
hello sir without math or fancy degrees.
do you have an implementation about (medical image Find
analysis withindeep
out how learning).
this free and practical course.
i need to start with medical image NOT real world image
thanks for your help.
Email Address

START MY EMAIL COURSE

REPLY 
Jason Brownlee October 29, 2019 at 1:47 pm #

Not really, sorry.

REPLY 
shivan October 31, 2019 at 9:18 am #

so, what do you recommend me about it

thanks.

REPLY 
Jason Brownlee October 31, 2019 at 1:36 pm #

Start Machine Learning

Perhaps start by collecting a dataset.

Then consider reviewing the literature to see what types of data prep and models other have
used for similar data.

REPLY 
Nasir Shah October 30, 2019 at 7:27 am #

Sir. i am new to neural network. so from where i start it. or which tutorial i watch . i didn’t have
any idea about it.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 192/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Jason Brownlee October 30, 2019 at 1:55 pm # REPLY 

Yes, you can start here:

https://machinelearningmastery.com/start-here/#deeplearning

REPLY 
hima hansi November 3, 2019 at 1:35 pm #

hello sir, I’m new to this field. I’m going to develop monophonic musical instrument classification
system using python and Keras. sir,I want to find monophonic data set, how can I find it.
I try to get piano music from you tube and convert it to .waw file and splitting it. Is it a good or bad ? or an
other methods available to get free data set on the web.. give your suggestions please ??

Start Machine Learning ×

You can master applied Machine Learning REPLY 
Jason Brownlee November 4, 2019 at 6:37 am #
without math or fancy degrees.
Find out how in this free and practical course.
Perhaps this will help:
https://machinelearningmastery.com/faq/single-faq/where-can-i-get-a-dataset-on-___
Email Address

START MY EMAIL COURSE REPLY 

Mona Ahmed November 20, 2019 at 3:14 am #

i got score 76.69

REPLY 
Jason Brownlee November 20, 2019 at 6:20 am #

Well done!

REPLY 
Niall Xie November 26, 2019 at 8:26 am #

Hello, I just want to say that I am elated to use your tutorial. So, I am working on a group project
with my team and I used datasets representing heart disease, diabetes
Start Machine and breast cancer for this
Learning
tutorial. However, this code example will give an error when the cell contains a string value, in this
case… title names like clump_thickess and ? will produce an error. how do I fix this?

REPLY 
Jason Brownlee November 26, 2019 at 1:28 pm #

Thanks.

Perhaps try encoding your categories using a one hot encoding first:
https://machinelearningmastery.com/how-to-prepare-categorical-data-for-deep-learning-in-python/

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 193/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Mohamed November 28, 2019 at 10:46 pm #

thank you sir for this article, would you please suggest an example with testing data ?

REPLY 
Jason Brownlee November 29, 2019 at 6:49 am #

Sorry I don’t understand your question, can you elaborate?

REPLY 
Chris December 3, 2019 at 10:49 pm #
Start Machine Learning ×
I believe there is something wrong with the (150/10) 15 updates to the model weights. The
internal coefficients are updated after every single batch.
YouOur
can data is applied
master comprised of 768
Machine samples. Since
Learning
batch_size=10, we obtain 77 batches (76 with 10 samples andmath
without one or
with 8). Therefore,
fancy degrees. at each epoch we
should see 77 updates of weights and coefficients andFind
not out
15. how
Moreover, the and
in this free totalpractical
numbercourse.
of updates
must be: 150*77=11550. Am I missing something important?
Email Address
Really good job and very well-written article (all your articles). Keep up the good job. Cheers

START MY EMAIL COURSE

REPLY 
Jason Brownlee December 4, 2019 at 5:37 am #

You’re right. Not sure what I was thinking there. Simplified.

REPLY 
Justine December 14, 2019 at 9:58 am #

Thanks! This is my first foray into keras, and the tutorial went swimmingly. Am now training on
my own data. It is not performing worse than on my other machine learning models (that’s a win :).

REPLY 
Jason Brownlee December 15, 2019 at 6:02 am #
Start Machine Learning
Well done!

REPLY 
x December 17, 2019 at 8:37 am #

Hi，Jason. Thanks so much for your answer. Now my question is why I can’t found my directory
in Jupyter and put the ‘pima-indians-diabetes.csv’ in it.
OSError Traceback (most recent call last)
in
4 from keras.layers import Dense
5 # load the dataset

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 194/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

—-> 6 dataset = loadtxt(‘pima-indians-diabetes.csv’, delimiter=’,’)

7 # split into input (X) and output (y) variables
8 X = dataset[:,0:8]

D:\anaconda\lib\site-packages\numpy\lib\npyio.py in loadtxt(fname, dtype, comments, delimiter,

converters, skiprows, usecols, unpack, ndmin, encoding, max_rows)
966 fname = os_fspath(fname)
967 if _is_string_like(fname):
–> 968 fh = np.lib._datasource.open(fname, ‘rt’, encoding=encoding)
969 fencoding = getattr(fh, ‘encoding’, ‘latin1’)
970 fh = iter(fh)

D:\anaconda\lib\site-packages\numpy\lib\_datasource.py in open(path, mode, destpath, encoding,

newline)
267 Start Machine Learning ×
268 ds = DataSource(destpath)
–> 269 return ds.open(path, mode, encoding=encoding,
Younewline=newline)
can master applied Machine Learning
270 without math or fancy degrees.
271 Find out how in this free and practical course.

D:\anaconda\lib\site-packages\numpy\lib\_datasource.py in open(self, path, mode, encoding, newline)

621 encoding=encoding, newline=newline) Email Address
622 else:
–> 623 raise IOError(“%s not found.” % path) START MY EMAIL COURSE
624
625

OSError: pima-indians-diabetes.csv not found.

REPLY 
Jason Brownlee December 17, 2019 at 1:36 pm #

Perhaps try running the code file from the command line, as follows:
https://machinelearningmastery.com/faq/single-faq/how-do-i-run-a-script-from-the-command-line

REPLY 
Manohar Nookala December 22, 2019 at 9:32 pm #
Start Machine Learning
Hi sir,
My name is manohar. i trained a deep learning model on car price prediction. i got

loss: nan – acc: 0.0000e+00. if you give me your email ID then i will send you. you can tell me the
problem. please do this help because i am a beginner.

REPLY 
Jason Brownlee December 23, 2019 at 6:48 am #

Perhaps you need to scale the data prior to fitting?

Perhaps you need to use relu activation?

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 195/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Perhaps you need some type of regularization?

Perhaps you need a larger or smaller model?

REPLY 
Shone Xu January 5, 2020 at 1:24 am #

Hi Jason,

thanks and it is a great tutorial. just 1 question. do we have to train the model by “model.fit(x, y,
epochs=150, batch_size=10)” every time before making the prediction because it takes a very long time
to train the model. I am just wondering whether it is possible to save the trained model and go straight to
the prediction skipping the model.fit (eg: pickle)?

many thanks for your advice in advance

Start Machine Learning ×
cheers
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Jason Brownlee January 5, 2020 at 7:06 am #
Email Address
No, you can fit the model once, then save it:
https://machinelearningmastery.com/save-load-keras-deep-learning-models/
START MY EMAIL COURSE
Then later load it and make predictions.

REPLY 
Shone Xu January 7, 2020 at 2:16 pm #

Thanks and will check it out

REPLY 
ustengg January 8, 2020 at 7:40 pm #

Thank you so much for this tutorial sir but How can I use the model to predict using data outside
the dataset?

Start Machine Learning

REPLY 
Jason Brownlee January 9, 2020 at 7:24 am #

Call model.predict() with the new inputs.

See the “Make Predictions” section.

REPLY 
ustengg January 9, 2020 at 4:07 pm #

Nice! Thank you so much, Sir. I figured it out using the link on the “Make predictions”
section. I’ve learned a lot from your tutorials. You’re the best!
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 196/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee January 10, 2020 at 7:22 am #

Nice work!

Thanks.

REPLY 
monica January 23, 2020 at 4:00 am #

Hi Jason,

Thanks for sharing this post. Start Machine Learning ×

I have a question, when I tried to split the dataset You can master applied Machine Learning
(X = dataset[:,0:8] without math or fancy degrees.
y = dataset[:,8]) Find out how in this free and practical course.

it gives me an error: TypeError: ‘(slice(None, None, None), slice(0, 8, None))’ is an invalid key
Email Address
how can I fix it?

Thanks,
START MY EMAIL COURSE
monica

REPLY 
Jason Brownlee January 23, 2020 at 6:41 am #

Sorry to hear that, this might help:

https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-
me

REPLY 
Sam Sarjant January 23, 2020 at 9:11 pm #

Thanks for the tutorial! This is a wonderful ‘Hello World’ to Deep Learning
Start Machine Learning

REPLY 
Jason Brownlee January 24, 2020 at 7:51 am #

Thanks, I’m happy it was helpful.

REPLY 
Keerthan January 24, 2020 at 4:01 pm #

Hello Jason! hope you are doing good.

I am actually doing a project on classification of thyroid disease using back propagation with stocastic

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 197/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

gradient descent method,can you help me out with the code a little bit?

REPLY 
Jason Brownlee January 25, 2020 at 8:31 am #

Perhaps start by adapting the code in the above tutorial?

REPLY 
Shakir January 25, 2020 at 1:29 am #

Dear Sir
I want to predict air pollution using deep learning techniques please suggest how to go about with my
data sets Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course. REPLY 
Jason Brownlee January 25, 2020 at 8:39 am #

Start here: Email Address

https://machinelearningmastery.com/start-here/#deep_learning_time_series

START MY EMAIL COURSE

REPLY 
Yared February 7, 2020 at 4:36 pm #

AttributeError: module ‘tensorflow’ has no attribute ‘get_default_graph’AttributeError: module

‘tensorflow’ has no attribute ‘get_default_graph’

REPLY 
Jason Brownlee February 8, 2020 at 7:05 am #

Perhaps confirm you are using TF 2 and Keras 2.3.

REPLY 
Yared February 7, 2020 at 4:41 pm # Start Machine Learning

I went to detect agreement errors in a sentence using LSTM techniques please suggest how to
go about with my data sets

REPLY 
Jason Brownlee February 8, 2020 at 7:05 am #

You can get started with NLP problems here:

https://machinelearningmastery.com/start-here/#nlp

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 198/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Pavitra Nayak February 29, 2020 at 2:55 pm # REPLY 

Hello Jason
I am using this code for my project. It works perfectly for your dataset. But I have a dataset which has too
many 0’s and 1’s. So I am getting the wrong prediction. What can I do to solve this problem?

REPLY 
Jason Brownlee March 1, 2020 at 5:22 am #

Here are some suggestions:

https://machinelearningmastery.com/improve-deep-learning-performance/

Start Machine Learning ×

REPLY 
nurul March 6, 2020 at 5:50 pm #
You can master applied Machine Learning
without
hi. I wanna ask. i had follow all the steps but i’m stuckmath orfitfancy
at the degrees.
the model. This error occured.
How can I solve this problem? Find out how in this free and practical course.

Email Address

REPLY 
kiki March 6, 2020 at 6:44 pm #
START MY EMAIL COURSE

I have already tried this step and stuck at the fit phase and got this error. Do you have any
solution for my problem?

—————————————————————————
ValueError Traceback (most recent call last)
in
1 # fit the keras model on the dataset
—-> 2 model.fit(x, y, batch_size=10,epochs=150)

~\Anaconda4\lib\site-packages\keras\engine\training.py in fit(self, x, y, batch_size, epochs, verbose,

callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch,
steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers,
use_multiprocessing, **kwargs)
1152 sample_weight=sample_weight,
1153 class_weight=class_weight,
-> 1154 batch_size=batch_size) Start Machine Learning
1155
1156 # Prepare validation data.

~\Anaconda4\lib\site-packages\keras\engine\training.py in _standardize_user_data(self, x, y,
sample_weight, class_weight, check_array_lengths, batch_size)
577 feed_input_shapes,
578 check_batch_axis=False, # Don’t enforce the batch size.
–> 579 exception_prefix=’input’)
580
581 if y is not None:

~\Anaconda4\lib\site-packages\keras\engine\training_utils.py in standardize_input_data(data,
names, shapes, check_batch_axis, exception_prefix)
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 199/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

143 ‘: expected ‘ + names[i] + ‘ to have shape ‘ +

144 str(shape) + ‘ but got array with shape ‘ +
–> 145 str(data_shape))
146 return data
147

ValueError: Error when checking input: expected dense_133_input to have shape (16,) but got array
with shape (17,)

REPLY 
Jason Brownlee March 7, 2020 at 7:15 am #

Perhaps this will help you copy the code from the tutorial:
Start Machine Learning
https://machinelearningmastery.com/faq/single-faq/how-do-i-copy-code-from-a-tutorial ×
You can master applied Machine Learning
without math or fancy degrees.
Jason Brownlee March 7, 2020 at 7:13 am #Find out how in this free and practical course. REPLY 

I’m sorry to hear that, perhaps this will help:

Email Address
https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-
me
START MY EMAIL COURSE

REPLY 
kiki March 9, 2020 at 12:18 pm #

Thanks for the answer jason

REPLY 
Jason Brownlee March 10, 2020 at 5:34 am #

You’re welcome.

REPLY 
laz March 7, 2020 at 2:59 pm # Start Machine Learning

Hey, Jason!

Again… Thanks for your awesome tutorials and for giving your knowledge to the public! >800 comments
and nearly all answered, you’re great. I can’t understand how you manage all that, writing great content,
do ml stuff, teach, learn, great respect!

2 general questions:

Question(1):

Why and when do we need to flatten() inputs and in which cases not?

For example 4 numeric inputs, a lag of 2 of every input means 4*2=8 values per batch:

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 200/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

I always do this, no matter how many inputs or lags, i give that as flat array to the input:

1 set/batch: [[1.0,1.1, 2.0,2.1, 3.0,3.1, 4.0,4.1]]

Input(shape=(8,)) # keras func api

Does it make sense to input a structure like this, if so – why/when?

Better? [[[1.0,1.1], [2.0,2.1], [3.0,3.1], [4.0,4.1]]]

Question(2):

Are you still using Theano? As they do not update it, it becomes older, but not worse ;). I tried Tensorflow
a lot – but always with lower performance in terms of speed. Theano is much faster (factor 3-10) for me.
But using more than 1 core is always slower for me, in both theano and tf. Did you experienced similar
things? I also tried torch, nice but it was also slower as the good old theano. Any ideas or alternatives (i
can’t use gpu/external/aws)?
Start Machine Learning ×
I would be happy to see you doing some deep reinforcement learning (DRL) stuff, what do you think?
You can master applied Machine Learning
Are you?
without math or fancy degrees.
Regards, keep it up Find out how in this free and practical course.

Email Address
REPLY 
Jason Brownlee March 8, 2020 at 6:07 am #
START MY EMAIL COURSE
You need to flatten when the output shape of one layer does not match the input shape of
another, e.g. CNN output to a Dense.

No. I use and recommend tensorflow and have for years. Tensorflow used to not work for windows
users, so I recommend theano for them – and still do if they have trouble. Theano works fine and will
continue to work fine for most applications.

No, RL is not practical/useful:

https://machinelearningmastery.com/faq/single-faq/do-you-have-tutorials-on-deep-reinforcement-
learning

REPLY 
laz March 8, 2020 at 11:29 am #

Dear Jason, thanks for your answer ;)…

Start Machine Learning
“flatten when the output shape of one layer does not match the input shape of another, e.g. CNN output
to a Dense.”

Thanks. The question about the “flatten” operation was not about the flatten() between layers, it was
about how to present inputs to the input layer. Sorry for being vague. Maybe I misunderstood something,
are there use cases where the FEATURES/INPUTS/LAGS are not flattened?

“RL is not practical/useful”

Is this statement based on your experience or do you take the opinion of others without checking it
yourself here ;)? Please do not misunderstand, you are the expert here. However, i can refute some
arguments against RL.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 201/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Rewards are hard to create: depends on your environment

Unstable: depends on your environment, code, setup

I started experimenting with a simple DQN, I expanded it step by step and now I have a “Dueling Double
DQN”. It learns well and quick. I admit – on simple data. But it does it repeatable and reproducible! So i
would say: In general, it works.

I have to see how it works with more complicated data. That is why I emphasized that the performance
of this method strongly depends on the area of application.

But there is a huge problem, most public sources contain incorrect code or incorrect implementations. I
have never reported or found so many bugs on any subject. These errors are copied again and again
and in the end many think that they are correct. I have collected tons of links and pdf files to understand
and debug this beast.

No matter, you have to decide for yourself. If you want to take a look at it, take a simple example, even
Start Machine Learning ×
the DQN (without dueling or double) is able to learn – if the code is correct. And although I’m not a
mathematician: to understand how it works and what possibilities it offers
You can master – made
applied meLearning
Machine smile …
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Jason Brownlee March 9, 2020 at 7:14 am # Email Address

For more on the input shape of LSTMs/1d CNNs, see this:

https://machinelearningmastery.com/faq/single-faq/what-is-the-difference-between-samples-
START MY EMAIL COURSE
timesteps-and-features-for-lstm-input

I don’t yet see an ROI for “developers at work” in covering RL as described in the link.

REPLY 
laz March 8, 2020 at 10:44 pm #

Interesting read:

“We use a double deep Q-learning network (DDQN) to find the right material type and the optimal
geometrical design for metasurface holograms to reach high efficiency. The DDQN acts like an intelligent
sweep and could identify the optimal results in ~5.7 billion states after only 2169 steps. The optimal
results were found between 23 different material types and various geometrical properties for a three-
layer structure. The computed transmission efficiency was 32% for high-quality metasurface holograms;
this is two times bigger than the previously reported results
Start under the Learning
Machine same conditions.”

https://www.nature.com/articles/s41598-019-47154-z

REPLY 
Jason Brownlee March 9, 2020 at 7:16 am #

Thanks for sharing.

REPLY 
YzN March 11, 2020 at 4:01 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 202/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Literally the best “first neural network tutorial”

Got 85.68 acc by adding layers and decreasing batch size

REPLY 
Jason Brownlee March 11, 2020 at 5:29 am #

Thanks.

Well done!

REPLY 
Neha March 14, 2020 at 12:05 am #
Start Machine Learning ×
Hello Jason,
I have a quick question. You can master applied Machine Learning
I am trying to build just 1 sigmoid neuron for a binary classification
without mathtask, basically
or fancy I am implying this is
degrees.
how 1 sigmoid model is: Find out how in this free and practical course.

model = Sequential()
model.add(Dense(1, activation=’sigmoid’)) Email Address

My inputs are images of size = (39*39*3)

START MY EMAIL COURSE
I am unsure as to how to input these images to my Dense layer (which is the only layer I am using)

I am currently using below for inputting my images:

train_generator = train_datagen.flow_from_directory(train_data_dir,
target_size=(39, 39),
batch_size=batch_size)
class_mode=’binary’)

But somehow Dense layer cannot accept input shape (39, 39, 3).

So my question is, how do I input my images data to the Dense layer?

REPLY 
Jason Brownlee March 14, 2020 at 8:13 am #

You can flatten the input or use a CNN asStart Machine

the input Learning
instead that is designed for 3d input
samples.

REPLY 
Bertrand Bru March 29, 2020 at 12:38 am #

Hi Jason,

Thank you very much for your tutorial.

I am new in the world of deep leraning. I have been able to modify your code and make it work for a set
of data I recorded with a 3 axis accelerometer. My goal was to detect if I was walking or running. I
recorded around 50 trials of each activities. From the signal, I calculated specific parameters that enable
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 203/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

the code to differenciate the two activities. Amongst the parameters, I calculated for all axis, the mean,
min and max values, and some parameters in the domain frequencies (the 3 first peak of the power
spectrum and their respective position).

It works very well and I am able to easily detect if I am running or walking.

I then decided to add a thrid activities: standing. I also recorded 50 trials of this activity. If I train my
model with standing and running, I can identify the two activity. Same if I train it with standing and
walking or with walking and running.

It is more complicated if I train my model with the three activities. In fact, it can’t do it. It can only
recgonise the first two activities. So for example if standing, walking and running have the following ID: 0,
1 and 2, then it can only detect 0 and 1 (standing and walking). It thinks that all running trials are walking
trials. If standing, running and walinking have the following ID: 0, 1 and 2, then it can only detect 0 and 1
(standing and running). It thinks that all walking trials are running trials.
Start Machine Learning ×
So here is my question: Assuming you have the dataset, if you needed to adapt your code so it can
detect if people are 0: not diabetic, 1: people are diabetic
You type 1, and applied
can master 2: people are diabetic
Machine type 2, how
Learning
would you modify your script? without math or fancy degrees.
Find out how in this free and practical course.
Thank you very much for your help.

Email Address

REPLY 
Jason Brownlee March 29, 2020 at 6:00 am #
START MY EMAIL COURSE

You’re welcome.

Well done.

This is called multi-class classification, this tutorial will help:

https://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/

REPLY 
Bertrand Bru March 29, 2020 at 7:15 am #

Thank you so much for coming to me so quickly.

This is exactly what I was looking for.
Cheers,

Start Machine Learning

REPLY 
Jason Brownlee March 30, 2020 at 5:27 am #

You’re welcome.

I’m happy to hear that.

REPLY 
Dipak Kambale March 31, 2020 at 10:16 pm #

Hi Jason,

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 204/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

I got accuracy 75.52 . Is it ok?? please let me know

REPLY 
Jason Brownlee April 1, 2020 at 5:49 am #

Well done. Try running the example a few times.

REPLY 
islamuddin April 1, 2020 at 6:20 pm #

hello sir jason.

sir how to satiable accuracy run the cod one given out for example 86% next time 82% how to solve this!
Start Machine Learning ×
#import
from numpy import loadtxt You can master applied Machine Learning
from keras.models import Sequential without math or fancy degrees.
from keras.layers import Dense Find out how in this free and practical course.

# load the dataset

dataset = loadtxt(‘E:/ms/impotnt/iwp1.csv’, delimiter=’,’)Email Address
# split into input (X) and output (y) variables
X = dataset[:,0:8]
START MY EMAIL COURSE
y = dataset[:,8]
# define the keras model

#model = Sequential()
model = Sequential()
#model.add(Dense(25, input_dim=8, init=’uniform’, activation=’relu’))
model.add(Dense(30, input_dim=8, activation=’relu’))
model.add(Dense(95, activation=’relu’))
model.add(Dense(377, activation=’relu’))
model.add(Dense(233, activation=’relu’))
model.add(Dense(55, activation=’relu’))
model.add(Dense(1, activation=’sigmoid’))

# compile the keras model

model.compile(loss=’binary_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])
# fit the keras model on the dataset Start Machine Learning
model.fit(X, y, epochs=150, batch_size=10)
# evaluate the keras model
_, accuracy = model.evaluate(X, y)
print(‘Accuracy: %.2f’ % (accuracy*100))

output

0.1153 – accuracy: 0.9531

Epoch 149/150
768/768 [==============================] – 0s 278us/step – loss: 0.1330 – accuracy: 0.9401
Epoch 150/150
768/768 [==============================] – 0s 277us/step – loss: 0.1468 – accuracy: 0.9375

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 205/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

768/768 [==============================] – 0s 41us/step

Accuracy: 94.01

REPLY 
Jason Brownlee April 2, 2020 at 5:46 am #

This is a common question that I answer here:

https://machinelearningmastery.com/faq/single-faq/can-you-read-review-or-debug-my-code

REPLY 
M Husnain Ali Nasir April 3, 2020 at 2:55 am #

Traceback (most recent call last): Start Machine Learning ×

File “keras_first_network.py”, line 7, in
You can master applied Machine Learning
dataset = loadtxt(‘pima-indians-diabetes.csv’, delimiter=’,’)
without math or fancy degrees.
File “C:\Users\Hussnain\anaconda3\lib\site-packages\numpy\lib\npyio.py”, line 1159, in loadtxt
Find out how in this free and practical course.
for x in read_data(_loadtxt_chunksize):
File “C:\Users\Hussnain\anaconda3\lib\site-packages\numpy\lib\npyio.py”, line 1087, in read_data
items = [conv(val) for (conv, val) in zip(converters, vals)]Email Address
File “C:\Users\Hussnain\anaconda3\lib\site-packages\numpy\lib\npyio.py”, line 1087, in
items = [conv(val) for (conv, val) in zip(converters, vals)]START MY EMAIL COURSE
File “C:\Users\Hussnain\anaconda3\lib\site-packages\numpy\lib\npyio.py”, line 794, in floatconv
return float(x)
ValueError: could not convert string to float: ‘”6’

I AM HAVIN THE ABOVE ERROR WHILE RUNNING IT PLEaSE HELP. I am using Anaconda 3 , Python
3.7 , tensorflow ,keras

REPLY 
Jason Brownlee April 3, 2020 at 6:57 am #

Sorry to hear that, this will help:

https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-
me

Start Machine Learning

REPLY 
Madhawa Akalanka April 9, 2020 at 6:22 pm #

(base) C:\Users\Madhawa Akalanka\python codes>python keras_first_network.py

Using TensorFlow backend.
2020-04-09 13:42:28.003791: I tensorflow/core/platform/cpu_feature_guard.cc:142]
Your CPU supports instructions that this TensorFlow binary was not compiled to
use: AVX AVX2
2020-04-09 13:42:28.014066: I tensorflow/core/common_runtime/process_util.cc:147
] Creating new thread pool with default inter op setting: 2. Tune using inter_op
_parallelism_threads for best performance.
Traceback (most recent call last):

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 206/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

File “keras_first_network.py”, line 12, in

model.fix(X,Y,epochs=150,batch_size=10)
AttributeError: ‘Sequential’ object has no attribute ‘fix’

I had this error while it’s being run. please help.

REPLY 
Jason Brownlee April 10, 2020 at 8:25 am #

Sorry to hear that, see this:

https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-
me

Start Machine Learning ×

REPLY 
Rahim Dehkharghani April 14, 2020 at 2:05 am You
# can master applied Machine Learning
without math or fancy degrees.
Dear Jason Find out how in this free and practical course.
Thanks for your wonderful website and books. I am a PhD holder and one of your fans in Deep Learning.
Sometimes I get disappointed because I cannot achieve my goal
Email in this area. My goal is to discover
Address
something new and publish it. Although I understand your codes mostly but having contribution in this
field is difficult and requires understanding the whole theory which I have not been able to do so far. Can
you please give me some tips to continue? Thanks a lotSTART MY EMAIL COURSE

REPLY 
Jason Brownlee April 14, 2020 at 6:25 am #

You’re welcome.

Keep working on it every day. That’s my best advice.

REPLY 
MattGurney April 16, 2020 at 10:52 pm #

There is a typo “input to the model lis defined”

Start Machine Learning

REPLY 
Jason Brownlee April 17, 2020 at 6:21 am #

Thanks! Fixed.

REPLY 
MattGurney April 16, 2020 at 11:28 pm #

Using the latest libraries today I get a number of warnings due to latest numpy: 1.18.1 not being
compatible with latest TensorFlow: 1.13.1.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 207/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

i.e:
FutureWarning: Passing (type, 1) or ‘1type’ … (6 times)
to_int32 (from tensorflow.python.ops.math_ops) is deprecated

Options are to revert to an older numpy or suppress the warnings, I took the suppress route with this
code:

# first neural network with keras tutorial

# Suppress warnings due to TF / numpy version incompatibility:

https://github.com/tensorflow/tensorflow/issues/30427#issuecomment-527891497
import warnings
warnings.filterwarnings(‘ignore’, category=FutureWarning)

import tensorflow

Start Machine Learning

# Suppress warning from TF: to_int32 (from tensorflow.python.ops.math_ops) is deprecated:
×
https://github.com/aamini/introtodeeplearning/issues/25#issuecomment-578404772
You can master applied Machine Learning
import logging
without math or fancy degrees.
logging.getLogger(‘tensorflow’).setLevel(logging.ERROR)
Find out how in this free and practical course.
import keras
from numpy import loadtxt Email Address
from keras.models import Sequential
from keras.layers import Dense
START MY EMAIL COURSE

REPLY 
Jason Brownlee April 17, 2020 at 6:21 am #

I recommend using Keras 2.3 and TensorFlow 2.1.

REPLY 
MattGurney April 17, 2020 at 12:18 pm #

Yes, upgrading to tensorFlow 2.1 fixed it, I have now removed my warnings
suppression and I don’t see the warnings in the output

I upgraded TF like this:

pip install –upgrade tensorflow
Start Machine Learning
I did follow your installation instructions from https://machinelearningmastery.com/setup-python-
environment-machine-learning-deep-learning-anaconda/ and ended up with TF version 1.13.1.
The command I ran was:
conda install -c conda-forge tensorflow

I am on Mac, I see possible relevant discussion here on TF2.1 not on conda:

https://github.com/tensorflow/tensorflow/issues/35754

REPLY 
Jason Brownlee April 17, 2020 at 1:31 pm #

Well done!

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 208/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

I use macports myself:

https://machinelearningmastery.com/install-python-3-environment-mac-os-x-machine-
learning-deep-learning/

REPLY 
meryem April 17, 2020 at 1:25 am #

Thank you Jason for the tutoriel.I applied your example to mine by adding dropout and
standarisation of X

X = dataset[:, 0:7]
y = dataset[:, 7]

scaler = MinMaxScaler(feature_range=(0, 1))

X = scaler.fit_transform(X) Start Machine Learning ×
# define the keras model
You can master applied Machine Learning
model = Sequential()
without math or fancy degrees.
model.add(Dense(6, input_dim=7, activation=’relu’)) Find out how in this free and practical course.
model.add(Dropout(rate=0.3))
model.add(Dense(6, activation=’relu’))
Email Address
model.add(Dropout(rate=0.3))
model.add(Dense(1, activation=’sigmoid’))
model.compile(loss=’binary_crossentropy’, optimizer=’adam’,
STARTmetrics=[‘accuracy’]
MY EMAIL COURSE
history=model.fit(X, y, epochs=30, batch_size=30, validation_split=0.1)
_, accuracy = model.evaluate(X, y)
print(‘Accuracy: %.2f’ % (accuracy*100))

shows me an accuracy of 100 which is not normal. to adjust my model, what should I do?

REPLY 
Jason Brownlee April 17, 2020 at 6:22 am #

Well done!

Perhaps evaluate your model using k-fold cross validation.

Start Machine Learning REPLY 

meryem April 17, 2020 at 7:29 am #

yes i followed your example using k-flod cross validation it gives me always 100%

if i move standarisation he gives 83% ,can you guide me please

seed = 4
numpy.random.seed(seed)
dataset = loadtxt(‘data.csv’, delimiter=’,’)
X = dataset[:, 0:7]
Y = dataset[:, 7]
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 209/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

X = sc.fit_transform(X)
kfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=seed)
cvscores = []
for train, test in kfold.split(X,Y):
model = Sequential()
model.add(Dense(12, input_dim=7, activation=”relu”))
model.add(Dropout(rate=0.2))
model.add(Dense(6, activation=”relu”))
model.add(Dropout(rate=0.2))
model.add(Dense(1, activation=”sigmoid”))
model.compile(loss=’binary_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])
model.fit(X[train], Y[train], epochs=20, batch_size=10, verbose=1)
scores = model.evaluate(X[test], Y[test], verbose=0)
Start Machine Learning
print(“%s: %.2f%%” % (model.metrics_names[1], scores[1]*100)) ×
cvscores.append(scores[1] * 100)
print(“%.2f%% (+/- %.2f%%)” % (numpy.mean(cvscores), You numpy.std(cvscores)))
can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Jason Brownlee April 17, 2020 at 7:48 am # Email Address

Nice work! Perhaps your prediction task is trivial?

START MY EMAIL COURSE

REPLY 
meryem April 17, 2020 at 8:08 am #

you are very helpful .

or because I don’t have enough data.So there is nothing else I can use?

REPLY 
Jason Brownlee April 17, 2020 at 1:28 pm #

Perhaps.

Start Machine Learning REPLY 

Farjad Haider April 17, 2020 at 11:00 pm #

Sir Jason you are awesome! Such a nice and easy to comprehend the tutorial. Great Work!

REPLY 
Jason Brownlee April 18, 2020 at 5:57 am #

Thanks!

REPLY 
Joan Estrada April 19, 2020 at 3:51 am #
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 210/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

“Note, the most confusing thing here is that the shape of the input to the model is defined as an
argument on the first hidden layer. This means that the line of code that adds the first Dense
layer is doing 2 things, defining the input or visible layer and the first hidden layer.”

Could you better explain this? Thanks, nice work!

REPLY 
Jason Brownlee April 19, 2020 at 6:02 am #

Yes, see this:

https://machinelearningmastery.com/faq/single-faq/how-do-you-define-the-input-layer-in-keras

Start Machine Learning ×

REPLY 
Hany April 19, 2020 at 9:57 am #
You can master applied Machine Learning
Actually, I cannot thank you enough Dr. Brownlee.
without math or fancy degrees.
Find out how in this free and practical course.
God Bless you.

Email Address

REPLY 
Jason Brownlee April 19, 2020 at 1:14 pm #
START MY EMAIL COURSE

Thanks. You’re very welcome!

REPLY 
Rahim April 22, 2020 at 5:56 am #

Dear Jason
Thanks for this interesting code. I tested this code on pima-indians-diabetes in my computer with keras
2.3.1 but strangely I got the accuracy of 52%. I wonder why there is this much difference between your
accuracy (76%) and mine (52%).

REPLY 
Jason Brownlee April 22, 2020 at 6:10 am #
Start Machine Learning
You’re welcome.

Perhaps try running the example a few times?

REPLY 
Sarmad April 24, 2020 at 8:04 pm #

want to ask: in the first layer(a hidden layer) as we defined input_dim=8 w.r.t features we have
right. and we specify neurons = 12. but concerned is that a thing i studied is that we specify neurons w.r.t
to inputs(features) . Means if we have 8 inputs so neurons will also be 8. but you specified as 12. Why?
2) In any of problem we have to specified a neural network right. it can be any eg: convolutional,
recurrent etc. so which neural network we have choose here. and where?
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 211/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

3) we have to assign weights. so where we have assigned?

please let me know. Thanks sir.

REPLY 
Jason Brownlee April 25, 2020 at 6:44 am #

The first line of the model defines 2 things, the input or visible layer (8) and the first hidden
layer (12). More here:
https://machinelearningmastery.com/faq/single-faq/how-do-you-define-the-input-layer-in-keras

These two things can have different values, they are not directly related.

Yes, this will help you choose models:

https://machinelearningmastery.com/when-to-use-mlp-cnn-and-rnn-neural-networks/
Start Machine Learning ×
Weights are assigned small random numbers automatically when you call compile():
You can master applied Machine Learning
https://machinelearningmastery.com/why-initialize-a-neural-network-with-random-weights/
without math or fancy degrees.
Find out how in this free and practical course.

REPLY 
Sarmad April 26, 2020 at 7:44 pm # Email Address

sir still confuse that as in ML algorithm we specify which algorithm to implement wrt to
scenario like for regression we can choose linear START MY EMAIL
regression COURSE
, logistic regression etc.
now at this time what neural net we have chosen? convoltiona, rntn etc?

REPLY 
Jason Brownlee April 27, 2020 at 5:33 am #

Linear regression is for regression, logistic regression is for classification.

Here are some regression algorithms to try on a regression task:

https://machinelearningmastery.com/spot-check-regression-machine-learning-algorithms-
python-scikit-learn/

REPLY 
Sarmad April 24, 2020 at 8:31 pm #
Start Machine Learning
where are the weights, bias and input values?

REPLY 
Jason Brownlee April 25, 2020 at 6:46 am #

Weights are initialized to small random values when we call compile().

REPLY 
mouna April 26, 2020 at 8:51 pm #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 212/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Hello Jason,

Congratulations fro all the good job, i want to ask you:

How we can know of all epochs the average of training time and validation time for a model?

REPLY 
Jason Brownlee April 27, 2020 at 5:34 am #

You could extrapolate the time of one epoch to the number of epochs you want to train.

REPLY 
Jason Chia April 28, 2020 at 2:41 pm #
Start Machine Learning ×
Hi Jason,
I am very new to deep learning. I understand that you You do model.fit to applied
can master fit the data and model.predict
Machine Learning to
predict the values of the class variable y. However, is itwithout
also possible
math ortofancy
extract the parameter estimate
degrees.
and derive f(X) = y (similar to regression)? Find out how in this free and practical course.

Email Address
REPLY 
Jason Brownlee April 29, 2020 at 6:15 am #
START MY EMAIL COURSE
Perhaps for small models, but it would be a mess with thousands of coefficients. The model
is complex circuit.

REPLY 
Dina April 28, 2020 at 4:34 pm #

Hi JAson, do you have an idea on how to predict price or range of value?

REPLY 
Dina April 28, 2020 at 4:39 pm #

If I use keras model to predict price/range of value, it is possible for me to find the accuracy
of keras model?because in your article only to predict the binary output
Start Machine Learning

REPLY 
Jason Brownlee April 29, 2020 at 6:19 am #

You are describing a regression problem, I recommend starting here:

https://machinelearningmastery.com/regression-tutorial-keras-deep-learning-library-python/

REPLY 
Jason Brownlee April 29, 2020 at 6:17 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 213/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

A prediction range is called a prediction interval, learn more here:

https://machinelearningmastery.com/prediction-intervals-for-machine-learning/

REPLY 
Hume May 5, 2020 at 10:54 am #

thank you for your explanation, i am a beginner for machine learning as well as python.woluld
you please help me in getting the exact CSV data file for predicting the Hepatitis B virus.

REPLY 
Jason Brownlee May 5, 2020 at 1:37 pm #

This will help you locate a dataset: Start Machine Learning ×

https://machinelearningmastery.com/faq/single-faq/where-can-i-get-a-dataset-on-___
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Ababou Nabil May 12, 2020 at 2:01 pm #
Email Address
768/768 [==============================] – 2s 3ms/step
Accuracy: 76.56
START MY EMAIL COURSE

REPLY 
Jason Brownlee May 13, 2020 at 6:21 am #

Well done!

REPLY 
MAHESH MADHUSHAN May 24, 2020 at 11:29 am #

Why didn’t you normalize data? Is not that necessary ? I have seen on some tutorials, they
normalize data for common scale using as –>from sklearn.preprocessing import StandardScaler . What
is the difference that method and your method?

Start Machine Learning

REPLY 
Jason Brownlee May 25, 2020 at 5:43 am #

It can help for some algorithms to normalize or standardize the data input data. Perhaps try
it and see.

REPLY 
Henry Levkine May 26, 2020 at 7:49 am #

Jason,

You are the best!

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 214/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

My name for your program here is “helloDL.py”

I am sure your future book “Hello Deep Learning” will be the most popular on the market.

People need in programs

helloClassification.py
helloRegression.py
helloHelloPrediction.py
helloDogsCats.py
helloFaces.py

and so on!

Thank you for your hard work!

Start Machine Learning ×

Jason Brownlee May 26, 2020 at 1:19 pm # You can master applied Machine Learning REPLY 

without math or fancy degrees.

Thanks. Find out how in this free and practical course.

You can find all of these on the blog, use the search.
Email Address

START MY EMAIL COURSE

REPLY 
Thijs June 12, 2020 at 12:46 am #

Hello,

is there a possibility to access the accuracy of the last epoch? If yes, how can i access this and save it?

Kind regards

REPLY 
Jason Brownlee June 12, 2020 at 6:14 am #

Yes, the history object contains the scores calculated on each epoch:
https://machinelearningmastery.com/display-deep-learning-model-training-history-in-keras/

Start Machine Learning

REPLY 
Krishan June 16, 2020 at 11:08 am #

Accuracy: 82.42
epochs=1500
batch_size=1

I don’t know if what I did was appropriate. Any advise is appreciated.

REPLY 
Jason Brownlee June 16, 2020 at 1:39 pm #

Well done!
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 215/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Saad June 19, 2020 at 9:20 pm #

Hi Jason,

Thanks a lot for this wonderful learning platform.

Why were 12 neurons used in the first hidden layer, what is the criteria behind it? Is it random or there is
an underlying reason/calculation?

(I presumed that the number of neurons in a hidden layer would always be between the number of inputs
and the number of outputs)

Start Machine Learning ×

REPLY 
Jason Brownlee June 20, 2020 at 6:12 am #You can master applied Machine Learning
without math or fancy degrees.
I chose the configuration after a little trial Find
and out
error.
how in this free and practical course.

There is no good theory for configuring neural nets:

Email Address
https://machinelearningmastery.com/faq/single-faq/how-many-layers-and-nodes-do-i-need-in-my-
neural-network

START MY EMAIL COURSE

REPLY 
Paras Memon July 30, 2020 at 9:05 am #

Hello Jason,

I have this shape of training and testing data sets:

xTrain_CN.shape, yTrain_CN.shape, xTest_CN.shape
((320, 56, 6251), (320,), (80, 56, 6251))

I am getting this error: ValueError: Error when checking input: expected dense_20_input to have 2
dimensions, but got array with shape (320, 56, 6251)

Below is the code:

def nn_keras(xTrain_CN, yTrain_CN, xTest_CN):

model = Sequential() Start Machine Learning

model.add(Dense(12, input_dim=6251, activation=’relu’))
model.add(Dense(8, activation=’relu’))
model.add(Dense(1, activation=’sigmoid’))
# compile the keras model
model.compile(loss=’binary_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])
# fit the keras model on the dataset
model.fit(xTrain_CN, yTrain_CN, epochs=150, batch_size=10)
# evaluate the keras model
_, accuracy = model.evaluate(xTrain_CN, yTrain_CN)
print(‘Training Accuracy: %.2f’ % (accuracy*100))

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 216/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

_, accuracy = model.evaluate(xTrain_CN, yTrain_CN)

print(‘Testing Accuracy: %.2f’ % (accuracy*100))

nn_keras(xTrain_CN, yTrain_CN, xTest_CN)

REPLY 
Jason Brownlee July 30, 2020 at 1:44 pm #

A MLP must take 2d data as input (rows and columns) and 1d data as output during
training.

Joanne August 12, 2020 at 1:25 am # Start Machine Learning ×

REPLY 

Hi Jason, You can master applied Machine Learning

without math or fancy degrees.
This is a great tutorial, very easy to understand!! Is there a tutorial for how to add weight and bias into
Find out how in this free and practical course.
our model?

Email Address

REPLY 
Jason Brownlee August 12, 2020 at 6:11 am #
START MY EMAIL COURSE

Thanks!

REPLY 
Luis Cordero August 20, 2020 at 12:05 pm #

Hello, if I have a prediction problem, it is absolutely necessary to scale the input variables to
use the sigmoid or relu activation functions or the one you decide to use?

REPLY 
Jason Brownlee August 20, 2020 at 1:37 pm #

No, but try it and compare results.

Start Machine Learning

REPLY 
Luis Cordero August 20, 2020 at 1:15 pm #

how I can create a configuration that has more than one output, i.e. the output layer has 2 or
more values

REPLY 
Jason Brownlee August 20, 2020 at 1:39 pm #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 217/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Yes, just specify the number of targets in the output layer and prepare your training data accordingly.

I have a tutorial on exactly this written and scheduled – for next week I think.

REPLY 
Luis Cordero September 1, 2020 at 4:29 pm #

what will been name of tutorial to find it

REPLY 
Jason Brownlee September 2, 2020 at 6:24 am #

Right here:
Start Machine Learning ×
https://machinelearningmastery.com/deep-learning-models-for-multi-output-regression/
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Simon Suarez August 30, 2020 at 8:27 am #
Email Address
Hi Jason.

I thank you for the great quality of this article. I am experienced with Machine Learning using Scikit-
START MY EMAIL COURSE
Learn, and reading this post (and some of your previous on the topic) helped me a lot to get into making
Multilayer Perceptrons.
I tested the knowledge I learned here with the Wisconsin Diagnostic Breast Cancer (WDBC) dataset. I
got around 92.965% Accuracy for train and 96.491% for test, only using 3 features (radius, texture,
smoothness) and the following topology:
• Epochs = 250
• Batch_size = 60
• Función de activación = ReLu
• Optimizador = ‘Nadam’

Layer; Number of neurons; Activation function

Input; 3; None
Hidden 1; 4; ReLu
Hidden 2; 4; ReLu
Hidden 3; 2; ReLu
Output; 1; Sigmoid Start Machine Learning

Train and test were splitted using: train_test_split(X, y, test_size=0.33, random_state=42)

Thanks!

REPLY 
Jason Brownlee August 31, 2020 at 5:58 am #

Thanks.

Well done on your results Simon!

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 218/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Berns Buenaobra September 7, 2020 at 7:32 am #

0s 833us/step – loss: 0.4607 – accuracy: 0.7773

REPLY 
Jason Brownlee September 7, 2020 at 8:36 am #

Well done!

REPLY 
Berns Buenaobra September 7, 2020 at 7:37 am #

Second iteration with laptop GPU gives: Start Machine Learning ×

0s 958us/step – loss: 0.4119 – accuracy: 0.8216
You can master applied Machine Learning
Accuracy: 82.16
without math or fancy degrees.
Find out how in this free and practical course.

Ahmed Nuru September 8, 2020 at 5:01 pm # Email Address REPLY 

Hi janson how can predict image forgery and genuine using pretrained deep-learning model
START MY EMAIL COURSE

REPLY 
Jason Brownlee September 9, 2020 at 6:44 am #

Perhaps prepare a dataset of real and fake images and train a binary classification model to
differentiate the two.

Perhaps this tutorial will help you to get started:

https://machinelearningmastery.com/how-to-develop-a-convolutional-neural-network-to-classify-
photos-of-dogs-and-cats/

REPLY 
Fatma Zohra September 11, 2020 at 2:30 am #
Start Machine Learning
Hello Jason ,

Can you please guide me how to make a query and a document as an input in our NN (knowing that
they both are represented by frequency vectors ) ?

REPLY 
Jason Brownlee September 11, 2020 at 6:01 am #

Perhaps start here:

https://machinelearningmastery.com/start-here/#nlp

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 219/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
fatma zohra September 13, 2020 at 2:41 am #

Hi Dr Jason,

Thanks a lot for the reply , the link was useful for me ,
yet i’am still lost a bit since i’am new dealing with NN, actualy i want to calculate the similarity between
the query and the doc using the NN , the inputs are (the TF vector of the doc and TF vector of the query ,
and the output is the similarity (0 if no , 1 if yes ) , i have the idea of my NN but i don’t know from where
to start…
i would be gratful if you could help me (a similar code that i can take as exemple maybe ),

Waiting for your reply..thanks in advance

Start Machine Learning ×

REPLY 
Jason Brownlee September 13, 2020 at 6:10 am #
You can master applied Machine Learning
I think you’re asking about calculating textwithout
similarity.
math If so, sorry degrees.
or fancy I don’t have tutorials on that
topic. Find out how in this free and practical course.

Email Address
REPLY 
fatma zohra September 13, 2020 at 6:38 am #
START MY EMAIL COURSE
yeah , this is what i was asking for , anyways thanks a lot for your tutorials they are very
clear and fruitful..

REPLY 
Jason Brownlee September 13, 2020 at 8:28 am #

You’re welcome.

REPLY 
yibrah fisseha September 22, 2020 at 11:41 pm #

I would like to thank you a lot for your tutorials. can you please guide me on how to evaluate the
model using confusion matrix parameters such as recall, precision, f1 score?
Start Machine Learning

REPLY 
Jason Brownlee September 23, 2020 at 6:40 am #

Yes, here are examples:

https://machinelearningmastery.com/how-to-calculate-precision-recall-f1-and-more-for-deep-
learning-models/

REPLY 
derya September 23, 2020 at 5:03 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 220/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

great tutorial helped a lot !

REPLY 
Jason Brownlee September 23, 2020 at 6:44 am #

Thanks!

REPLY 
Sean H. Kelley September 23, 2020 at 6:38 am #

Hi Jason, thank you very much for this.

I appreciate the extra in depth explanations in the links to other pages.

Start Machine Learning ×
I am wondering how to keep the state of mind. Like you train it while it runs and get a level of accuracy. If
You can master applied Machine Learning
you finally get the level of accuracy from training a certain configuration, how do you keep that
without math or fancy degrees.
configuration/state of mind/level of accuracy of the artificial neural net without having to train it all over
Find out how in this free and practical course.
again?

Can you store a snapshot of that “state of mind” somewhere so that when you have a good working
Email Address
model, you just use that to run new data against or am I still missing some key elements in my
attempting to grasp this?
START MY EMAIL COURSE
Thank you!

REPLY 
Jason Brownlee September 23, 2020 at 6:46 am #

You can save your model and load it later to make predictions, see this tutorial:
https://machinelearningmastery.com/save-load-machine-learning-models-python-scikit-learn/

REPLY 
Sean H. Kelley September 24, 2020 at 12:53 am #

Thank you very much!

Start Machine Learning

REPLY 
Jason Brownlee September 24, 2020 at 6:16 am #

You’re welcome.

REPLY 
Muhammad Asad Arshed October 10, 2020 at 12:34 am #

Awesome blog and technical skill would you like to refer me to some other blogs.

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 221/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee October 10, 2020 at 7:06 am #

Thanks!

REPLY 
Brijesh October 10, 2020 at 5:57 pm #

Hi

Can we use only CSV file format?

Jason Brownlee October 11, 2020 at 6:44 amStart

# Machine Learning ×
REPLY 

No, deep learning can use images, text data, audio

You can data,applied
master almostMachine
anything that can be
Learning
represented with numbers. without math or fancy degrees.
Find out how in this free and practical course.

Email Address REPLY 

imene October 18, 2020 at 4:49 am #

with epoch =10000 and batch-size = 20 a got accuracy

START MY= EMAIL
84% and loss =loss: 0.3434
COURSE

REPLY 
Jason Brownlee October 18, 2020 at 6:12 am #

Well done!

REPLY 
YAŞAR SAİD DERDİMAN December 27, 2020 at 4:12 pm #

this is good but probably, your model’s generalization error is higher. Because more epoch
means more overfitting, Therefore you should use less epoch for any deep learning training.

Start Machine Learning

REPLY 
Jason Brownlee December 28, 2020 at 5:58 am #

Good advice.

REPLY 
imene October 18, 2020 at 4:59 am #

first thanks for your good explanation,

how can i save the trained model to be used for test becaus the trainnig repeat each time i try to execute
the program
tanks.
https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 222/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Jason Brownlee October 18, 2020 at 6:12 am #

Good question, this will show you how:

https://machinelearningmastery.com/save-load-keras-deep-learning-models/

REPLY 
Fatima October 24, 2020 at 5:18 am #

Hi Jason, I applied the Deep Neural Network algorithm(DNN) to do the prediction, It works and
it is perfect, I have a problem in evaluating the predicted results I used (metrics.confusion_matrix), It
gave me this error: Start Machine Learning ×
ValueError: Classification metrics can’t handle a mix of binary and continuous targets
You can master applied Machine Learning
any suggestions to solve the error?
without math or fancy degrees.
note: my class label (outcome variable) is binary (0,1)
Find out how in this free and practical course.
Thanks in advanced
Email Address

Jason Brownlee October 24, 2020 at 7:12 am #START MY EMAIL COURSE REPLY 

See this tutorial:

https://machinelearningmastery.com/how-to-calculate-precision-recall-f1-and-more-for-deep-
learning-models/

REPLY 
K Al October 27, 2020 at 2:53 am #

First of all, please allow me to thank you for this great tutorial and for your valuable time.
I wonder: you trained and evaluated the network on the same data set. Why did not it generate a 100%
accuracy then?

Thanks

Start Machine Learning

REPLY 
Jason Brownlee October 27, 2020 at 6:46 am #

All models have error.

If we get perfect skill/100% accuracy then the problem is likely too simple and machine learning is
not required:
https://machinelearningmastery.com/faq/single-faq/what-does-it-mean-if-i-have-0-error-or-100-
accuracy

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 223/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

REPLY 
Zuzana November 1, 2020 at 11:15 pm #

Hi, great tutorial, everything works, except when trying to add predictions, I get the following
error message. Could you please, help? Thanks a lot.

WARNING:tensorflow:From C:/Users/ZuzanaŠútová/Desktop/RTP
new/3_training_deep_learning/data_PDS/keras_first_network_including_predictions.py:27:
Sequential.predict_classes (from tensorflow.python.keras.engine.sequential) is deprecated and will be
removed after 2021-01-01.
Instructions for updating:
Please use instead:* np.argmax(model.predict(x), axis=-1), if your model does multi-class
classification (e.g. if it uses a softmax last-layer activation).* (model.predict(x) >
0.5).astype("int32"), if your model does binary classification (e.g. if it uses a sigmoid last-layer
activation). Start Machine Learning ×
Warning (from warnings module): You can master applied Machine Learning
File “C:\Users\ZuzanaŠútová\AppData\Roaming\Python\Python38\site-
without math or fancy degrees.
packages\tensorflow\python\keras\engine\sequential.py”,
Findline
out 457
how in this free and practical course.
return (proba > 0.5).astype(‘int32’)
RuntimeWarning: invalid value encountered in greater
Email Address
Traceback (most recent call last):
File “C:\Users\ZuzanaŠútová\AppData\Local\Programs\Python\Python38\lib\site-
packages\pandas\core\indexes\base.py”, line 2895, in get_loc
START MY EMAIL COURSE
return self._engine.get_loc(casted_key)
File “pandas\_libs\index.pyx”, line 70, in pandas._libs.index.IndexEngine.get_loc
File “pandas\_libs\index.pyx”, line 101, in pandas._libs.index.IndexEngine.get_loc
File “pandas\_libs\hashtable_class_helper.pxi”, line 1032, in
pandas._libs.hashtable.Int64HashTable.get_item
File “pandas\_libs\hashtable_class_helper.pxi”, line 1039, in
pandas._libs.hashtable.Int64HashTable.get_item
KeyError: 0

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File “C:/Users/ZuzanaŠútová/Desktop/RTP
new/3_training_deep_learning/data_PDS/keras_first_network_including_predictions.py”, line 30, in
print(‘%s => %d (expected %d)’ % (X[i].tolist(), predictions[i], y[i]))
Start Machine Learning
File “C:\Users\ZuzanaŠútová\AppData\Local\Programs\Python\Python38\lib\site-
packages\pandas\core\frame.py”, line 2902, in __getitem__
indexer = self.columns.get_loc(key)
File “C:\Users\ZuzanaŠútová\AppData\Local\Programs\Python\Python38\lib\site-
packages\pandas\core\indexes\base.py”, line 2897, in get_loc
raise KeyError(key) from err
KeyError: 0

REPLY 
Jason Brownlee November 2, 2020 at 6:40 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 224/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Sorry to hear that, this may help:

https://machinelearningmastery.com/faq/single-faq/why-does-the-code-in-the-tutorial-not-work-for-
me

REPLY 
Zuzana November 2, 2020 at 6:50 am #

I am sorry but none of that helped :/

REPLY 
Julian A Epps November 3, 2020 at 7:58 am #

Start
Where can I find documentation on these keras Machine
functions Learning
that you are using. I don’t know how
×
any of these functions work.
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Jason Brownlee November 3, 2020 at 10:08 am #
Email Address
Good question, here:
https://keras.io/api/
START MY EMAIL COURSE

REPLY 
Umair Rasool November 8, 2020 at 4:42 am #

Hello Sir, i am not actually familiar with ML so someone doing my task for prediction using
raster dataset with python. He just giving final results and CSV file rather than final prediction map as
raster, Could you please guide me ML works like this or he is missing something to generate final map.
Please Response. Thanks

REPLY 
Umair Rasool November 8, 2020 at 4:44 am #

sorry i have a little mistake “final result as CSV file”

Start Machine Learning

REPLY 
Jason Brownlee November 8, 2020 at 6:42 am #

Perhaps this framework will help:

https://machinelearningmastery.com/start-here/#process

REPLY 
Halil November 27, 2020 at 6:09 am #

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 225/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Thank you for this brilliantly explained tutorial ! Actually, I am bored of watching videos which have lots of
boring talks and superficial explanations. I discovered my main resource now

By the way, I guess there is an error here. No?

rounded = [round(x[0]) for x in predictions] —> should be “round(X…..”

REPLY 
Jason Brownlee November 27, 2020 at 6:44 am #

You’re welcome.

There are many ways to round an array.

Start Machine Learning ×

REPLY 
Halil November 30, 2020 at 5:45 am #
You can master applied Machine Learning
without math or fancy degrees.
I mean, that “x” should be “X”. No?
Find out how in this free and practical course.

Email Address
REPLY 
RAJSHREE SRIVASTAVA November 28, 2020 at 4:05 am #
START MY EMAIL COURSE
Hi jason,

Hope you are doing well. I am working on ANN for image classification in google colab. I am getting this
error , can you help me to find solution for this?

InvalidArgumentError: Incompatible shapes: [100,240,240,1] vs. [100,1]

[[node gradient_tape/mean_squared_error/BroadcastGradientArgs (defined at :14) ]]
[Op:__inference_train_function_11972]

Function call stack:

train_function

Waitting for your reply.

REPLY 
Jason Brownlee November 28, 2020 at 6:41 am #
Start Machine Learning
Sorry, I don’t know about colab:
https://machinelearningmastery.com/faq/single-faq/do-code-examples-run-on-google-colab

REPLY 
RAJSHREE SRIVASTAVA November 28, 2020 at 8:14 pm #

Hi jason thanks for your reply.

ok in python I am working on ANN for image classification . I am getting this error , can you help me to
find solution for this?

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 226/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

InvalidArgumentError: Incompatible shapes: [100,240,240,1] vs. [100,1]

[[node gradient_tape/mean_squared_error/BroadcastGradientArgs (defined at :14) ]]
[Op:__inference_train_function_11972]

Function call stack:

train_function

REPLY 
Jason Brownlee November 29, 2020 at 8:12 am #

Sorry, the cause of the error is not clear, you may need to debug your model.

Here are some suggestions:

https://machinelearningmastery.com/faq/single-faq/can-you-read-review-or-debug-my-code
Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees.
REPLY 
Hanem December 17, 2020 at 11:07 am # Find out how in this free and practical course.

Thanks a million, it helped me a lot. Actually, all of your articles are informative and goog guide
for me. Email Address

START MY EMAIL COURSE

REPLY 
Jason Brownlee December 17, 2020 at 12:59 pm #

You’re welcome, I’m happy to hear that!

REPLY 
John Smith December 28, 2020 at 7:58 am #

This was a brilliant tutorial I think what could be done to improve this is adding an example of
actual predictions.

The prediction bit is quite brief I don’t quite have an understanding how to use that array of “predictions”
to actually predict something.

Like if I wanted to feed it some test data and get a prediction how could I do that?
Start Machine Learning
I will consult some of your other helpful guides but would be great to have it all in this 1 tutorial.

REPLY 
John Smith December 28, 2020 at 8:07 am #

I did not have my coffee when I wrote this.

I see now we are passing the original variables back into the model and predicting and printing out
the predication vs actual.

Thanks – you made a great tutorial!

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 227/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

Have a good christmas and new year.

REPLY 
Jason Brownlee December 28, 2020 at 8:19 am #

No problem at all!

I’m happy it helped you kick start your journey with deep learning.

Leave a Reply
Start Machine Learning ×
You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

Email Address

START MY EMAIL COURSE

Name (required)

Email (will not be published) (required)

Website

SUBMIT COMMENT

Welcome!
I'm Jason Brownlee PhD
Start Machine
and I help developers get results with machine learning. Learning
Read more

Never miss a tutorial:

Picked for you:

Your First Deep Learning Project in Python with Keras Step-By-Step

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 228/229
1/2/2021 Your First Deep Learning Project in Python with Keras Step-By-Step

How to Grid Search Hyperparameters for Deep Learning Models in Python With Keras

Regression Tutorial with the Keras Deep Learning Library in Python

Multi-Class Classification Tutorial with the Keras Deep Learning Library

Start Machine Learning ×

You can master applied Machine Learning
How to Save and Load Your Keras Deep Learning Model
without math or fancy degrees.
Find out how in this free and practical course.

Email Address
Loving the Tutorials?

The Deep Learning with Python EBook is

START MY EMAIL COURSE
where you'll find the Really Good stuff.

>> SEE WHAT'S INSIDE

© 2020 Machine Learning Mastery Pty. Ltd. All Rights Reserved.

Address: PO Box 206, Vermont Victoria 3133, Australia. | ACN: 626 223 336.
LinkedIn | Twitter | Facebook | Newsletter | RSS

Privacy | Disclaimer | Terms | Contact | Sitemap | Search

Start Machine Learning

https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/ 229/229