Getting started with neural networks (NNs)
0 likes22 views
The document provides an overview of neural networks, specifically using Keras and TensorFlow, detailing the anatomy of neural networks, including layers, models, loss functions, and optimizers. It explains the setup for a deep learning workspace, emphasizing the advantages of using GPUs and Jupyter notebooks. Additionally, it outlines introductory steps for using TensorFlow, including tensor manipulation and a regression example using the Boston housing price dataset.
![1) Layers
— A layer is a data-processing module that takes as input one or more tensors and outputs one
or more tensors.
— Different layers are appropriate for different tensor formats and different types of data
processing.
▪ For example, simple vector data, stored in 2D tensors (samples, features)
✓ often processed by densely connected (aka fully connected or dense) layers.
✓ For Example, (samples=1000 houses, features=[size, no. rooms, location]).
▪ Sequence data (text, audio, etc.), stored in 3D tensors (samples, timesteps, features)
✓ typically processed by recurrent layers.
✓ For Example, (samples=500 text documents, timesteps=100 words, features=dimensionality of the
word embeddings (e.g., 300-dimensional word embeddings)).
▪ Image data, stored in 4D tensors (samples, height, width, channels)
✓ usually processed by 2D convolution layers (Conv2D) to extract features.
✓ For Example, (samples=10,000 images, height=28 pixels, width=28 pixels, channel=1).](https://image.slidesharecdn.com/gettingstartedwithneuralnetworks-241124094100-735fdf59/85/Getting-started-with-neural-networks-NNs-5-320.jpg)
Getting started with neural networks (NNs)
- 1. Getting started with neural networks using Keras andTensorflow By Mohamed Gamal Deep Learning
- 2. ▪ Anatomy of a neural network. ▪ A closer look at TensorFlow, Keras and their relationship. ▪ Setting up deep learning workspace. ▪ Example. Contents
- 3. Anatomy of a neural network Training a neural network revolves around the following objects: 1) Layers, which are combined into a network (or model). 2) The input data and corresponding targets. 3) The loss function, which defines the feedback signal used for learning. 4) The optimizer, which determines how learning proceeds.
- 5. 1) Layers — A layer is a data-processing module that takes as input one or more tensors and outputs one or more tensors. — Different layers are appropriate for different tensor formats and different types of data processing. ▪ For example, simple vector data, stored in 2D tensors (samples, features) ✓ often processed by densely connected (aka fully connected or dense) layers. ✓ For Example, (samples=1000 houses, features=[size, no. rooms, location]). ▪ Sequence data (text, audio, etc.), stored in 3D tensors (samples, timesteps, features) ✓ typically processed by recurrent layers. ✓ For Example, (samples=500 text documents, timesteps=100 words, features=dimensionality of the word embeddings (e.g., 300-dimensional word embeddings)). ▪ Image data, stored in 4D tensors (samples, height, width, channels) ✓ usually processed by 2D convolution layers (Conv2D) to extract features. ✓ For Example, (samples=10,000 images, height=28 pixels, width=28 pixels, channel=1).
- 6. 1) Layers (Cont.) — Some layers in a neural network are stateless, meaning they do not have any internal state or memory. They perform computations solely based on the input data. — However, many layers have a state, which refers to the layer's weights. ▪ These weights are learned during the training process using techniques like stochastic gradient descent (SGD). ▪ The weights contain the network's knowledge and are adjusted iteratively during training to minimize the loss function. ▪ Often processed by densely connected (aka fully connected or dense) layers.
- 7. 2) Models (networks of layers) — Network topologies, some common ones include the following ▪ Two-branch networks ▪ Multihead networks ▪ Inception blocks — The outputs are often combined in some way to produce the final output of the network.
- 8. 3) Loss functions and optimizers — Once the network architecture is defined, you still have two more things ▪ Loss function (objective function): ✓ The quantity that will be minimized during training. ✓ It represents a measure of success for the task at hand. ▪ Optimizer: ✓ Determines how the network will be updated based on the loss function. ✓ It implements a specific variant of stochastic gradient descent (SGD) — e.g., RMSprop, Adam … etc.
- 9. What is TensorFlow? ▪ Python–based , free, open-source machine learning platform • Developed primary by Google. • Computes automatically the gradient (i.e., the derivative) of any differentiable expression (highly suitable for DL). • Runs on CPU, GPU, TPU. • Computations can be distributed to many machines. • Supports other programming languages (e.g., C++, JavaScript, etc.).
- 10. What is Keras? ▪ Keras is a deep leaning API for python, built on top of TensorFlow. • GPU: Graphics Processing Unit. • CPU: Central Processing Unit. • TPU: Tensor Processing Unit (by Google).
- 11. Setting up a deep learning workspace — Prefer using GPU instead of CPU when possible (5-10x faster). ▪ GPU options: ✓ Buy one ($1500 – $2000). ✓ Rent one (AWS , Google cloud , paperspace … etc.) for $250 per hour. ✓ Use free one (Google Colab or paperspace Gradient). — Use Unix (i.e., Linux) for Compatibility with Deep Learning Libraries.
- 12. 1) Jupyter notebooks — The preferred way to run deep learning experiments. — A way to edit code in your browser. — Can add markdown text for annotation/comments. — Can run blocks of code independently while maintaining the current state in memory (no need to rerun the whole script).
- 13. 2) Google Colab — Free Jupyter notebook service. — No installation needed. — Runs entirely on the cloud. — Free (but limited) GPU/TPU runtime. Link: https://colab.research.google.com
- 14. First Steps with TensorFlow — Low–level tensor manipulation ▪ Tensors/Variables ▪ Tensor Operations (Relu, matmul) ▪ Backpropagation (TF Gradient Tape) — High–level (i.e., Keras APIs) ▪ Layers ▪ Loss function ▪ Optimizer ▪ Metrics ▪ Training loop
- 15. Constant Tensors/Variables ▪ All ones tensors ▪ All zeros tensors
- 16. Random Tensors
- 17. NumPy arrays are assignable
- 18. ▪ Assigning value to TensorFlow variable ▪ Assigning value to a subset to a TensorFlow variable ▪ Creating a variable TensorFlow
- 19. Tensor operations: Doing Math in TensorFlow ▪ A few Basic math operations
- 20. The Boston Housing Price dataset We’ll attempt to predict the median price of homes in a given Boston suburb in the mid-1970s, given data points about the suburb at the time, such as the crime rate, the local property tax rate, and so on. Regression Example