# Regression with PyTorch

## Overview

Teaching:0 min

Exercises:0 minQuestions

How using PyTorch for multi-layered neural network?

Which neural network for regression?

Objectives

Learn how to build multi-layered neural network with PyTorch

Letâ€™s start with a dummy dataset (radonmly generated data) and see how to build a neural network with PyTorch.

# Input data

```
import torch
# 64 samples
N = 64
# 1000 neurons in the input layer
D_in = 1000
# and 10 neurons in the output layer
D_out = 10
# Create random Tensors to hold inputs and outputs
x = torch.randn(N, D_in)
y = torch.randn(N, D_out)
```

We have 64 samples and each sample contains 1000 columns.

The goal is now to find a curve that describes best your dataset with neural network.

# Neural network class definition

Our multi-layered neural network will have one hidden layer (it is still not deep learning!) so in total 3 layers:

- one input layer
- one hidden layer
- one output layer

We still use Linear model for each layer and we take a ReLU (Rectified Linear Unit) Activation Function to be applied one the hidden layer:

```
import torch
class Model(torch.nn.Module):
def __init__(self, D_in, H, D_out):
super().__init__()
self.l1 = torch.nn.Linear(D_in, H)
self.relu = torch.nn.ReLU()
self.l2=torch.nn.Linear(H, D_out)
def forward(self, X):
return self.l2(self.relu(self.l1(X)))
```

# Training

```
# Define 100 neurons in the hidden layer
H = 100
model = Model(D_in, H, D_out)
loss_fn = torch.nn.MSELoss(reduction='sum')
# Use the optim package to define an Optimizer that will update the weights of
# the model for us. Here we will use Adam; the optim package contains many other
# optimization algoriths. The first argument to the Adam constructor tells the
# optimizer which Tensors it should update.
learning_rate = 1e-4
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
for t in range(500):
# Forward pass: compute predicted y by passing x to the model.
y_pred = model(x)
# Compute and print loss.
loss = loss_fn(y_pred, y)
print(t, loss.item())
# Before the backward pass, use the optimizer object to zero all of the
# gradients for the variables it will update (which are the learnable
# weights of the model). This is because by default, gradients are
# accumulated in buffers( i.e, not overwritten) whenever .backward()
# is called. Checkout docs of torch.autograd.backward for more details.
optimizer.zero_grad()
# Backward pass: compute gradient of the loss with respect to model
# parameters
loss.backward()
# Calling the step function on an Optimizer makes an update to its
# parameters
optimizer.step()
```

# Test/validation

```
y_last = model(x)
print(y_last - y)
import matplotlib.pyplot as plt
%matplotlib inline
plt.figure(figsize=(5,5))
plt.plot(y.detach().numpy(), y_last.detach().numpy(), 'o')
plt.xlabel('Targeted y', fontsize=16)
plt.ylabel('Modeled y', fontsize=16)
plt.savefig('Model_validation')
```

## Challenge

Change your neural network and your dataset:

- try different values for the learning rate
- add more samples and generate a training and testing dataset
- Add a new hidden layer

## Key Points

multi-layered neural network