Introduction to Neural Networks with PyTorch

Building tensors is as simple as shown below. Notice the data-types too.

import torch
import numpy as np

def play_with_empty_and_random_tensors( ):
    tensors = []
    #Empty ones
    tensors.append(torch.empty(5, dtype=torch.float32))
    tensors.append(torch.empty(5,5, dtype=torch.int))
    tensors.append(torch.empty(5,5,5, dtype=torch.int))

    #Random ones
    tensors.append(torch.rand(5, dtype=torch.float32))
    tensors.append(torch.rand(5,5, dtype=torch.float32))
    tensors.append(torch.rand(5,5,5, dtype=torch.float32))

    #Tensor with zeros and ones
    tensors.append(torch.zeros(5, dtype=torch.float32))
    tensors.append(torch.ones(5, dtype=torch.float32))
    tensors.append(torch.ones(5,5, dtype=torch.float32))
    tensors.append(torch.ones(5,5,5, dtype=torch.float32))

    #Tensords with arbirary value
    tensors.append(torch.tensor([1,2,3,4,5], dtype=torch.float32))
    tensors.append(torch.tensor([[1,2],[3,4],[5,6]], dtype=torch.float32))
    for tensor in tensors:
        print(tensor,tensor.size())

Either it is slicing numpy or torch arrays, same principles apply. The basic construct is [start:step:end,start:step:end.....] for all possible dimensions. If we want all data from a particular dimension just leave start, end blank and no need to specify step like "[:]".

Reshaping tensors is simple process. As long as the number of items match, we may reshape any array of any size. Example we may reshape an array of dims \(5 \times 5 \times 5\) into \(5 \times 25\) or \(25 \times 5\) or \(1 \times 125\) or \(125 \times 1\). And one important fact while reshaping, if we put one of the dimensions as -1, pytorch automatically replaces it with the corrects size.

import torch
import numpy as np

def lets_reshape_tensors( ):
    my_arr = torch.rand(5,5,5)
    print(my_arr)
    print(my_arr.size())
    my_arr_made_1D = my_arr.view(125)
    print(my_arr_made_1D)
    print(my_arr_made_1D.size())
    my_arr_made_2D = my_arr.view(25,5)
    print(my_arr_made_2D)
    print(my_arr_made_2D.size())
    my_arr_made_2D = my_arr.view(5,25)
    print(my_arr_made_2D)
    print(my_arr_made_2D.size())

    #To let pytorch detect one of the reshaped sizes just put -1 there for example
    my_arr_made_2D = my_arr.view(5,-1) #The minus one will automatically be 25
    print(my_arr_made_2D)
    print(my_arr_made_2D.size())

Converting numpy arrays to torch and torch to numpy is as simple as shown below. Again notice that reference is given to each identifier so all of them point to same location in memory so that changing one will change the other.

import torch
import numpy as np

def lets_convert_numpy_tensor_vice_versa( ):
    #Convert from torch to numpy
    my_arr_torch = torch.rand(5,5,5)
    print(my_arr_torch)
    my_arr_np = my_arr_torch.numpy()
    print(my_arr_np)

    #Convert from numpy to torch
    my_arr_torch_from_np = torch.from_numpy(my_arr_np)
    print(my_arr_torch_from_np)

    #But be very carefull now that my_arr_torch_from_np, my_arr_np, my_arr_torch
    #are all pointing to the same memory location, and mofification of one will modify all of the others

This is one of the most important and basic parts of torch. To understand gradients properly lets create some array and do some calculations as follows:
\(\textbf{x}\) = [2,3,4]
\[y = 2 \times (x+2)^2\] \[z =\frac{1}{3} \sum_{i=1}^{3} \left( 2 \times (x_i+2)^2 \right)\] And
\[\frac{dz}{dx_i} = \frac{1}{3} \times 4(x_i+2) =\frac{4}{3} \times (x_i+2) \] Note that differentiating with respect to \(x_i\) will remove all other x's i.e while differentiating wrt \(x_1\) the sigma will vanish because differentiating the sum portion wrt \(x_2\) or \(x_3\) will give zero. And putting the values of x we get:
\[\frac{dz}{dx} = [5.33,6.67,8]\] And from the code below we get exactly the same. In the code there are two things to take care about. One is to set the "required_grad " property of the tensor to true. Doing this will make pytorch to keep record of all the places with x has been used and hence enable gradient computation. And gradient computation is only for dtypes of floats. ALSO NOTICE THAT ALL THE OPERATIONS ARE DONE ELEMENT WISE And differentiation is only possible when z is a number and not a vector/tensor.

import torch
import numpy as np

def lets_find_gradients():
    x = torch.from_numpy(np.array([2,3,4],dtype=np.float32))
    #We could have also done x = torch.tensor([2,3,4],dype = tensor.float32)
    x.requires_grad = True

    z = 2*(x+2)**2
    z = z.mean()
    print(z)
    z.backward() #This computes dz/dx and puts the values in x.grad
    print(x.grad)
    #to remove the requires_grad and prevent pytorch from 
    # further tracking where x is used and creating gradient functions in our computational graph, we mays simply
    x.requires_grad = False # or we maydo x.requires_grad_(False). Any function in pytorch with trailing "_" modifies
    #the variable to which the function is attached to 
    print(x)

There is another important thing to keep into account. Continuously calling the .backward() function adds on to the preexisting gradient. So before calling the .backward() function the second time we have to call the .grad.zero_() function as shown below:

import torch
import numpy as np

def be_aware_of_gradient_accumulation():  
    x = torch.tensor([1,2,3],dtype = torch.float32,requires_grad=True)
    for i in range(2):
        y = (x**2).sum() #need a scalar
        y.backward()
        print(x.grad)

Doing above will add to the x.grad twice making the result [4,8,12] instead of [2,4,6]. To achieve such result we have to clear the x.grad as follows:

import torch
import numpy as np

def be_aware_of_gradient_accumulation():  
    x = torch.tensor([1,2,3],dtype = torch.float32,requires_grad=True)
    for i in range(2):
        y = (x**2).sum() #need a scalar
        y.backward()

        print(x.grad)
        x.grad.zero_() #zero the gradient

Doing above will clear the grad.

First we provide input and output data:

X = torch.tensor([[1],[2],[3],[4]],dtype=torch.float32)
Y = torch.tensor([[3],[5],[7],[9]],dtype=torch.float32)

Now just the look of it, you should be able to figure out that we are now dealing with vectors. X are Y are \(4 \times 1\) vectors now. Previously we had \(1 \times 4\), which we not normally used. We deal with vectors and matrices in deep learning.X need not be vector, it an be a huge matrix. As it is just a list of inputs. Each input may be as long as it wants. Sometimes the input itself can be a matrix like images in which case the input is converted to vector not X which is itself a list of inputs.

In the data preparation there are a lot of other things that have to be done like normalizing the data, splitting the data that we have into train and test set, etc. The example that we are doing is fairly simple now but the next one will involve everything.

After that we figure out the size of input and output NOT input and output "DATA". We already took care of input and output data as shown above. Now what sized input and prediction are expected are to be specified. In our case both are one and one i.e we want to know what output the system produces when we provide it "5" as input data,and we are expecting 11 as done previously. But things won't be always as simple, later we will see that inputs will themselves be an long array of numbers for example while training with images, inputs will be images i.e a huge list of numbers and output will also be an vector if numbers, while trying to find out whether a person has cancer or not, the input will be a list of "features" i.e input criteria for example age, does he/she consume cigarette, did the parents had cancer, etc. In this case the input will look like:

X = torch.tensor([[25,1,0],[70,0,1],[50,1,1],[61,1,1]],dtype=torch.float32)

In above example in the first data 25 is the age, 1 denotes that the person consumes cigarette, and 0 denotes that there is no heredity of cancer. And a sample output for the above data might be like:

Y = torch.tensor([[1],[0],[0]],dtype=torch.float32)

The output says that the first person i.e has cancer i.e 1 output, second and third do not have cancer. So we now have to train our model for such data where the input size is 3 and output is 1. Again note that the factors like age, consumes cigarette, etc are generally known as features.

After preparing data we prepare a model. First we do the same as we did previously, decide on the input data and the expected output data. This is exactly the same as done above, but the dimensions of arrays are different as shown. But first we have to import the neural network module as:

import torch
import torch.nn as nn

Using the nn module we can now directly used the pre-existing loss functions as:

In the previous section we used the nn.Linear() model provide by pytorch. Since out example was quite simple, this did the job, but in real life case we have to build a custom model deriving from nn.Module as shown below:

class linear_regression(nn.Module):
    def __init__(self,input_size,output_size) -> None:
        super().__init__()
        #define layers here
        self.linear_regression = nn.Linear(input_size,output_size)

        def forward(self,x):
            return self.linear_regression(x)

Below we have coded first without using a custom model, then finally with a custom model

Also note that the class that we custom built is callable i.e nn.Module must have __call__ method, which calls the forward method. So we also need to defined the forward method. And here in the forward method we again did the same thing: used nn.Linear class which is again a callable class. now putting above code altogether:

loss = nn.MSELoss()

Notice that it is exactly the loss function that we used previously i.e mean squared error loss function but note that nn.MSELoss is a class with __call__ method implemented inside it so the class is callable. The class expects the tensors of both the expected output and real output as inputs as we have done previously. i.e to call the loss function defined as __call__ of the nn.MSELoss class we need to just loss(Yp,Y) where Yp is predicted, and Y is just the real outputs.

Now instead of manually updating the gradients we use torch optimizer as follows. Notice that even though we already know that we need not manually declare weights and biases, but for now lets first do things manually then we will do everything using torch:

optimizer = torch.optim.SGD([w,b],lr=learning_rate)

Now the part that we did manually of gradient descent is done by optimizer now. SGD stands for Stochastic Gradient Descent. Stochastic just means probabilistic i.e well modeled by random probability distribution.

Now see that in the training step we used the loss function, the only difference from previous case is that the loss function take Y and predicted Y, instead of out defined loss function which took X and Y. And then in the step where is updated our weights and biases there is simply optimzer.step() we does the same thing. And finally setting the grad to zero has to be done as done previously but only in a different way using optimizer.

for i in range(n_iters):
        error = loss(Y,predicted(X))
        error.backward()
        optimizer.step()
        optimizer.zero_grad()
        print(f"w:{w:.3f} and b:{b:.3f} and loss:{error:.3f} grad_w = {w.grad} grad_b = {b.grad}")

Now with everything in place lets check our code:

import torch
import torch.nn as nn

def do_everything_in_torch():
    X = torch.tensor([[1],[2],[3],[4]],dtype=torch.float32)
    Y = torch.tensor([[3],[5],[7],[9]],dtype=torch.float32)
    learning_rate = 0.1
    n_iters = 100

    w = torch.tensor(0,dtype =torch.float32, requires_grad=True)
    b = torch.tensor(0,dtype=torch.float32,requires_grad=True)        

    def forward(x):
        return w*x+b

    def predicted(X):
        return forward(X)

    loss = nn.MSELoss() #This class is callable
    optimizer = torch.optim.SGD([w,b],lr=learning_rate)
    print(f"Before training f(5) = {forward(5)}")

    for i in range(n_iters):
        error = loss(Y,predicted(X))
        error.backward()
        optimizer.step()
        optimizer.zero_grad()
        print(f"w:{w:.3f} and b:{b:.3f} and loss:{error:.3f} grad_w = {w.grad} grad_b = {b.grad}")


    print(f"After training iterations f(5) = {forward(5):.3f}")  

Before training f(5) = 0.0
w:3.500 and b:1.200 and loss:41.000 grad_w = 0.0 grad_b = 0.0
After many other steps…
w:2.005 and b:0.986 and loss:0.000 grad_w = 0.0 grad_b = 0.0
After training iterations f(5) = 11.010

As you can see we removed the loss function, the gradient updating part with the ones provided by torch. And the outputs are fine but what you will realize is grad_w and grad_b are zeros all the time. It looks like the loss function/class is doing things in its own way inside so that computing loss and doing error.backward() did not put the gradients in w.grad and b.grad.

Now we further remove the manual declaration of weights and biases, and also remove the manual forward function and instead just used the model provided by torch as shown:

First we remove the manual declaration of weight and biases, as the pytorch model knows how to generate weights and biases based on the number of inputs and outputs expected, the model type and number of layers in the model. Pytorch automatically generates and initializes the weights and biases to some random value.

Now we first create a model as shown

In summary we do

1. Prepare Data
2. Define the model
3. Choose loss,function and optimizer
4. Train the model
   • Forward Pass
   • Backward Pass
   • Update weights

Now combining everything we learned so far:

import torch
import torch.nn as nn


def do_completely_everything_in_torch():
    X = torch.tensor([[1],[2],[3],[4]],dtype=torch.float32)
    Y = torch.tensor([[3],[5],[7],[9]],dtype=torch.float32)
    learning_rate = 0.1
    n_iters = 100

    X_test = torch.tensor([5],dtype=torch.float32)

    input_size = 1
    output_size = 1
    model = nn.Linear(input_size,output_size)

    loss = nn.MSELoss() #This class is callable
    optimizer = torch.optim.SGD(model.parameters(),lr=learning_rate)

    print(f"Before training f(5) = {model(X_test).item():.3f}")

    for i in range(n_iters):
        Y_pred = model(X) #This is forward pass
        error = loss(Y,Y_pred) #Error computation
        error.backward() # This is back propagation
        optimizer.step() # This is updating of wights and biases
        optimizer.zero_grad() # setting the gradients to zero as they will accumulate if we don't do so

        [w,b] = model.parameters()
        print(w,b)
        print(f"w:{w.item():.3f} and b:{b.item():.3f} and loss:{error:.3f} grad_w = {w.grad.item():.3f} grad_b = {b.grad.item():.3f}")


    print(f"After training iterations f(5) = {model(X_test).item():.3f}")  

Before training f(5) = 0.102
Parameter containing:
tensor([[3.2461] ], requires_grad=True) Parameter containing:
tensor([1.7379], requires_grad=True)
w:3.246 and b:1.738 and loss:37.377 grad_w = 0.000 grad_b = 0.000
After more iterations:…
Parameter containing:
tensor([[1.9953] ], requires_grad=True) Parameter containing:
tensor([1.0139], requires_grad=True)
w:1.995 and b:1.014 and loss:0.000 grad_w = 0.000 grad_b = 0.000
After training iterations f(5) = 10.990

Lets load the breast cancer data set as follows:

breast_cancer_data = datasets.load_breast_cancer()

Above data set gives a dictionary of data containing the input data, the target data which is the list of 0 and 1s which signify no cancer and cancer. To see how the data inside looks like lets print the data and target and see:

import torch
import numpy as np
import torch.nn as nn
from sklearn import datasets
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split


def logis_regression():
    breast_cancer_data = datasets.load_breast_cancer()

    data = breast_cancer_data.data
    target = breast_cancer_data.target

    print(f"Shape of data:{data.shape}")
    print(data)

    print(f"Shape of target:{target.shape}")
    print(target)

logis_regression()

Shape of data:(569, 30)
[[1.799e+01 1.038e+01 1.228e+02 ... 2.654e-01 4.601e-01 1.189e-01]
 [2.057e+01 1.777e+01 1.329e+02 ... 1.860e-01 2.750e-01 8.902e-02]
 [1.969e+01 2.125e+01 1.300e+02 ... 2.430e-01 3.613e-01 8.758e-02]
 ...
 [1.660e+01 2.808e+01 1.083e+02 ... 1.418e-01 2.218e-01 7.820e-02]
 [2.060e+01 2.933e+01 1.401e+02 ... 2.650e-01 4.087e-01 1.240e-01]
 [7.760e+00 2.454e+01 4.792e+01 ... 0.000e+00 2.871e-01 7.039e-02]]
Shape of target:(569,)
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 1 0 0 0 0 0 0 0 0 1 0 1 1 1 1 1 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 0 0
 1 0 1 0 0 1 1 1 0 0 1 0 0 0 1 1 1 0 1 1 0 0 1 1 1 0 0 1 1 1 1 0 1 1 0 1 1
 1 1 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 1 0 1 0 0 1 0 0 1 1 0 1 1 0 1 1 1 1 0 1
 1 1 1 1 1 1 1 1 0 1 1 1 1 0 0 1 0 1 1 0 0 1 1 0 0 1 1 1 1 0 1 1 0 0 0 1 0
 1 0 1 1 1 0 1 1 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 1 0 1 0 0 0 0 1 1 0 0 1 1
 1 0 1 1 1 1 1 0 0 1 1 0 1 1 0 0 1 0 1 1 1 1 0 1 1 1 1 1 0 1 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 1 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1
 1 0 1 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 0 1 0 1 1 1 1 0 0 0 1 1
 1 1 0 1 0 1 0 1 1 1 0 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 0 0 1 0 0
 0 1 0 0 1 1 1 1 1 0 1 1 1 1 1 0 1 1 1 0 1 1 0 0 1 1 1 1 1 1 0 1 1 1 1 1 1
 1 0 1 1 1 1 1 0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0 0 1 0 1 1 1 1 1 0 1 1
 0 1 0 1 1 0 1 0 1 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 0 1
 1 1 1 1 1 1 0 1 0 1 1 0 1 1 1 1 1 0 0 1 0 1 0 1 1 1 1 1 0 1 1 0 1 0 1 0 0
 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 0 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
 1 1 1 1 1 1 1 0 0 0 0 0 0 1]

So we can clearly see that there are 569 data each with 30 features, so out input size is 30. Likewise the output is a binary value of 0 or 1 so the output size is naturally 1 and the neural net if no hidden layers are there will look like:

And the size of bias will be 1 and and of weights will be \(1 \times 30\). i.e

Input is obviously:

bias is simply: \[b = [b_1]\]

Lets further prepare the data, lets put the data inside X and target i.e output inside Y as

X = breast_cancer_data.data
Y = breast_cancer_data.target  

Then lets get the number of samples and features as:

n_samples,n_features = X.shape

Then now lets split the data into test and train set:

X_train,X_test,Y_train,Y_test = train_test_split(X,Y,test_size=0.2,random_state=1234)

Lets see inside the train and test data:

import torch
import numpy as np
import torch.nn as nn
from sklearn import datasets
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split


def logis_regression():
    breast_cancer_data = datasets.load_breast_cancer()
    X = breast_cancer_data.data
    Y = breast_cancer_data.target

    n_samples,n_features = X.shape

    X_train,X_test,Y_train,Y_test = train_test_split(X,Y,test_size=0.2,random_state=42)
    print(X_train.shape,X_test.shape,Y_train.shape,Y_test.shape)
    print(f"X-Train:{X_train}")   
    print(f"X-Test:{X_test}")
    print(f"Y-Train:{Y_train}")   
    print(f"Y-Test:{Y_test}")
logis_regression()

(455, 30) (114, 30) (455,) (114,)
X-Train:[[9.029e+00 1.733e+01 5.879e+01 ... 1.750e-01 4.228e-01 1.175e-01]
 [2.109e+01 2.657e+01 1.427e+02 ... 2.903e-01 4.098e-01 1.284e-01]
 [9.173e+00 1.386e+01 5.920e+01 ... 5.087e-02 3.282e-01 8.490e-02]
 ...
 [1.429e+01 1.682e+01 9.030e+01 ... 3.333e-02 2.458e-01 6.120e-02]
 [1.398e+01 1.962e+01 9.112e+01 ... 1.827e-01 3.179e-01 1.055e-01]
 [1.218e+01 2.052e+01 7.722e+01 ... 7.431e-02 2.694e-01 6.878e-02]]
X-Test:[[1.247e+01 1.860e+01 8.109e+01 ... 1.015e-01 3.014e-01 8.750e-02]
 [1.894e+01 2.131e+01 1.236e+02 ... 1.789e-01 2.551e-01 6.589e-02]
 [1.546e+01 1.948e+01 1.017e+02 ... 1.514e-01 2.837e-01 8.019e-02]
 ...
 [1.152e+01 1.493e+01 7.387e+01 ... 9.608e-02 2.664e-01 7.809e-02]
 [1.422e+01 2.785e+01 9.255e+01 ... 8.219e-02 1.890e-01 7.796e-02]
 [2.073e+01 3.112e+01 1.357e+02 ... 1.659e-01 2.868e-01 8.218e-02]]
Y-Train:[1 0 1 1 1 0 1 1 1 0 1 0 0 1 1 0 0 0 1 1 1 0 1 1 1 0 1 0 1 1 0 1 0 0 0 1 0
 1 1 1 1 0 0 1 1 1 1 1 1 1 0 1 1 1 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 1 1
 0 1 0 1 1 1 1 0 1 1 0 1 1 1 0 1 0 0 1 1 1 0 1 1 1 1 0 1 1 1 1 1 0 1 0 0 1
 1 0 1 1 1 1 1 1 1 0 0 0 1 1 0 1 1 0 1 0 1 0 1 0 1 1 0 1 1 1 0 1 0 1 0 1 0
 1 1 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 1 1 1 1 1 1 1 0 1 1 1 0 1 0 1 1 1 0 1
 0 0 1 1 0 1 0 0 0 1 1 1 0 1 1 0 1 0 1 1 1 0 1 0 1 1 0 0 1 1 0 1 0 0 1 0 0
 1 1 0 0 0 1 1 1 1 0 1 0 0 0 0 1 1 1 1 1 1 1 1 0 0 1 1 0 1 1 1 1 1 0 1 1 0
 0 1 0 1 0 1 1 1 1 1 1 0 1 1 0 1 1 1 1 1 1 0 1 1 0 1 0 0 0 1 0 1 1 0 0 0 1
 1 1 1 1 1 1 0 1 1 1 0 1 1 0 0 1 0 1 0 0 1 1 0 1 0 0 1 0 0 1 1 0 1 0 1 1 0
 1 1 0 0 0 1 1 1 0 0 1 0 0 1 1 1 0 1 0 0 0 0 1 1 0 1 1 0 0 0 0 0 0 1 1 1 1
 1 1 1 0 0 0 0 1 1 1 1 0 1 0 1 1 1 1 1 0 0 0 1 1 0 1 1 0 0 0 0 1 1 0 0 1 1
 1 0 0 0 1 1 0 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 0 1 1 0 1 1 0 0 0
 1 0 0 1 0 1 1 1 1 0 1]
Y-Test:[1 0 0 1 1 0 0 0 1 1 1 0 1 0 1 0 1 1 1 0 0 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 0
 1 0 1 1 0 1 1 1 1 1 1 1 1 0 0 1 1 1 1 1 0 0 1 1 0 0 1 1 1 0 0 1 1 0 0 1 0
 1 1 1 0 1 1 0 1 0 0 0 0 0 0 1 1 1 1 1 1 1 1 0 0 1 0 0 1 0 0 1 1 1 0 1 1 0
 1 1 0]

Now most of the above details are self explanatory like the 0.2 i.e the test data is 0.2 times the total i.e 114 out of 569. But the only thing that need explanation is random_state. This is simply the seed need for random number generation. If we donot provide it, each time we run the data the sample will be different. Just for consistency of testing we are now making it fixed so that we always get the same train and test data.

Now lets normalize the data and see how the looks like:

import torch
import numpy as np
import torch.nn as nn
from sklearn import datasets
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split


def logis_regression():
    breast_cancer_data = datasets.load_breast_cancer()
    X = breast_cancer_data.data
    Y = breast_cancer_data.target

    n_samples,n_features = X.shape

    X_train,X_test,Y_train,Y_test = train_test_split(X,Y,test_size=0.2,random_state=42)

    sc = StandardScaler()

    X_train = sc.fit_transform(X_train)
    X_test = sc.transform(X_test)
    print(X_train.shape,X_test.shape)
    print(f"X_train:{X_train}")
    print(f"X_test:{X_test}")

logis_regression()

(455, 30) (114, 30)
X_train:[[-1.44075296 -0.43531947 -1.36208497 ...  0.9320124   2.09724217
   1.88645014]
 [ 1.97409619  1.73302577  2.09167167 ...  2.6989469   1.89116053
   2.49783848]
 [-1.39998202 -1.24962228 -1.34520926 ... -0.97023893  0.59760192
   0.0578942 ]
 ...
 [ 0.04880192 -0.55500086 -0.06512547 ... -1.23903365 -0.70863864
  -1.27145475]
 [-0.03896885  0.10207345 -0.03137406 ...  1.05001236  0.43432185
   1.21336207]
 [-0.54860557  0.31327591 -0.60350155 ... -0.61102866 -0.3345212
  -0.84628745]]
X_test:[[-0.46649743 -0.13728933 -0.44421138 ... -0.19435087  0.17275669
   0.20372995]
 [ 1.36536344  0.49866473  1.30551088 ...  0.99177862 -0.561211
  -1.00838949]
 [ 0.38006578  0.06921974  0.40410139 ...  0.57035018 -0.10783139
  -0.20629287]
 ...
 [-0.73547237 -0.99852603 -0.74138839 ... -0.27741059 -0.3820785
  -0.32408328]
 [ 0.02898271  2.0334026   0.0274851  ... -0.49027026 -1.60905688
  -0.33137507]
 [ 1.87216885  2.80077153  1.80354992 ...  0.7925579  -0.05868885
  -0.09467243]]

Now lets build a custom model. Here we will be using sigmoid function to squish the output to a range of 0 and 1.

class logistic_regression(nn.Module):
    def __init__(self,input_size,output_size):
        super().__init__()
        self.linear_model = nn.Linear(input_size,output_size)

    def forward(self,X):
        return torch.sigmoid(self.linear_model(X))    

model = logistic_regression(n_features,1)

Lets talk about a few things here, apart from most of stuff, the only thing new here is the sigmoid function. Sigmoid function, ReLu, etc are a family of functions known as activation functions. They are required to squish the output data between 0 and 1, and why do that? Because in most cases out outputs have to be zero and one. For example lets take an example of hand written digit recognition which we shall be doing later, for an input image the ouput we expect is a digit between 0 and 9, but how do we represent the output, we donot just see the output and expect it to be 0 to 9,but instead we have a vector of 10 values. The output vector with a value of 1 in the first value and 0 in the rest is a zero, and like wise the output vector with a value of 1 in the second value and rest zeros is a 1 and so on. While preparing the data, we encode like this. So activation functions are integral part of NN.

\[S = \frac{1}{1+e^{-x}}\]

which looks like:

Above sigmoid is not the only activation function, in fact it is not even one of the good ones. Later we will be using other activation functions.

In the previous example we used Mean Squared Loss function (which is what we should use we we expect a real number output whose value could be any number), now lets use and understand another loss function, and also the reason why we are shifting from MSE to the new BCE i.e Binary Cross Entropy Loss. When the output of our net is a binary number 0 or 1, or a value between 0 or 1, which is true in our case, where a person either has or doesn't have cancer based on the 30 features input. And since we know that the output has to be between 0 and 1, but our neural net doesn't necessarily ouput a value between 0 and 1 we used the sigmoid function that squished the data between 0 and

loss_fun = nn.BCELoss()

We have our same old optimizer,and lets keep the learning rate to 0.01

learning_rate = 0.01
optimizer = torch.optim.SGD(model.parameters(),lr = learning_rate)       

Remember that there are 5 steps to training:

1. Forward pass
2. Loss computation
3. Backward pass
4. Weights Updating
5. Reset weights grad to 0 to prevent accumulation

epochs = 100

for epoch in range(epochs):
    #forward pass
    Y_pred = model(X_train)

    #Loss calculation
    loss = loss_fun(Y_pred,Y_train)

    #back prop
    loss.backward()

    #update weights
    optimizer.step()

    #set grads to zero to prevent acculumation
    optimizer.zero_grad()

    #print loss every 10th step
    if ((epoch+1) % 10==0):
        print(f"epoch:{epoch+1} Loss:{loss.item():.4f}")

Now that our model is trained, we now used our test data to see, how well our model performs.

with torch.no_grad():
    Y_test_predicted = model(X_test)

    #Rounding up because the outputs are either 0 or 1
    Y_test_predicted = Y_test_predicted.round()

    #Compute how many predicted matched the test_output set and divide it by total number of cases to get fractional match
    matched_fraction = Y_test_predicted.eq(Y_test).sum()/float(Y_test.shape[0])
    print(f"Accuracy:{matched_fraction * 100}%")

Putting everything together:

Final Code:

import torch
import numpy as np
import torch.nn as nn
from sklearn import datasets
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split


def logis_regression():
    breast_cancer_data = datasets.load_breast_cancer()
    X = breast_cancer_data.data
    Y = breast_cancer_data.target

    n_samples,n_features = X.shape

    X_train,X_test,Y_train,Y_test = train_test_split(X,Y,test_size=0.2,random_state=42)

    sc = StandardScaler()

    X_train = sc.fit_transform(X_train)
    X_test = sc.transform(X_test)

    #Convert numpy arrays to torch tensors
    X_train = torch.from_numpy(X_train.astype(np.float32))
    Y_train = torch.from_numpy(Y_train.astype(np.float32))
    X_test = torch.from_numpy(X_test.astype(np.float32))
    Y_test = torch.from_numpy(Y_test.astype(np.float32))

    #Reshape Y to make it a vector
    Y_train = Y_train.view(Y_train.shape[0],1)
    Y_test = Y_test.view(Y_test.shape[0],1)

    class logistic_regression(nn.Module):
        def __init__(self,input_size,output_size):
            super().__init__()
            self.linear_model = nn.Linear(input_size,output_size)

        def forward(self,X):
            return torch.sigmoid(self.linear_model(X))    

    model = logistic_regression(n_features,1)
    loss_fun = nn.BCELoss()

    learning_rate = 0.01
    optimizer = torch.optim.SGD(model.parameters(),lr = learning_rate)       

    epochs = 100

    for epoch in range(epochs):

        #forward pass
        Y_pred = model(X_train)

        #Loss calculation
        loss = loss_fun(Y_pred,Y_train)

        #back prop
        loss.backward()

        #update weights
        optimizer.step()

        #set grads to zero to prevent acculumation
        optimizer.zero_grad()

        #print loss every 10th step
        if ((epoch+1) % 10==0):
            print(f"epoch:{epoch+1} Loss:{loss.item():.4f}")

    #lets see how our 
    #lets check the accuracy of model by comparing the test data
    with torch.no_grad():
        Y_test_predicted = model(X_test)

        #Rounding up because the outputs are either 0 or 1
        Y_test_predicted = Y_test_predicted.round()
        matched_fraction = Y_test_predicted.eq(Y_test).sum()/float(Y_test.shape[0])
        print(f"Accuracy:{matched_fraction * 100}%")

logis_regression()

Note that there is torch.no_grad then end, why? Because we don't want further computation like Test prediction to be the part of torch's computational graph. So be careful with this. Also be well informed on the type of date that you are providing as input, the ones as output, and the output generated by torch. While providing input X is a 2D array each element in the array consisting of anther 1D array with a list of numbers known as features and this determines the size of inputs in out neural net. Likewise in our case above where the ouput was either 0 or 1 for each input, the Y was a 1D array of 0s and 1s, and it had to be reshaped to make it 2D.

Finally while computing the output from the test data using our trained model, again note that the Y_test_predicted is a 2D array of outputs. In general we are dealing is vectors is all I want to say. A vector is a 2D array i.e a column array eg \( 4 \times 1\).

Do install the necessary packages:

pip install pandas
pip install openpyxl

Make sure that "xlrd" package is uninstalled, or it is upto date else the syntax will be from python 2 and you will get syntax error.

import pandas as pd

You can just read the excel file as follows:

import pandas as pd
df = pd.read_excel('V2.xlsx')
print(df)

To select a particular column as index column i.e now the row can be indexed with that column value. index_col parameter could be provided: usecols=[] can be provided to select particular columns only.

import pandas as pd
df = pd.read_excel('V2.xlsx',index_col = 2)
df = pd.read_excel('V2.xlsx',usecols=[1,2,3])
print(df)

Import the columns as a particular datatype as follows: Also note clearly that pandas' data frames are simply dictionaries which can be accessed as follows. Also note that any column can to sent to numpy array. Also notice the slicing using loc and iloc. Realize that iloc is even powerful slicing than normal numpy array it can accept values like [0:4,1:4] representing 0,1,2,3 rows and 1,2,3 cols. It also accepts arrays i.e [[1,2,3],[4,5,6]] and so on.

import pandas as pd
df = pd.read_excel('V2.xlsx',dtype = {"Cspeed":float,"lnWaitingTime":float})
print(df)
print("Only Cspeed")
speed = df["Cspeed"]

print("Cspeed in numpy")
speed_numpy = speed.to_numpy(speed)
print(speed_numpy)

print("\n\ndf sliced i.e from rows 0 to 2")
print(df.iloc[0:3,])

Manually creating data frame is as simple as:

import pandas as pd

mydataset = {
  'cars': ["BMW", "Volvo", "Ford"],
  'passings': [3, 7, 2]
}

myvar = pd.DataFrame(mydataset)

print(myvar)

Simple queries like "where col_A > col_B" etc can be achieved as simply as follows. Just refer to the column names as shown. Also note that the use & for and and | for or i.e the bit wise operators.

import pandas as pd
df = pd.read_excel('V2.xlsx')
print(df[df.Cspeed>0.5])
print(df[(df.Cspeed>0.5) & (df.Width > 20.0)])

Adding new columns in pandas is as simple as adding new as adding new item in a dictionary. Simply

df["Name_of_new_column"] = [<some_data>]
df.insert(2, "Age", [21, 23, 24, 21], True)

Note that we can insert columns the second way too where we may specify the location of column to insert, name of column, the data. The coolest feature is that of excel i.e adding columns based on already existing column as:

The syntax looks like this:

DataFrame.apply(func, axis=0, raw=False, result_type=None, args=(), **kwargs)

As implemented in pandas:

df['Discounted_Price'] = df.apply(lambda row: row.Cost - (row.Cost * 0.1), axis = 1)

Above code will add new column "Discounted_Price" as the end using the formula as shown. The axis is 0 to apply function to each column and 1 to apply function to each row. For more info refer to this link. Normally we are interested in 1 as in most operation done in excel.