I am fairly comfortable building Deep Neural Networks (DNN) with Keras library and infact I built a CNN model - Plantmd to predict plant diseases as part of Insight Data Science fellowship program. However as people often say, it’s always important to learn a new language/program to see how it differs from the language/program that you are comfortable. So with this in mind, I have decided to take spend time to do the exciting Deep Learning with PyTorch: Zero to GANs workshop on Jovian.ml.
As part of the first assignment, we were asked to pick out five
torch.Tensor functions in Pytorch and use them with examples. So here are my five ‘torch.Tensor` functions.
In Pytorch the recommended way to build tensors are either using
torch.as_tensor(). So what is the difference?
torch.tensor() always copies the data whereas
torch.as_tensor() always tries to avoid copies of the data. This is especially useful if you have a
numpy array and want to avoid copying the
numpy array into a tensor.
1.1 Let’s understand this by creating a
numpy array first
1.2 Create a tensor using
1.3 Create a tensor using
As you can see, there is no difference in the outputs between
1.4 Now let’s create a sligtly complicated
The tensor creation failed with
torch.as_tensor() function here because the
numpy array that was created is a 2-D array with rows that have different lengths which works for
numpy arrays but fails during conversion from
numpy to a
torch.tensor() copies the
numpy array whereas
torch.as_tensor() shares the memory with the
This function returns a 1-D tensor of size (end-start/step) with values from the intervals start, end taken with common difference
2.1 Let’s first create a tensor
In the above example,
torch.arange() function returns numbers 0, 1, 2, 3, 4, 5 with an interval of 1. Instead of specifiying the start (the default is 0) and the step size (the default is 1), you can just give the end number (in this example 5) and
torch.arange() will generate numbers 0-5 similar to example 1.
2.2 Now let’s see an example where things can go wrong
In this example, the function assumes that start is 0, end is -5 and step size is 1 and it is not able to return the numbers from 0, -5 with a step size of 1. This can be fixed by giving an upper bound (for example 0).
torch.arange() is a very convenient function to generate a sequence of numbers from
end with a user specified
This function returns a
tensor with the same data and number of elements as
input but with the specified shape.
3.1 Let’s see this with an example
As can be seen in the above example,
torch.reshape() has returned a tensor with the same data but with a specified shape of
2 x 2 for earlier shape of
1 * 4.
What if you want to convert the
input tensor of shape
4 x 1 into an array of
1 x 4, then you can just specify
(-1, 1) as shown in this example. Infact you can leave the second number and just specify
3.2 Specifying wrong reshape parameter
This is the frequent mistake that people often do with
torch.reshape() function. The above error indicates that the original input is a size of
4 x 1 whereas, we ask it to generate a tensor of shape
2 x 3 which is not possible. This is because
2 x 3 = 6 whereas the original size is
4 x 1 = 4.
To summarize, reshaping a tensor using
torch.reshape() is quite important in matrix multiplication and other functionalities.
torch.abs() computes element wise absolute value of an
4.1 Let’s see an simple example
As can be seen,
torch.abs() was able to convert all the negative values into positive values from an
input tensor consisting of 3 elements.
torch.abs() not only works on the 1-D tensor but also can also work on 2-D tensors such as matrices as show in this example.
torch.abs() on a wrong tensor type
The error here indicates the the
torch.abs() only works on tensors that have numbers (floats, integers etc.,) and not on boolean values (TRUE/FALSE, Yes/No etc.,). So make sure to check the tensor type before using
torch.abs() is a nice little functionality in Pytorch that returns the absolute value of a numerical tensor that is of any dimension.
leaf node/variable is a variable that is at the beginning of the graph. That means that no operation is tracked by the
autograd engine created it. All tensors that have
requires_grad set to False will be leaf tensors by convention.
5.1 Let’s understand this with an example
a tensor is a leaf variable
Let’s create another example where a tensor is not a leaf variable.
As can be seen in the above example, the tensor
b was not created by the user but was created by an operation that cast a float tensor into a double tensor. So the tensor
b is not a leaf variable.
5.2 Creating a tensor with gradient that has wrong data type
In this example, we tried to check if the tensor is a leaf variable or not, but before we can check that out, we get an error as shown here.
If you want to create gradients for an tensor, floating point data type are only acceptable. So make sure you fix that before you can use
is_leaf() is a convenient function to check if the variable is at the beginning of the graph or not so that you don’t have to keep a track of it manually.
This notebook explores five Tensor functions in Pytorch with detailed examples including the cases where things go wrong. Understanding Tensor functions in Pytroch is very important so that they can be applied correctly in downstream analysis.
If you want to try these functions by yourself, visit this notebook, fork/clone and press Run.