.. _ch:linear_perceptron:
Linear Perceptron
=================
This lab gets us started with PyTorch's workflow used for deep learning training.
You are given a mystery dataset which consists of 1000 labeled points in :math:`\mathbb{R}^3`.
The labels are binary, i.e., we only have the two labels 0 and 1.
Our goal is to find a linear classifier which separates points having label 0 from those having label 1.
The lab's individual steps are given as "Data Exploration", "Datasets and Data Loaders", "Training" and "Visualization".
We'll roughly follow this structure when training more complex neural networks in future.
.. hint::
Look outside of the class for additional guidance on how to use PyTorch for Machine Learning workloads.
`PyTorch's tutorials `_ are a great way to get started.
.. _ch:exploration:
Data Exploration
----------------
The points are given in the file :download:`data_points.csv `.
Each line in the file represents a single point where the points' x-, y- and z-coordinates are separated by commas.
The corresponding labels are given in the file :download:`data_labels.csv `, where a single line gives the label of the respective point.
.. code-block::
:linenos:
:caption: First five lines of the file ``data_points.csv``.
:name: lst_data_points
-8.494544704477987596e-01,-5.033859511649513979e-02,3.873837307225119070e-02
-7.883425406506253674e-01,-2.232586812852378061e-02,9.367344753561300530e-02
1.585747745378497942e-01,4.059444628687230994e-01,1.031322306402090661e+00
-9.682621200468680689e-01,-2.450696604084433294e-01,1.179431003097544117e-01
-1.707633608738729214e-01,1.420339570729253209e-01,9.003685159064296339e-01
.. code-block::
:linenos:
:caption: First five lines of the file ``data_labels.csv``.
:name: lst_data_labels
0
0
1
0
1
For example, the first five lines of the two files are given in :numref:`lst_data_points` and in :numref:`lst_data_labels`.
This means the the first point :math:`P_1` has approximately the coordinates (-8.5e-01, -5.0e-02, 3.9e-02).
The label of :math:`P_1` is 0.
:math:`P_3` is the first point which has label 1.
The approximate coordinates of :math:`P_3` are (1.6e-01, 4.1e-01, 1.0e+00).
.. admonition:: Task
Visualize the mystery dataset using `matplotlib `__.
Use `3D scatterplots `__. Color points with label 0 black and those with label 1 red.
Datasets and Data Loaders
-------------------------
Now, that we have an understanding for our dataset, we are ready to advance towards the targeted linear classifier.
Let's first define a data loader which wraps our dataset.
Data loaders are PyTorch's way of tackling the preprocessing.
The goal of an efficient data loader is to have the preprocessed data ready once the compute units require it.
For the time being we'll start simple: Our data loader does nothing else than returning the data in batches.
Later on, we'll see use more advanced data loaders.
For example, :numref:`ch:distributed_ml` uses special data loaders to perform data-parallel training.
.. admonition:: Tasks
#. Use `torch.utils.data.TensorDataset `__ to wrap the dataset.
#. Construct a data loader which operates on the wrapped dataset.
Use `torch.utils.data.DataLoader `_ and illustrate the parameter ``batch_size``.
Training
--------
Great, our data is in place.
We are ready to train our first classifier in PyTorch!
We'll train a very simple linear perceptron.
Our perceptron applies a single linear PyTorch layer (see `torch.nn.Linear `__) to the three-dimensional inputs and produces scalar outputs.
This leads to four degrees of freedom in the model:
Three for the "matrix" :math:`A` of the linear layer and one for the bias :math:`b`.
For the training procedure we'll also have to think about a loss function and an optimizer.
For now:
* Use `torch.nn.BCELoss `_ as your loss function; and
* Use `torch.optim.SGD `_ as your optimizer.
.. literalinclude:: data_linear_perceptron/trainer.py
:linenos:
:language: python
:caption: Template for the module ``eml.perceptron.trainer``.
:name: lst:perceptron_trainer
:numref:`lst:perceptron_trainer` contains a template for the module ``eml.perceptron.trainer``.
We'll use this template to separate our training procedure from the main code.
.. admonition:: Tasks
#. Implement the class ``Model`` in the module ``eml.perceptron.model``.
Define a single linear layer within ``Model`` in the constructor of the class.
Apply the sigmoid function after the linear layer in the ``forward`` function (see `torch.nn.Sigmoid `__).
#. Implement a training loop and print the total training loss after every epoch.
For the time being, implement the training loop directly in your main function.
#. Move the training loop to the module ``eml.perceptron.trainer``.
Use the template in :numref:`lst:perceptron_trainer` to guide your implementation.
Visualization
-------------
Now, let's professionalize our visualization efforts.
We'd like to get visual feedback from our classifier during training.
In our case, this means that we follow :numref:`ch:exploration` and plot the points colored by their predicted labels.
.. literalinclude:: data_linear_perceptron/points.py
:linenos:
:language: python
:caption: Template for the module ``eml.vis.points``.
:name: lst:vis_points
.. admonition:: Tasks
#. Implement a visualization module in ``eml.vis.points``.
Use the template in :numref:`lst:vis_points` to guide your implementation.
#. Monitor your training process by visualizing the labeled points after every :math:`x` epochs. Use reasonable number for :math:`x`. 😉