In this notebook we’ll use PyTorch to build a multiclass classification model to predict wine quality based on a couple physicochemical features of wine.

Data Preparation¶

The analysis will be based on the ‘Wine Quality Dataset’ dataset available on kaggle.com – https://www.kaggle.com/datasets/yasserh/wine-quality-dataset. The datasets is related to red variants of the Portuguese “Vinho Verde” wine.

This dataset consists of 1143 records containing information on particular samples, regarding 11 physicochemical features like acidity, various chemicals, density, pH and so on. Detailed information on each feature is included below.

Let’s import the libraries we need:

Data Collection¶

Let’s download and view the data:

As this is multiclass classification, there should be more than two labels. Let’s see how many there are:

According to the dataset’s description, the target class, which is quality, is an integer value between 0 and 10. As we can see, not all classes are represented in the dataset. The worst wines’ quality is 3, the best wines’ quality is 7.

The dataset is not balanced. The classes are not represented evenly. There are mostly wines of quality 5, 6 or 7. There aren’t many of other qualities, so, to keep things simple, let’s focus on just these three qualities and keep only them:

Data Description¶

Let’s look at basic information about the data:

The DataFrame consists of 1088 rows, each of which represents a single wine sample, and 13 columns, each of which represents a single feature.

The data is complete, there are no missing values. All columns are numerical (of type float64 or int64 in case of the last two columns).

Let’s have a look at the features:

• fixed acidity
• volatile acidity
• citric acid
• residual sugar
• chlorides
• free sulfur dioxide
• total sulfur dioxide
• density
• pH
• sulphates
• alcohol
• quality
• Id

The penultimate column, quality, serves as the target variable for prediction.

Data Preprocessing and Cleaning¶

Removing Redundant Rows and Columns¶

There is one column that we don’t need, id, so let’s remove it:

We already removed the rows with wines of qualities other than 5, 6 or 7. There are no more rows to remove for now.

Missing Data¶

There is no missing data:

Duplicates¶

Let’s check for duplicates:

Turns out, there are some duplicates. Let’s see how many:

There are 964 unique rows and 124 duplicated ones. Let’s remove the duplicates:

Let’s see how the particular classes are represented now:

There is no big difference.

Data Transformations¶

As we can see, the quality values are between 5 and 7. Let’s modify them to be between 0 and 2:

A Statistical Summary of the Numeric Features¶

Let’s have a look at a statistical summary of the numeric features for the entire dataset:

If you’re familiar with the domain of the analysis, you will be able to tell whether the statistical values are reasonable or not. We can also check the outliers to see if there are any.

Outliers¶

Let’s check if there are any outliers:

There are quite a few outliers. We could remove some of them or even all, but let’s keep them and see how the model will handle them. We can treat them as noise that will reduce the probability of overfitting.

Data Visualization¶

Let’s visualize the data to get a deeper insight.

Correlations Between Features¶

Let’s have a look at the relationships between the particular features. In particular, we’re interested in how the features correlate with the target feature, which is quality.

Let’s plot the pairwise relationships between the features first:

Next, let’s check the correlations between the features:

There are both positive and negative correlations. Let’s create a correlation matrix:

The feature we want to examine the correlations with is quality. The strongest positive correlation is with alcohol. The strongest negative correlation is with volatile acidity. Some correlations with other features are pretty weak (with absolute values below 0.1), so let’s remove these features:

Quantitative Analysis of the Features¶

Let’s create a set of histograms for all numeric features:

Inputs and Outputs¶

The target feature, quality, is our output. The other features are our inputs:

Let’s check the input and output shapes:

So, there are 964 records. This is what the first dimension tells us. The second dimension for X means it has 8 features (vector) whereas y has a single feature (scalar). We have 8 inputs for 1 output.

Training Set and Test Set¶

Before we build our model, we should split the data into two separate sets, a training set and a test set:

We have 771 training samples and 193 testing samples. Now we can start building the model.

Data Scaling¶

The features are in different scales. Let’s have a look:

For example, the values of volatile acidity are very small. The values of total sulfur dioxide are much higher, and so on. To make them more comparable, let’s scale them using StandardScaler:

Now the data is rescaled in such a way that the values’ mean is 0 and the standard deviation is 1.

Tensors¶

PyTorch works with tensors, so we have to turn our data into tensors:

Model Building¶

We have the data in place, it’s time to build a model. Besides the model, we’ll define a loss function and optimizer.

But before that, let’s make our code device agnostic. This is not strictly necessary for such a small dataset as ours, but it’s good practice. This way, we’ll make use of GPU if it’s available, otherwise we’ll make use of CPU:

In Google Colab, we can change the runtime to GPU in the Runtime menu under Change runtime type. We’re not going to do it here, though.

Defining the Model¶

We now want to build a model that will take our inputs and produce outputs similar to our current outputs. In other words, given the features, the model will predict the labels.

There are a couple ways of approaching this in PyTorch, but we’ll create the model as a class. If we create a model as a class, we almost always inherit from nn.Module. Then, inside the __init__ method, we create the layers of the neural network. In our case, we need both linear and nonlinear layers. We’ll use the ReLU activation function for the latter.

It’s up to us how many layers we create. It depends on how much space we want to give the model to learn. In any case, we pass two arguments to the nn.Linear method: in_features, set to the number of inputs in a particular layer, and out_features, set to the number of outputs from a particular layer. We can set these arguments to any numbers we want. We just have to follow the following rules:

• In the first layer, we set in_features to the number of inputs to the model.
• In the last layer, we set out_features to the number of outputs from the model.
• In the hidden layers (the layers between the first and last layers), the number if in_features must be equal to the number of out_features in the preceding layer.

We also have to define a forward method, which will contain the forward pass computation of the model.

In order to inherit from nn.Module and create the layers of the neural network, we have to import nn:

And now, let’s build the model:

Let’s create an instance of the model and send it to the target device:

Loss Function and Optimizer¶

The loss function is used to measure how wrong your model’s predictions are compared to the truth labels (the labels in the y_test set).

The optimizer instructs your model to update its internal parameters to lower the loss.

There are a lot of loss functions in PyTorch we can choose from. For multiclass classification, a common choice is Cross Entropy Loss, which we’re going to use. To this end, we’ll use the torch.nn.CrossEntropyLoss method.

There are also a lot of optimizers. Some common ones are Adam and SGD (stochastic gradient descent). Let’s pick SGD. In order to do that, we’ll use the torch.optim.SGD method.

The SGD optimizer takes two parameters:

• params – these are the model’s parameters that we want to optimize,
• lr – this is the learning rate – the higher it is the faster the optimizer will update the parameters.

We must be careful with the learning rate. It should be neither too high nor too low or it will fail to work.

So, here are the loss function and the optimizer:

Metrics¶

We’ll evaluate our model using just one metric, accuracy. It’s calculated as the total number of correct predictions divided by the total number of predictions. Let’s define a function that will calculate accuracy for us:

Model Training¶

Training the model involves two loops: a training loop, where the model learns the relationships between the features and labels, and a testing loop, where the model is evaluated. Let’s see what exactly each of the loops contains.

Training Loop¶

As mentioned above, in the training loop, the model goes through the training data and learns how the features are related to the labels.

The steps inside a training loop contain:

1. Forward pass – the model performs the forward method on all training data,
2. Loss calculation – the model’s predictions are compared to the test values to see how badly the model performs,
3. Gradient zeroing – the optimizer’s gradients are set to zero (by default, they’re accumulated) so that they can be calculated from scratch for this step,
4. Backpropagation – the gradient of the loss with respect to each parameter with requires_grad set to True is calculated,
5. Gradient descent – the parameters are updated.

Testing Loop¶

The testing loop consists of the following steps:

1. Forward pass – the model performs the forward method on all testing data,
2. Loss calculation – the model’s predictions are compared to the test values to see how badly the model performs,
3. (optionally) Evaluation metrics – we can calculate metrics like accuracy, precision or recall on the test set; we’re not going to do it here.

Training the Model¶

Now, let’s implement all these steps in the training and testing loops:

As you can see, the model performs pretty well, although not perfectly.

Model Evaluation¶

Let’s evaluate the model and see how it performs on data it never saw:

Let’s see how the test data and the predictions compare:

As we can see, the model classifies most of the test data correctly. These are the cases where the difference between test data and prediction is 0. But it’s not perfect and there’s definitely room for improvement.

Conclusion¶

The model we created performs pretty well. It’s not perfect and we could fine-tune it for better results. Although most of the test data is classified correctly, some of it isn’t. Things we could try to improve it include hyperparameter tuning (different learning rates, numbers of hidden units or numbers of layers), using a different optimizer or training the model for a different number of epochs. Feel free to try it out.