loss¶

Loss functions for neural network training.

This module contains implementations of common loss functions used in neural network training, such as Mean Squared Error and Cross Entropy.

Classes:: MeanSquaredError: Calculates the Mean Squared Error loss. CrossEntropy: Calculates the Cross Entropy loss.

class CrossEntropy[source]¶

Bases: Op

Calculates Cross Entropy loss.

This class implements the Cross Entropy loss function, which is commonly used for classification tasks. It computes the loss given logits and target indices (as opposed to one-hot encoded tensors).

_y_true¶: The true labels (cached for backward pass).

_log_softmax_pred¶: The log softmax of predictions (cached for backward pass).

_out¶

The computed loss (cached for backward pass).

Type:: numpy._typing._array_like._SupportsArray[numpy.dtype[Any]] | numpy._typing._nested_sequence._NestedSequence[numpy._typing._array_like._SupportsArray[numpy.dtype[Any]]] | bool | int | float | complex | str | bytes | numpy._typing._nested_sequence._NestedSequence[bool | int | float | complex | str | bytes] | None

_grad¶: The computed gradients (cached for backward pass).

backward(grad)[source]¶

Computes the backward pass for Cross Entropy loss.

Parameters:: grad (Tensor) – A Tensor containing the gradient from the previous layer.
Returns:: A Tensor containing the computed gradients.
Raises:: NotImplementedError – If the input tensor has an unsupported number of dimensions.
Return type:: Tensor

forward(y_true, y_pred)[source]¶

Computes the forward pass for Cross Entropy loss.

Parameters:

y_true (Tensor) – A Tensor containing the true labels.
y_pred (Tensor) – A Tensor containing the predicted logits.

Returns:

A Tensor containing the computed Cross Entropy loss.

Raises:

NotImplementedError – If the input tensor has an unsupported number of dimensions.

Return type:

Tensor

log_softmax(tensor)[source]¶

Computes the log softmax of the input tensor.

Parameters:: tensor (Tensor) – A Tensor containing the input values.
Returns:: The log softmax of the input tensor.

class MeanSquaredError[source]¶

Bases: Op

Calculates Mean Squared Error loss.

This class implements the Mean Squared Error (MSE) loss function, which measures the average squared difference between the predicted and true values.

diff¶: The difference between predicted and true values.

divisor¶: A scaling factor for the loss calculation.

backward(grad)[source]¶

Computes the backward pass for Mean Squared Error loss.

Parameters:: grad (Tensor) – A Tensor containing the gradient from the previous layer.
Returns:: A Tensor containing the computed gradients.
Return type:: Tensor

forward(y_true, y_pred)[source]¶

Computes the forward pass for Mean Squared Error loss.

Parameters:

y_true (Tensor) – A Tensor containing the true values.
y_pred (Tensor) – A Tensor containing the predicted values.

Returns:

A Tensor containing the computed MSE loss.

Raises:

ValueError – If the computed loss is infinite.

Return type:

Tensor