Loss¶
Loss module includes a set of training loss implmentations. Some are converted from C++ implementation, and the rest are implemented directly using python Tensor.
Example usage:
from singa import tensor
from singa import loss
x = tensor.Tensor((3, 5))
x.uniform(0, 1) # randomly genearte the prediction activation
y = tensor.from_numpy(np.array([0, 1, 3], dtype=np.int)) # set the truth
f = loss.SoftmaxCrossEntropy()
l = f.forward(True, x, y) # l is tensor with 3 loss values
g = f.backward() # g is a tensor containing all gradients of x w.r.t l

class
singa.loss.
Loss
¶ Bases:
object
Base loss class.
Subclasses that wrap the C++ loss classes can use the inherited foward, backward, and evaluate functions of this base class. Other subclasses need to override these functions

backward
()¶  Returns
the grad of x w.r.t. the loss

evaluate
(flag, x, y)¶

forward
(flag, x, y)¶ Compute the loss values.


class
singa.loss.
SigmoidCrossEntropy
(epsilon=1e08)¶ Bases:
singa.loss.Loss
This loss evaluates the crossentropy loss between the prediction and the truth values with the prediction probability generated from Sigmoid.

backward
()¶ Compute the gradient of loss w.r.t to x.
 Returns
dx = pi  yi.

evaluate
(flag, x, y)¶ Compuate the averaged error.
 Returns
a float value as the averaged error


class
singa.loss.
SoftmaxCrossEntropy
¶ Bases:
singa.loss.Loss
This loss function is a combination of SoftMax and CrossEntropy loss.
It converts the inputs via SoftMax function and then computes the crossentropy loss against the ground truth values.
For each sample, the ground truth could be a integer as the label index; or a binary array, indicating the label distribution. The ground truth tensor thus could be a 1d or 2d tensor. The data/feature tensor could 1d (for a single sample) or 2d for a batch of samples.

class
singa.loss.
SquaredError
¶ Bases:
singa.loss.Loss
This loss evaluates the squared error between the prediction and the truth values.
It is implemented using Python Tensor operations.

backward
()¶ Compute the gradient of x w.r.t the error.
 Returns
x  y

evaluate
(flag, x, y)¶ Compuate the averaged error.
 Returns
a float value as the averaged error

forward
(flag, x, y)¶ Compute the error as 0.5 * xy^2.
