Edit

Apache SINGA-3.1.0 Release Notes

SINGA is a distributed deep learning library.

This release includes following changes:

• Tensor core:

  • Support tensor transformation (reshape, transpose) for tensors up to 6 dimensions.
  • Implement traverse_unary_transform in Cuda backend, which is similar to CPP backend one.

• Add new tensor operators into the autograd module, including CosSim, DepthToSpace, Embedding, Erf, Expand, Floor, Pad, Round, Rounde, SpaceToDepth, UpSample, Where. The corresponding ONNX operators are thus supported by SINGA.

• Add Embedding and Gemm into the layer module.

• Add SGD operators to opt module, including RMSProp, Adam, and AdaGrad.

• Extend the sonnx module to support DenseNet121, ShuffleNetv1, ShuffleNetv2, SqueezeNet, VGG19, GPT2, and RoBERTa.

• Reconstruct sonnx to

  • Support creating operators from both layer and autograd.
  • Re-write SingaRep to provide a more powerful intermediate representation of SINGA.
  • Add a SONNXModel which implements from Model to provide uniform API and features.

• Add one example that trains a BiLSTM model over the InsuranceQA data.

• Replace the Travis CI with Github workflow. Add quality and coverage management.

• Add compiling and packaging scripts to creat wheel packages for distribution.

• Fix bugs

  • Fix IMDB LSTM model example training script.
  • Fix Tensor operation Mult on Broadcasting use cases.
  • Gaussian function on Tensor now can run on Tensor with odd size.
  • Updated a testing helper function gradients() in autograd to lookup param gradient by param python object id for testing purpose.