TOSA v0.30.0 released

TOSA
#1

Announcing the TOSA v0.30.0 Release

Here’s what’s new in v0.30.0 of TOSA.

Overview:

TOSA 0.30 adds new Fast Fourier Transform (FFT) operators to the Main Inference and Main Training profiles. It also adds limited support for unsigned 16-bit integer tensors.

Initial release of the TOSA conformance test suite. The first release of the conformance tests provide coverage of the expected behavior of a TOSA Base Inference profile system . It contains test cases that are expected to produce the results as defined in the specification. It also tests the cases marked in the specification as ERROR_IF, to verify that a TOSA implementation catches the errors properly. The TOSA conformance suite is located on the mlplatform.org git at this address: https://git.mlplatform.org/tosa/conformance_tests.git/

Initial release of the TOSA MLIR translator . This is a set of passes that can be used in conjunction with an MLIR based TOSA compiler to add the ability to serialize MLIR’s TOSA dialect into the TOSA FlatBuffers format as implemented in the TOSA serialization library. The TOSA MLIR translator is located at: https://git.mlplatform.org/tosa/tosa_mlir_translator.git/

Specific floating-point types have been added to the Main Inference and Training profiles. Currently the specification requires support for standard 16-bit floating point, 32-bit floating point, and the bfloat-16 types.

If you have questions or feedback on any aspect of TOSA, please feel free to post on the mlplatform discourse: TOSA - Discourse

Specification:

  • The FFT2D and RFFT2D operators are added to the Main Inference and Main Training profiles for floating point operations.
  • Unsigned 16-bit integer (uint16) support added. The only operator supporting uint16 is the RESCALE operator. It is expected that networks consuming uint16 input immediately use a RESCALE to convert to signed 16-bit values (int16).
  • Floating point data types have been defined. Instead of a generic floating point format, fp16, fp32, and bf16 data types have been introduced.
  • Chapter one has been reorganized to try to match the overall specification better, including an updated description of integer operations/quantization within TOSA.
  • The specification now describes the status of each profile separately, to better convey the stability level of the profile.
  • Floating point edge-case behavior definition has been improved.
  • In the pseudocode and data type tables for operators, if the input and output data types are the same, an in_out_t type is used to attempt to make this easier to read.
  • Specifies the output formulae to define the output dimensions for tensor operators.
  • Restricted the slope between adjacent 16-bit TABLE entries.
  • Removed support for -1 dimensions in the RESHAPE operator so that all TOSA shapes are fully determined.
  • Miscellaneous other clarifications and corrections. (See the git log for details)

Reference Model:

  • Improved error condition testing aligned with the specification.
  • Included scripts to create and run the test suite.
  • Updated to align with the changes in the serialization library and specification

Serialization Library:

  • Aligned the schema variables to the same words as in the specification for easier comparison.
  • Added uint16 data type support.
  • TRANSPOSE_CONV2D now matches the specification and does not support a dilation attribute.
  • Removed separate quantization attributes. Any required attributes are now part of each operator’s attributes directly. This aligns with the specification.

Conformance Tests:

  • First release of the conformance tests.
  • Positive and negative coverage of the Base Inference operators.

TOSA MLIR Translator:

  • First release of the MLIR translator.
  • Provides an MLIR pass to serialize MLIR TOSA dialect to the TOSA FlatBuffers format.
2 Likes