Differences in handling of accumulate and result type for Average Pool and MatMul/Conv

According to the tosa spec, the input/output type for average pool is always the same, and the accumulator type can be different (especially also larger than the result type, e.g. fp16 with fp32 accumulate).

However, operators such as conv2/3d and matmul do not have an accumulate attribute, and their output type seems to be matching the accumulator type (implicitly).

Is there a reason why these operators have a different handling of the output types and the accumulate attribute?

I came across this as tosa.matmul is currently not properly lowered from torch to tosa, the result type will be the same as the input type:

p.s.: I wanted to add links to the tosa spec on mlplatform, but I wasn’t allowed to.

Hi Tina,

In general, I believe that the separation of the accumulation and rescale as is done in the conv2/3d operators leads to simpler operation for the compiler. The reason that average pool is different and has the scale back to the original type is due to average pool dividing the sum by only the number of valid elements in the kernel region not including any padding. So on the corners of tensors the number of elements to divide by varies based on kernel size, padding, and strides. This would be hard to specify cleanly if the operators were separate. Once the division is done we expected that the best use of the result would be in the size of the inputs, as the average value is representable in the same range as the inputs.

I think after your first posting or two you should be able to post links. I believe it is an anti-spam mechanism. (For anyone looking, average pool is here: TOSA 0.70.0 specification, and conv2d is just below it)


1 Like

Thank you for the clarification!
We will fix the lowering from Torch → TOSA for matmul in torch-mlir.