Custom matmul converter for torch2trt

NVIDIA is developing a high-performance SDK for deep learning inference. It performs up to 40X times faster than if using a single CPU or up to 3X times faster inference than if using standard deep learning framework like PyTorch on GPU.

Deep neural network written in a common deep learning framework like PyTorch, TensorFlow, Onnx or Matlab can be converted to TensorRT using conversion scripts delivered by NVIDIA. In the article I will focus on the PyTorch converter. NVIDIA delivers converter called torch2trt. It is publically available on github under the following link: https://github.com/NVIDIA-AI-IOT/torch2trt . Unfortunately not all PyTorch layers have written converters for themselves. Here comes a nice feature from NVIDIA - everyone can write his own converter if it is missing. And it is pretty straightforward. Here below I present the matmul layer converter, that was missing, so I had to implement it by myself.

Matmul converter for torch2trt.