Verified & Integrated MXINT Implementation and Evaluation on NeRF (ADLS Group 0)

[omaralkhatib03]

Description

This PR introduces a significant refactor of the MXINT implementation. It enhances verification and fully integrates MXINT quantization into the Mase Framework. These changes enable the emission of deep fully connected PyTorch networks and facilitate software analysis of MXINT quantization using Optuna. Additionally, we provide enhancements to NeRF vision and leverage the model for evaluation purposes.

Contributions

1. An automated tooflow for emitting pytorch models after applying MXINT quantisation.
2. A set of verified OCP-compliat parametric verilog modules:

• MXINT Linear Layer
  • MXINT Accumulator
  • MXINT Cast
  • MXINT Dot Product
• MXINT ReLU Layer
• MXINT Concatenation

3. Evaluation of MXINT Quantisation onf NeRF Model
4. Evaluation of MXINT Resource - Accuracy Pareto front of an MLP Network using Optuna Multi-objective study

Detailed Implementation Notes

Changes to the analysis

• Overall change, rename all instances of modules referring to integer to fixed-point to avoid double naming
• Changes to the add_hardware_metadata pass
  • Introduce the ability so support different data_types for the hardware modules
    • Define a python type variable to capture the structure of the INTERNAL_COMP dictionary and the individual ip entries
    • Define the list of supported hw quantizations as a string literal (this is used when checking for compatibility when running the pass)
    • Change the analysis to account for different datatypes
  • support MXint_relu
• Changes to the quantize pass
  • Automatically quantize the data_out variables, using the data_in config if a data_out config is not explicitly defined
  • override integer to be fixed-point if it is encountered

Changes to emission

• At a high level, allow for non-fixed-point datatypes, specifically the mxint variety
• BRAM emission (emit_bram)
  • Introduce an emit_parameters_in_mem_internal_mxint function
    • computes the required constants and inserts them in the mxint_bram_template
    • write the bram to a file
  • Refactor emit_parameters_in_dat_internal to handle mxint datatype
    • write the values out as binary instead of hex because of issues with encoding packed arrays for non multiple of 4 bitwidths
• Testbench (emit_tb)
  • Handle non-fixed point datatype, namely mxint
  • Refactor to have custom drivers and monitors based on the data type (inheriting from the standard ones)
    • introduce the custom monitor for mxint off by one errors as described in the report
  • Move the quantisation step to occur inside of the custom driver/monitor
• Top emission (emit_top)
  • Handle non-fixed point datatype!
    • Abstract the wiring/port/interface generation logic into common function
    • Handle the type-specific cases within these abstracted functions

Changes to the cocotb interfaces

• introduce generic off-by-one multi-signal monitor
• Changes to the testing
• Introduce test_emit_verilog_linear_mxint
  • This test is responsible for ensuring that verilog emissions works for the mxint datatype
  • Has two test functions that each take a seed and generate some parameters to test on
    • ..._mlp generates and tests a multi layer perceptron with random number of layers and random dimensions
    • ..._linear simply tests the linear layer, to ensure the weighs and biases being emitted are correct
  • In all cases the test does the following:
    • given some model and some configuration it will quantize the modeldd_hardware_metadata` pass
  • Introduce the ability so support different data_types for the hardware modules
    • Define a python type variable to capture the structure of the INTERNAL_COMP dictionary and the individual ip entries
    • Define the list of supported hw quantizations as a string literal (this is used when checking for compatibility when running the pass)
    • Change the analysis to account for different datatypes
  • support MXint_relu
• Changes to the quantize pass
  • Automatically quantize the data_out variables, using the data_in config if a data_out config is not explicitly defined
  • override integer to be fixed-point if it is encountered

References

• Open Compute Project. OCP Microscaling Formats (MX) Specification Version 1.0. September 2023. [Online]. Available: https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf