TODOs

[mostafaelhoushi]

If you would like to contribute to the repo, please feel free to comment on what task you would like to work on, and feel free to go ahead to create a PR for it, or convert it to an Issue.

Refactoring / Cleanup:

• Unify quantize cli arguments into one file shared among eval.py, generate.py, diff.py, analyze.py
• Docstrings for all functions
• Text formatting
• Install formatting tools (e.g., black?)
• Rename any4.py to quantize.py
• Remove redundancies:
  • group_q vs group_q1
  • degroup_q vs degroup_q1
  • eval.py vs eval_perplexity.py
  • pseudo_**_quantizer in llm-awq
• Address TODOs in Code

README / Documentation:

• Add commands/table to reproduce results from paper
• Add Notebook tutorial
• Add banner links to Paper, Slides, Tutorial

Verbosity

• Reduce verbosity of convert in any4.py function that prints every layer name. Perhaps by printing to a tqdm bar?
• Reduce verbosity of eval.py that prints different groups of results ("Perplexity (GPTQ Implementation) Eval Results", "Perplexity Eval Results", "Code Eval Results", "NLP Eval Results"

Tinygemm:

• Ensure end-to-end correctness

APIs:

• Support passing dataset/dataloader/list of samples to sample_weight argument
• Make anyq default to what we have in the paper? [skip lm_head, scale_sample weight, sample_weight]

Tests:

• Add tests for APIs: int4, int8, any4, nf4, fp4
• Add tests for commands and examples in README
• Add tests for diff.py
• Add tests for analyze.py
• Add tests for calibrate.py
• Add tests for NF4
• Add tests for FP4
• Add tests for MX4
• Add tests for AWQ

Packaging:

• Make tinygemm a pip-installable library
• Make any4 a pip-installable library

[KSAGlory]

I’d like to handle a small part of the docstring TODO by documenting the model-structure helpers in utils.py: get_layers_from_model, get_attention_from_layer, get_mlp_from_layer, and get_lm_head_from_model. I’ll keep it documentation-only with no behavior changes.