Integration of Fast-LLM

[bigximik]

Fast-LLM Integration (Draft)

This is a basic integration of Fast-LLM and is a work in progress:

• Based on the existing HFLM integration.
• The HFLM constructor is incompatible, so all components are initialized explicitly in our constructor.
• Tests are adapted from sglang since neither sglang nor our implementation supports "EleutherAI/pythia-70m"; we use Qwen2 instead. However, the test coverage is not yet as complete as in the HF integration tests.
• Quantized models are not supported in this iteration.

Still to be done:

• Integration of distributed inference.
• A test case for passing an instantiated Fast-LLM model?

Notes:

• Currently works with this branch of the Fast-LLM only: Sandbox for Implementation of generate and integration of lm_eval (evaluation harness) ServiceNow/Fast-LLM#222

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

Toolkit User seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}