Does a LLama (llama.cpp, huggingface) local deployable server exist yet, that lets me load different kinds of QLora finetunings on demand and does tasks in a priority queue?

So that I could have for instance and bulk archive processor that has a specific fintuning work, while the server remains useful in handling user requests with a higher priority?

Would be good if it would be locally deploy-able, but a cloud works too.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.