2

I'd like to integrate an LLM into my free, open-source, cross-platform desktop Markdown editor. In particular, bundle the LLM into the editor's ancillary podman container (that's used for typesetting). A few areas where the integration may help:

  • Grammar check
  • Idea generation
  • Rephrasing passages
  • Translating short snippets

Requirements:

  • Free, open source, MIT/Apache/BSD/etc. license
  • Local only (no cloud)
  • GPT-4 quality (or very close)
  • Fallback to CPU, if GPU unavailable (or CPU-based)
  • Trained on public works (no copyright material)
  • Container-friendly (i.e., works with podman)
  • No Python, if at all possible (not a deal-breaker)

Is there something that could be integrated at this time, or would you suggest waiting until the end of the year to see what shakes out? For example, llama.cpp with Llama2-7B HQQ+?

Thank you!

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.