I have implemented a LocalAI LLM backend on my server and have decided to host it here on my website for fun. I have very old hardware and it is on cpu only due to pci passthrough not working properly with my laptop. Neverthe less it is a fun project and once I get some more powerful hardware I feel it will function much better. I do have an extra GTX 1080, perhaps if I get around to building a tower server I can throw that in and pass it through to the AI LXC container for much faster AI operations.

I am currently, as of July 2025, converting from a Phi LLM to DeepSeek-R1, trying out different backends for the llm. Particaully ones that can utilize the GPU effectively

localai,
author: Ettore Di Giacinto,
title: LocalAI: The free, Open source OpenAI alternative,
year: 2023,
publisher: GitHub,
journal: GitHub repository,
howpublished: https://github.com/go-skynet/LocalAI