AI Assistants and Tools integration with Ollama

Wish List
1

We actually are looking at providing the AI Assistants with a localized LLM service with Ollama. It has OpenAPI Compatible API Endpoints and it would be handy to enter your own server based on this.

We actually prefer to run our LLM’s on an Air Gapped network to provide security for internal documents etc, this would be handy to have integrated with VitalPBX for internal extensions to use this service.

ollama/ollama: Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 2, and other large language models.

1 Like
2

Thank you for sharing your approach! Running an LLM locally with Ollama and an air-gapped network is a great way to ensure data privacy and control over AI-generated responses. However, there are several factors to consider when deploying AI models in a PBX environment:

  1. Hardware Requirements – LLMs require significant computing power, including a high-performance CPU/GPU, sufficient RAM, and fast storage. Running them on a PBX server could impact overall system performance.
  2. Resource Allocation – PBX systems prioritize real-time voice processing, so running an AI model alongside could introduce latency or affect call quality. Dedicated hardware or a separate instance may be necessary.
  3. Security & Compliance – While an air-gapped setup enhances security, maintaining updates, model improvements, and integrations without external connectivity requires careful planning.
  4. Scalability & Maintenance – Managing and optimizing a local LLM deployment involves regular model updates, fine-tuning, and monitoring resource usage to prevent performance bottlenecks.

At this time, VitalPBX is not considering providing a built-in localized LLM service!

3

I don’t think he is referring to have a LLM on the PBX server. But being able to use a local LLM via an API for the AI Assistants.

1 Like
4

@mo10 Correct, I’m awaiting the approval of a post explaining that :slight_smile:

5

@miguel Sorry for the misunderstanding I was not really looking for a built in LLM Service, More of the ability for VitalPBX to talk to it’s API’s, with the LLM Service running on another server.
As they are OpenAI compatible based API’s it’s quite easy to change the service endpoint your AI Assistants tools look at, and instead of a locked down Drop Down for Model selector, you could query the API to get a list of models available.

E.G, if you add a drop down to select between OpenAPI and Ollama, you can add an extra input box to request for the Ollama service endpoint.

1 Like
6

At this stage, the current status and development of Ollama are not enough to make it a viable AI provider for our AI Assistants module. Here’s why:

  1. Limited Features – Ollama currently only supports Chat Completion, while VitalPBX relies on multiple OpenAI features, including Text-to-Speech, Speech-to-Text, and AI Assistants.
  2. Experimental Implementation – As stated on their website, Ollama’s current implementation is experimental, meaning stability and long-term support are not guaranteed.
  3. Lack of Key APIs – Important features like the Embeddings API, Function Calling, etc. are still under consideration and not available yet.

Because of these limitations, Ollama doesn’t yet meet the requirements to be a full AI provider within VitalPBX. However, we’ll continue monitoring its development for future possibilities.

7

@miguel Sounds good, i’ll revisit when it’s more mature :slight_smile: