I’m starting to look into conversational AI for a couple of my clients and see it’s an option in VitalPBX 4.5 now. I’m still running 4.2 in production and haven’t made use of the AI side of things aside from transcription.
Has anybody set this up yet? I’m not finding any documentation.
I don’t understand why we need a provider here and it looks like the only option is Twilio but there’s no place to put API keys or anything for that. I’d expect this to just work with OpenAI keys only and am not understanding Twilio’s roll.
I also ran across this before realizing conversational AI had been added.
infinitocloud/asterisk_to_openai_rt: Asterisk to OpenAI RealTime
Might be interesting to follow. I tried setting it up but ran into problems when the nodejs app tries to connect to ARI.
1 Like
Here’s all the information:
Thanks for the video miguel. I’m not understanding why Twilio is required for this. Why not stream the call directly between VitalPBX and OpenAI using the realtime model?
Streaming calls from Asterisk to OpenAI is quite challenging. We are utilizing the integration that Twilio provides, which simplifies the process of streaming the call and connecting with OpenAI.
Got it, the author of that github project has things partially working with vitalpbx right now. Any interest in officially supporting it? I’m having him get it working on my VitalPBX test instance right now, having an issue with the buffer when I call from a SIP trunk to it however it does work fine when calling from an extension.
We have this working nicely now running a Node app on vitalpbx handling the audio stream over ARI directly between Asterisk and ChatGPT. We’ll be integrating this using the AI Assistant page for now. If VitalPBX as a company has interest in the working code let me know. This avoids the need for Twilio and their associated costs.
1 Like
Are you willing to share the code? I’m very interested in utilizing this moving forward, but I also didn’t like the idea of having to use Twilio as another service and the additional expense to them on top of what we are already paying. Also, our main carrier is Telnyx and they are supposed to be introducing AI as well and I’m thinking about possibly just integrating with their service and keep all of the SIP traffic local to the provider. Is your code available to choose which AI model you want to base it on? If you’d like to take this conversation offline, just let me know and I’ll give you my contact information. I’m on Facebook Messenger and WhatsApp as well. Or if you want to do a call, I’m up for that too. Thanks for your contributions and have a wonderful day.
Totally willing to share but I have paid the developer a bit and would like to split that in some way if others are interested in this. I’ll PM you and we can take this to WhatsApp for the moment.
We had it working with about 10-15 minutes of tests with users when I posted this but now are having an issue receiving every other response due to what looks like packet timing issues. We get audio for the first two responses then after that every other response is silent although we do get the text response. It only happens for an incoming call over a SIP channel. If I call from an extension it works fine with no audio loss.
I’d much rather have more people involved in this, always good to get a community around something.
1 Like
Just to update on this the developer for that github has been mostly focused on getting this working when calling in from an extension. It has some issues when calling in over a SIP trunk due to OpenAI using PCM16 at 24Khz vs ulaw 8-bit 8Khz. We were getting intermittent voice responses over some cellular carriers which I couldn’t fully explain.
We’ve tested AVR Agent Voice Response and have it working today although audio quality could use some improvement. I suspect switching OpenAI to use ulaw instead of PCM16 will help quality somewhat although this does hurt in terms of speech recognition accuracy for OpenAI. This is a bit more complex however it’ll work with other services and the developer is very motivated to get things working right now. Their Discord channel is also pretty active Agent Voice Response.
1 Like
