I’m running HAOS on a mini x86 PC with localai.io running on a local server with an Intel A750. The model I’m currently using is qwen3:4b-instruct-2507-q4_K_M which should support tool use.
I’ve tried other models and had similar results.
The debug spew for the above interaction is below. Of note, the data field of intent_output is empty.
Is this something I need to configure on the localai.io side? Something with the yaml file? Or is HA not catching the intents coming from localai?
stage: done
run:
pipeline: 01m4qn8aamt08pa8q3gprs8g9y
language: en
conversation_id: 01M4SQBP7364Q234NHBDAVQF4V
runner_data:
stt_binary_handler_id: null
timeout: 300
events:
- type: run-start
data:
pipeline: 01m4qn8aamt08pa8q3gprs8g9y
language: en
conversation_id: 01M4SQBP7364Q234NHBDAVQF4V
runner_data:
stt_binary_handler_id: null
timeout: 300
timestamp: '2026-10-13T04:12:03.684196+00:00'
- type: intent-start
data:
engine: conversation.ollama_conversation
language: en
intent_input: turn off the kitchen lights
conversation_id: 01M4SQBP7364Q234NHBDAVQF4V
device_id: null
satellite_id: null
prefer_local_intents: false
timestamp: '2026-10-13T04:12:03.684286+00:00'
- type: intent-progress
data:
chat_log_delta:
role: assistant
content: ''
timestamp: '2026-10-13T04:12:03.834864+00:00'
- type: intent-progress
data:
chat_log_delta:
content: intent
timestamp: '2026-10-13T04:12:03.835315+00:00'
- type: intent-progress
data:
chat_log_delta:
content: __
timestamp: '2026-10-13T04:12:03.888268+00:00'
- type: intent-progress
data:
chat_log_delta:
content: H
timestamp: '2026-10-13T04:12:03.941413+00:00'
- type: intent-progress
data:
chat_log_delta:
content: ass
timestamp: '2026-10-13T04:12:03.997781+00:00'
- type: intent-progress
data:
chat_log_delta:
content: Turn
timestamp: '2026-10-13T04:12:04.052061+00:00'
- type: intent-progress
data:
chat_log_delta:
content: 'Off'
timestamp: '2026-10-13T04:12:04.108197+00:00'
- type: intent-progress
data:
chat_log_delta:
content: ("
timestamp: '2026-10-13T04:12:04.160317+00:00'
- type: intent-progress
data:
chat_log_delta:
content: light
timestamp: '2026-10-13T04:12:04.214299+00:00'
- type: intent-progress
data:
chat_log_delta:
content: '",'
timestamp: '2026-10-13T04:12:04.267968+00:00'
- type: intent-progress
data:
chat_log_delta:
content: ' "'
timestamp: '2026-10-13T04:12:04.320906+00:00'
- type: intent-progress
data:
chat_log_delta:
content: Kitchen
timestamp: '2026-10-13T04:12:04.376033+00:00'
- type: intent-progress
data:
chat_log_delta:
content: '")'
timestamp: '2026-10-13T04:12:04.432053+00:00'
- type: intent-progress
data:
chat_log_delta:
content: ''
timestamp: '2026-10-13T04:12:04.485825+00:00'
- type: intent-progress
data:
chat_log_delta:
content: ''
timestamp: '2026-10-13T04:12:04.486312+00:00'
- type: intent-progress
data:
chat_log_delta:
content: ''
timestamp: '2026-10-13T04:12:04.486666+00:00'
- type: intent-end
data:
processed_locally: false
intent_output:
response:
speech:
plain:
speech: intent__HassTurnOff("light", "Kitchen")
extra_data: null
card: {}
language: en
response_type: action_done
data:
success: []
failed: []
conversation_id: 01M4SQBP7364Q234NHBDAVQF4V
continue_conversation: false
timestamp: '2026-10-13T04:12:04.488028+00:00'
- type: run-end
data: null
timestamp: '2026-10-13T04:12:04.488136+00:00'
started: 2026-10-13T04:12:03.684Z
intent:
engine: conversation.ollama_conversation
language: en
intent_input: turn off the kitchen lights
conversation_id: 01M4SQBP7364Q234NHBDAVQF4V
device_id: null
satellite_id: null
prefer_local_intents: false
done: true
processed_locally: false
intent_output:
response:
speech:
plain:
speech: intent__HassTurnOff("light", "Kitchen")
extra_data: null
card: {}
language: en
response_type: action_done
data:
success: []
failed: []
conversation_id: 01M4SQBP7364Q234NHBDAVQF4V
continue_conversation: false
finished: 2026-10-13T04:12:04.488Z
It can have it’s use cases. Family members don’t have to be as precise with correct sentences for example. If you want to use it with music assistant a simple LLM works better in my experience then trying to do this with intent. Can you do all this with the local intent recognition? Absolutely, but you will experience edge cases and it takes a while to get dialled in correctly.
An other big plus, LLMs often automatically support multi language. This is big for a lot of European users in my family we already speak 4 different languages and various dialects and now no matter who visits or stays over, they can use the voice control in native langue.
Personally, I use both. If the intent recognition doesn’t catch it, the LLM will. So for me it is just an extra layer that protects me from angry family members :)
Setup:
It runs decently well, especially for tool use tasks.
The multi language is great, true, but that starts with good multi language STT, which in my experience, Flemish Dutch has like a 70-80% error rate because they are all trained on Netherlands dutch, and that isn’t even with a west Flemish accent lol.