Yes, yes, YES! Finally, after years I can finally communicate with my house!

I know, sounds a bit cryptic. For years I have always wanted a voice integration to a home automation/home control system. Today I finally cracked it and the lid flew off Pandora's box! Steadramon has added some voice stuff to ESPGeiger which is quite fun and he also wrote FW for an ESPGeiger Gadget - a good tool (if you have multiple ESPGeigers running) and also a bit fun . That got me back in the mood for voice control and today I have managed to integrate AI/LLM into Home Assistant, and it is amazingly fluent, has finesse and just works. Some small hiccups at first, but once all the pipes were plumbed onto the right unions, it just worked.

A short video of it here (sorry about the loud/quiet audio).


I have several ESPGeigers running around my house, most in my lab, Radspod One in the garden, another in the bedroom. These are all sending data into my Home Assistant, along with other devices. For now whilst I am getting this running properly (there are still some small things to iron out) I am keeping the exposed entities down to a minimum as the more exposed devices the more processing the LLM will need if it has to decide which device I am talking about. I find it quite remarkable just how easy it is to talk with the thing, and get the right data out first time. The demo video was not a setup. I picked up my camera, put the voice assistant box in front of me, started recording and asked it questions and told it to do things. Nothing was staged. The last question I asked it took a little longer than the others as I was purposefully vague and didn't mention the actual light, just 'its too bright, set it at 40'. It had to go through past instructions and actually work out what to do about it being too bright for me, and got it right first time.

I have worked in the home automation/home control industry for over 11 years in the past (left it about 6 years ago) and never had anything close to this level of fluency and finesse. Alexa controlling Control4 was about the best it got, but was unreliable, slow and you had to give it exact key phrases or words or it just didn't understand, and thus for a simple operation like 'turn the light on' would have to be written several different ways so different people could call it up and Control4 perform the action. It was very basic, clunky and got it right maybe 50% of the time. This local LLM and voice integration has worked for me 100% all of the time since I enabled it. Absolutely marvelous! Even my wife is impressed! 

There's some bits to iron out still, such as it not saying 'micro sieverts' and stating the characters instead, but on the whole it is running really well for something practically right out of the box.

And you will probably want to know what hardware this is running on, so here it is:

Home Assistant (HAOS) is running on a HP EliteDesk 800 G2 SFF PC Intel Core i5-6500 8GB RAM SSD disk, but I forget how big. Maybe 1TB?

The LLM machine is my PC workstation:
Gigabyte Z890 AORUS PRO ICE mainboard
Intel Core Ultra 7 265K (8 p-cores, 12 e-cores, 1 NPU)
32GB (2x 16GB) G-Skill DDR5 8000MT/s
Gigabyte RTX 5070 12GB OC GPU
Samsung 9100 PRO 1TB NVMe (~14.5GB/s RW - PCIe 5.0 x4) system drive
Samsung 990 PRO 1TB NVMe (~7GB/s RW - PCIe 4.0 x4)
Corsair 1000W modular PSU
There's a bunch of other stuff, drives etc, but this list is what the LLM runs on.

The voice hardware is a Home Assistant Voice Preview Edition. Lame as f**k without an AI to plug into, but seems to get the job done with an LLM. I don't like the choice of wake words though, and they aren't customizable so I have ordered a M5Stack ATOM Echo S3R voice module to try as they can have customizable wake words. The cherry will be when I can say 'Hey Lala, blah blah.' I also like the Scottish female accent. With the customizable wake words it will feel a lot better because 'hey Jarvis' and it talks with a female voice just doesn't quite fit properly.

No ESPGeigers were harmed during construction.