> It’s mostly not hard to understand how a proximity sensor door will function. Using a black-box classifier to improve that won’t necessarily make people like it more.

This is fundamentally the same problem as with self-driving cars. There's "dumb and predictable" and there's "DWIM" and the space in between is "mostly useless or dangerous". For DWIM interfaces, you have to reach a level of reliability[0] where you just know what the device will do and trust it to do it. Like, you walk at the doors "on autopilot" while distracted by phone or conversation, and the doors open in time to let you comfortably through.

Note that the currenetly used automated doors rarely meet even basic reliability standards for a dumb, predictable device. Notably:

1. Most don't open fast enough to let a person walk through them at normal pace, forcing people to slow down or stop;

2. Many don't react well, or at all, when you approach from a side.

That 2. is IMO exemplary of the problem: it's typically some tuning to prevent the doors from getting triggered by people passing by or leaving the store through exit doors at 90deg to entry doors - but there is no complete solution here and any attempt just makes things inconvenient for users, because it's trying to solve the wrong problem. It's solving for "open for people approaching, ignore people passing by", whereas the problem that needs to be solved is, "open only for people who clearly intend to go through".

> And calling up to the cloud for every sensor event, ignoring privacy issues, adds weird latency and a huge failure mode during data center outages.

You do not need to call up cloud for this.

In fact, this post inspired me to get the equivalent working off local model to finally get "computer" activation DWIM (no bullshit "hotword recognition", but continuous awareness of conversations, and picking up only on those clearly intended for the "home's computer" -- exactly like Star Trek does it). Turns out, it's totally doable on the humble RTX 4070 Ti (which is like TFA card but only half the memory bandwidth).

Hell, a doors PoC would be too (gonna try this later for triggering lamps via Home Assistant), but since determining intent here requires processing multiple frames, that's gonna be tricky to do with local model in near real-time. Tricky, but not impossible.

[0] - Possible interactive reliability, so you're able to negotiate your intended meaning on the spot, without even thinking about it, through a gesture, word, face movement, pose change, etc.

A single function Jev-like wrapper for LLMs, including vision models