I built a concept for a VLM-powered piano assistant
Reddit r/computervision1w4 min read
You guys really loved my last sheet music search engine post, so I decided to make a mockup POC of a VLM-powered piano assistant that reads and transcribes notes, procedurally "generates" a visual tutorial, and listens to you play it back. I call it a concept because I still haven't figured out a way to actually generate a 3D animation on the fly. My best option was Concert Creator, but that was shut down over 4 years ago RIP. So for the time being, I'm having to pre-make the animations. On the more technical side, Qwen 3.6 27B is taking both video stream and natural language as input, and use
