Status Update - April 28 - Hand Tracking WIP

Hi everyone! Here's a progress report on what I'm up to:

For those who don't know, the next big new feature coming to kemorig is Leap Motion Hand Tracking. I was hoping to get this released in April, but it's gonna be early May :( It should definitely be out before May 8th.

I knew it would be a big feature, but I still underestimated how big. The amount of animation code in kemorig has more than doubled with the introduction of hand tracking!

More info about hand tracking:

• Hand tracking works in any pose. It does this by being relative to a bone you specify (e.g. the chest bone).

• You can blend between automatically-calculated elbow positioning, or animated bone pole targets in your model. This means you can have full range of elbow motion by default, or constrain it with your animations if you need to control where elbows go (with the recline pose above, I use this feature!)

• You can use stances to enable or disable hand tracking and finger tracking independently (per-hand). For example, if you have a stance where your avatar holds a prop you can keep hand tracking enabled but disable finger tracking.

• I had to write my own solution for IK arms to handle wrist and shoulder twist bones, and it supports any bone hierarchy in avatar models. Unreal's included IK solutions, turns out, are super rigid and couldn't handle Proto's skeleton! Kemorig's hand tracking will let you use twist bones (or not) and let you tweak how the arms twist as you move your wrist.

• I'm determined to make blending in/out of hand tracking look better than how it usually looks with vtubers (there's so much jank!). For starters, it supports different easing functions for blending in vs out, and handles interrupting blends in/out, so you should never see hands pop.

  • This last point turned out to be especially challenging! It's a big reason why the release is falling back to May.

Sorry for the wait! I think this update is going to be a pretty massive improvement to kemorig avatars. I can't wait to start using hand tracking on my own streams.