kemorig Status Update - May 27

Hi everyone, here's what's going on with kemorig!

Known Issues with Current Build

I've been using kemorig 0.7 on my stream for a few weeks now. Hand tracking has been a huge boon! But there are few issues myself and others have run into:

• Auto-reconnecting tracking devices doesn't work in the app (but does in the editor)

• The "Kemorig Tracking" animation node has an issue where it discards any morph target animation performed by Sequences earlier than it in the Anim Graph.

• There is a (rare!) crash bug in the app when a leap motion device is plugged-in on your PC.

  • (fun fact: I think this is the first crash in a kemorig release. Sad, but at least it's rare!)

• There is an edge case if your Leap rapidly detects/un-detects a hand, your avatar's hand won't smoothly blend and snaps back and forth.

  • (this is kind of hard/time consuming to improve and might not get a fix right away)

These issues are being targeted for the next kemorig release.

Next Features on the Horizon

Here is a shortlist of features that I'm targeting for releases in the near future:

Expressions System

This is the next big feature I want to do.

I'd like to make an "Expressions" system, where you can map your real-life face poses with canned facial animation. As in: smile at the camera, avatar plays a smile that you authored. This is especially useful for webcam tracking, due to rudimentary face tracking compared to ARKit.

I see it working like this:

• In the editor, you make a special Stance Group that is flagged as "Expressions". You can enter Morph Target name(s) or a Sequence to play. You can enter as many Expression Stances as you like, and give them any name you want.

• In the app, there will be a new "Expressions" menu, and your Stance Group will be detected and listed inside it. You can click a record button next to each Expression, and make a face at the camera from different angles.

• Then, the app will detect when faces you make are close to what you recorded, and blend between Expressions.

Again, this is a whole lot of work, but it's an important feature! I remember using VSeeFace's version of this back before I used ARKit.

More Hand Tracking Controls

I want to add more options to help polish hand tracking on avatars. Things like:

• boundaries for detecting hands (e.g. so you can put your hands down on your desk and kemorig always un-tracks them)

• boundaries the avatar's hands will not enter (e.g. so your avatar never tries to put its hands inside its torso or behind its back)

• more per-stance tweakables (e.g. modifying the origin of hand tracking for certain stances)

Twitch Integration

I haven't forgotten! However, Twitch Integration keeps getting kicked down the road because it's a) very complicated and b) very easy to use streamer.bot or SAMMI instead to integrate with Twitch. This will probably be one of the last features that gets put it before kemorig becomes 1.0.

What Do You Want To See Next?

Is there something I'm missing or overlooking? Got new ideas? Feel free to comment below with suggestions! I'm always open to hearing what is giving people trouble, and what people are hoping to see next.

Discord

Another reminder that if you want to share creations or get help, we have a kemorig section in the discord! Link here: DISCORD

-PROTO