feat: Add mutable infer to input graphs, eye expressions by dfgHiatus · Pull Request #253 · Project-Babble/Baballonia

dfgHiatus · 2026-05-26T19:45:26Z

No description provided.

CLAassistant · 2026-05-26T19:46:02Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ SummerSigh
❌ AeroScripts
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

dfgHiatus · 2026-05-26T19:55:00Z

-    private float[] dxPrev;
-    private DateTime tPrev;
-    public OneEuroFilter(float[] x0, float minCutoff = 1.0f, float beta = 0.0f)
+    private const float TwoPi = 2.0f * MathF.PI;


Use Math.Tau here!

dfgHiatus

Make sure to run things through a code formatter once you're finished! Thank you :)

dfgHiatus · 2026-05-28T23:44:41Z

+    private bool _hasModelMetadata;
+    private OrderedFloatMap _outputs;
+
+    private readonly List<List<string>> _knownMappings = new()


Two things:

Instead of a list of lists, let's make the outer collection a Dictionary<enum, <List<string>> so we can more explicitly index into it. IE _knownMappings[0] -> List becomes _knownMappings[ExpressionMappings.Eye] -> List. Yes we don't really index into the list anywhere but I don't want this sneaking up on us.

For (reused) strings, let's move this to a static class full of const strings.

I also see there's some string parsing in the VRCFT module, this would help us re-use/update strings in one "shared" file .

dfgHiatus · 2026-05-28T23:46:00Z

After this PR gets merged in, we need to implement separate filters for gaze, eye and face expressions.

dfgHiatus · 2026-05-28T23:47:54Z

+            }
        }
+
+        _logger.LogInformation("Initialized model that predicts {Expressions}", string.Join(", ", _outputExpressionNames));


Change this to LogDebug, this is more for us and less for the end user.

Suggested change

_logger.LogInformation("Initialized model that predicts {Expressions}", string.Join(", ", _outputExpressionNames));

_logger.LogDebug("Initialized model that predicts {Expressions}", string.Join(", ", _outputExpressionNames));

dfgHiatus · 2026-05-28T23:49:13Z

+            var outputSize = _session.OutputMetadata.Values.First().Dimensions[1];
+            foreach(List<string> mapping in _knownMappings)
+            {
+                if(mapping.Count == outputSize)


Shouldn't we throw a warning/error here if we're not able to load a face model at all? IE if model input size doesn't match what were are expecting

dfgHiatus · 2026-05-28T23:50:27Z


    public override void Update()
    {
-        if (needsEye)


We need to keep the use of needsEye and needsExpression. I forget the exact reason why but maybe @benaclejames can chime in on this.

dfgHiatus · 2026-05-28T23:52:04Z

Don't forget to bump the version number (we're on 3.2.0 now IIRC) as well the the module.json!

dfgHiatus · 2026-05-28T23:55:48Z

-            xPrev = (float[])x.Clone();
-            return x;
+            for (int i = 0; i < length; i++)
+            {


If all we're doing here is copying a span, wouldn't it make more sense to use Span.CopyTo or Span.TryCopyTo?

dfgHiatus · 2026-05-28T23:58:40Z

+    }
+
+    // Expose the raw array/span directly so the ONNX runner can write to it
+    public Span<float> ValuesSpan => _values;


I noticed in https://github.com/Project-Babble/Baballonia/pull/253/changes#diff-5431c54da43664d697c115e0d9478a7e3a1f380cd37ff0a176042848e2d82685R49 we're doing something like this:

OrderedFloatMap x; ReadOnlySpan<float> xSpan = x.ValuesSpan; int length = _xPrev.Length;

Should we return a ReadOnlySpan here instead of a Span? Why/why not?

Also, is there any difference between:

public Span<float> ValuesSpan => _values;

And

public Span<float> ValuesSpan => _values.ToSpan();

I think there's an implicit conversion happening here, but I'm not too certain 🤔

dfgHiatus · 2026-05-29T00:00:13Z

-            }
-        }
-
        Thread.Sleep(10);


If this method will be empty we should be able to delete this method outright. Any blocking will happen in BabbleOsc

Store a cloned orderedKeys array and use it for Keys and enumeration. This makes sure enumeration follows the provided key order (rather than relying on FrozenDictionary iteration) and protects against external mutation of the input array by cloning it in the constructor.

dfgHiatus · 2026-06-01T07:52:07Z

+    private void ProcessNativeVrcEyeTracking(OrderedFloatMap expressions, ConcurrentQueue<OscMessage> queue)
    {
-        var leftEyeX = expressions[0];
-        var leftEyeY = expressions[1];
-        var leftEyeLid = expressions[2];
-        var rightEyeX = expressions[3];
-        var rightEyeY = expressions[4];
-        var rightEyeLid = expressions[5];
+        var leftEyeX = expressions["/leftEyePitch"];
+        var leftEyeY = expressions["/leftEyeYaw"];
+        var leftEyeLid = expressions["/leftEyeLid"];
+        var rightEyeX = expressions["/rightEyePitch"];
+        var rightEyeY = expressions["/rightEyeYaw"];
+        var rightEyeLid = expressions["/rightEyeLid"];


@acmdf reported these OSC addresses need to be updated. This may also effect DFR.

Suggested change

private void ProcessNativeVrcEyeTracking(OrderedFloatMap expressions, ConcurrentQueue<OscMessage> queue)

{

var leftEyeX = expressions[0];

var leftEyeY = expressions[1];

var leftEyeLid = expressions[2];

var rightEyeX = expressions[3];

var rightEyeY = expressions[4];

var rightEyeLid = expressions[5];

var leftEyeX = expressions["/leftEyePitch"];

var leftEyeY = expressions["/leftEyeYaw"];

var leftEyeLid = expressions["/leftEyeLid"];

var rightEyeX = expressions["/rightEyePitch"];

var rightEyeY = expressions["/rightEyeYaw"];

var rightEyeLid = expressions["/rightEyeLid"];

private void ProcessNativeVrcEyeTracking(OrderedFloatMap expressions, ConcurrentQueue<OscMessage> queue)

{

var leftEyeX = expressions["/leftEyeX"];

var leftEyeY = expressions["/leftEyeY"];

var leftEyeLid = expressions["/leftEyeLid"];

var rightEyeX = expressions["/rightEyeX"];

var rightEyeY = expressions["/rightEyeY"];

var rightEyeLid = expressions["/rightEyeLid"];

DFR route should be handled fine I believe, still probably worth checking before release though

dfgHiatus · 2026-06-27T21:30:16Z

Superseded by #256

AeroScripts added 4 commits May 21, 2026 19:11

Push inference stack update

a0b41e3

Optimizations+fixes

823c40e

Prepend parameter names with "/" to match standard

78c6b96

Module rework

10d2894

dfgHiatus requested a review from AeroScripts May 26, 2026 19:45

dfgHiatus self-assigned this May 26, 2026

dfgHiatus added the enhancement New feature or request label May 26, 2026

dfgHiatus commented May 26, 2026

View reviewed changes

dfgHiatus commented May 29, 2026

View reviewed changes

AeroScripts and others added 5 commits May 30, 2026 00:43

Fix parameter mapping, optimizations

9385e2b

Fix calibration tab, enable eye expr sliders

0cf5390

Fix incorrect case

7fbd5b5

Fix calibration loading

e1fd379

dfgHiatus commented Jun 1, 2026

View reviewed changes

dfgHiatus closed this Jun 27, 2026

	_logger.LogInformation("Initialized model that predicts {Expressions}", string.Join(", ", _outputExpressionNames));
	_logger.LogDebug("Initialized model that predicts {Expressions}", string.Join(", ", _outputExpressionNames));

Uh oh!

Conversation

dfgHiatus commented May 26, 2026

Uh oh!

CLAassistant commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dfgHiatus May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dfgHiatus left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dfgHiatus commented Jun 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

CLAassistant commented May 26, 2026 •

edited

Loading

dfgHiatus May 26, 2026 •

edited

Loading