A talking-head model can score perfectly on every standard
benchmark and still feel completely dead on a real call. CJFD
quality is the score we built to catch that — and it's the one
number on the page that correlates with how the product
feels.
On a real video call the user is talking half the time, and the
avatar should be listening — nodding, looking at the
camera, leaning in. Most standard benchmarks (FID, FVD, PFD,
blinks, SyncNet, rPCC) don't measure that at all; they only check
whether the avatar's face looks natural in isolation. So an
idle-loop scores nearly as well as a real human while feeling
obviously wrong the moment a user tries it.
CJFD asks a different question: is the avatar reacting to
this specific person on the other end of the
call, or to anyone in general?
TLCC-Proactivity in the table above is the only
older metric that even tries to capture this — it correlates the
listener's face with the speaker's face (motion features) and
reports whether the model overshoots GT-level coupling. It's a
decent proxy, but it uses the speaker's face as the
reference signal. In a real call the avatar doesn't see the
partner's face, it hears the partner's voice — so
face↔face coupling is the wrong signal: it tells you whether
two video tracks co-move, not whether the avatar responds to
what's being said.
CJFD is the first metric in this set that brings the partner's
audio into the evaluation and scores the
avatar's face jointly with it. That's the same signal
the avatar actually receives in production. Once the audio is
part of the score, we can run two pairings and watch what
happens:
- Real — avatar paired with the partner who
was actually speaking to it.
- Shuffled — same avatar, partner from a
different conversation.
A reactive model scores much worse with the wrong audio. An
idle-loop scores the same either way. The gap is what CJFD
measures — across three independent components (the
CJFD marginal_fit, CJFD specificity,
and CJFD FMD gain rows in the table), then
combined with a geometric mean into
CJFD quality (composite) so a model has to pass
all three at once. FMD cos_mean is the raw
frame-level cosine distance behind FMD gain, useful when you
want to inspect the point-wise signal directly. Hover any row for
the exact formula.
On Seamless Interaction, AVTR-1 scores
CJFD quality = 0.765 — the only model in the
comparison where all three components are high simultaneously.
Every other system lands between 0.71 and 0.73. The spread
between models on CJFD quality is the spread users feel on a
real call.
