face_multitask_v1
Licensing scope: See the license and provenance notice before relying on this card's license metadata for pretrained-weight redistribution or commercial use. Existing valid grants are preserved.
A single multi-task convolutional model for facial behavior analysis, used by
py-feat's Detectorv2. From one face crop
it jointly predicts action units, categorical emotion, valence/arousal,
eye gaze, a 478-point face mesh, and 6-DoF head pose.
- Backbone: ConvNeXt-V2 Tiny (FCMAE + IN-22k/IN-1k pretrained)
- Heads: ME-GraphAU AU graph (AFG/FGG/MEFL/SC) + unified-feature emotion/V-A and gaze heads (v2.3 architecture) + landmark and pose regression heads
- Params: ~30M · Input: 224×224 RGB (from a 256×256 face crop) · File:
face_multitask_v1.pt(174 MB)
Outputs
| Task | Output | Notes |
|---|---|---|
| Action Units | 24 probabilities [0,1] | AU01,02,04,05,06,07,09,10,11,12,14,15,16,17,18,20,23,24,25,26,27,28,43,45 |
| Emotion | 8-class softmax | Neutral, Happy, Sad, Surprise, Fear, Disgust, Anger, Contempt |
| Valence / Arousal | 2 × [−1,1] | tanh |
| Gaze | (yaw, pitch) radians | head-centric; yaw+ = right, pitch+ = up |
| Face mesh | 478 × (x,y,z) | MediaPipe topology, chip-pixel coords (z = relative depth) |
| Head pose | (yaw, pitch, roll, tx, ty, tz) | radians / pixels |
| 68 landmarks | derived | dlib-68 subset sampled from the 478 mesh |
Benchmarks (held-out, file-verified)
| Task | Dataset | Metric | Score |
|---|---|---|---|
| AU | DISFA+ (12-AU, Cheong protocol) | macro-F1 | 0.679 |
| Emotion | RAF-DB official test (7-cls) | acc / macro-F1 | 0.839 / 0.751 |
| Emotion | AffectNet val (7-cls, drop Contempt) | acc / macro-F1 | 0.304 / 0.264 |
| Valence/Arousal | Aff-Wild2 official validation | CCC (V / A) | 0.788 / 0.736 |
| Gaze | MPIIGaze | mean angular err | 3.33° |
| Gaze | Gaze360 | mean angular err | 5.81° |
Notes: emotion is strong on RAF-DB / Aff-Wild2 but weaker on AffectNet (label noise + class imbalance); AffectNet-specific, not architectural.
Usage
from feat import Detectorv2
detector = Detectorv2(device="cuda")
fex = detector.detect("image.jpg") # returns a py-feat Fex
The model expects a face crop produced by RetinaFace + py-feat's
extract_face_from_bbox_torch(frame, bbox, face_size=256, expand_bbox=1.2),
then center-cropped to 224 and ImageNet-normalized. Detectorv2 handles this.
License and provenance
The weights retain their existing noncommercial research designation. This older multitask checkpoint is distinct from the classic modular Detectorv1 family and from face_multitask_v28.safetensors.
The card identifies a ConvNeXt-V2 backbone, but the exact initialization checkpoint and complete training manifest for this artifact still need verification. ConvNeXt-V2 source code is MIT; its ImageNet pretrained and fine-tuned weights have separate noncommercial terms. The commonly used timm checkpoint is CC BY-NC 4.0. This is not evidence that this older artifact used that exact checkpoint, and the code license must not be represented as its weight license.
Dataset memberships, splits, teacher assets and permission scope must be established for this release. The v2.8 training/evaluation roles must not be applied retroactively. A research-only designation does not establish unrestricted public redistribution. See the multitask licensing notice and dataset register; their current-checkpoint inventory is not a manifest for this artifact.
This notice clarifies scope; it does not revoke existing valid grants or create a new license for third-party material. Dataset and teacher terms do not automatically relicense every trained artifact or inference output. A software license or model-card badge alone does not establish all checkpoint redistribution or commercial-use permissions.