Perception Quality and Sensor Fusion

When two sensors agree, can the robot trust them?

Not automatically. Two sensors can agree because both are stale, misaligned, biased, or observing different moments. Fusion begins by deciding whether evidence is comparable.

Measure the error that matters

For a target detector:

• True positive (TP): target correctly reported.
• False positive (FP): target reported when absent.
• False negative (FN): target present but missed.
• Precision = TP / (TP + FP): when the model reports a target, how often is it correct?
• Recall = TP / (TP + FN): how many real targets did it find?

A threshold can trade recall for precision. The right balance depends on the action and the cost of each error.

Worked example

Across 50 labeled scenes, a detector produces TP=32, FP=8, and FN=10.

precision = 32/40 = 0.80 and recall = 32/42 ≈ 0.76.

That is not a universal quality score. Break results down by conditions: bright/dim light, near/far range, clear/occluded targets, and familiar/unfamiliar backgrounds. NIST emphasizes realistic test sets and ongoing deployed monitoring because performance can fail outside development conditions.

Retrieval pause: If missed people are more harmful than extra alerts, which metric deserves special attention? Recall, while still managing false alarms.

Distribution shift

Training examples do not define the whole world. New lighting, camera placement, weather, clothing, clutter, or sensor aging can change the input distribution. Monitor condition slices, not only an average. When the system operates beyond tested limits, reduce authority or request human review.

Fuse only comparable evidence

Before combining camera and lidar evidence, run an evidence gate:

1. Time: are acquisition timestamps close enough for the robot's motion?
2. Frame: are measurements transformed into the same named frame?
3. Validity: are values within documented range and free of invalid codes?
4. Uncertainty: does each source report a defensible error model?
5. Independence: could both share the same failure cause?

ROS ApproximateTimeSynchronizer matches messages using header timestamps and a configured tolerance. Arrival time is not a safe substitute when delays are unpredictable.

A simple uncertainty-weighted estimate

For independent scalar measurements z1 and z2 with variances v1 and v2, a teaching approximation is:

z_fused = (z1/v1 + z2/v2) / (1/v1 + 1/v2).

Lower variance receives more weight, but a precise biased sensor can still dominate incorrectly. Gate first, fuse second, then monitor residual disagreement.

Misconception checks

• Confidence is not probability of truth or safety.
• More sensors do not guarantee better estimates.
• Synchronization by arrival time can combine different physical moments.
• High average accuracy can hide failure in a critical condition slice.

Summary

Perception quality is conditional evidence. Measure task-relevant errors, test deployment conditions, preserve time and frame, gate invalid inputs, fuse uncertainty-aware measurements, and fail conservatively when evidence conflicts.

Further learning

• ROS message_filters
• NIST AI RMF 1.0