← Ledger · Entry 01

FineART: structurally clean, semantically unmeasured

Six sampled episodes of Scale AI × Hugging Face's 1,718-hour bimanual dataset pass every structural check available. The dataset's distinguishing feature — 533,913 dense subtask labels — is not reachable by any metric in current tooling. The default configuration of the audit tool also rejects 83% of legitimate episodes. Both statements are measurements.

FineART is a bimanual manipulation corpus recorded on a Trossen Stationary AI system with ALOHA2 grippers: four cameras at 360×640 and 30 fps, 14-dimensional joint-plus-gripper state and action, roughly 1.86 TB on the Hub. The property that separates it from other open bimanual datasets is annotation density: 533,913 human-written subtask segments, about 13 per episode, covering the full recorded duration rather than a filtered subset.

The sample is the first six episodes of data/chunk-000/file-000.parquet (116 MB) plus the 24 corresponding MP4s, four cameras per episode (≈130 MB). The subset was materialised as a standalone dataset directory with info.json rewritten to the subset's true total_episodes and total_frames, so that episode-count checks would be evaluated against real values rather than the full corpus.

All six episodes are drawn from a single task family (pick-and-place of candle holders). That is deliberate — it tests whether format handling works — and it is also the reason no distributional claim is made below.

One diagnostic surfaced that is worth flagging to the training side rather than as a defect: action_discontinuity reported 108 discontinuities across 5 episodes. With 30 fps teleoperation and 50 Hz control, single-frame action jumps are expected at grasp and release events; the count is consistent with that reading. It is recorded here because it is the one structural signal that is non-zero, not because it indicates damage.

A clean result is only meaningful if the checker can fail. To test that, two defects were injected into a copy of the subset and the audit re-run:

• Episode 3: 10 frames of observation.state set to NaN (dimension 2)
• Episode 4: 50 consecutive frames deleted from the middle, leaving a frame_index gap of 50

Both defects were caught, attributed to the correct episode, and reported with the exact count. The four untouched episodes stayed clean. Detection works. The failures below are failures of default sensitivity and of coverage, not of mechanism.

Run with defaults, the audit excludes five of six episodes on a single rule, video_freeze_while_moving. Run with the motion source switched to the state stream, the same data yields zero exclusions.

Independent re-measurement

The tool's verdict was not accepted. All 36 flagged regions were decoded again at full resolution with an independent script, and per-frame mean absolute difference was computed in grayscale — first at the tool's working resolution (64×64), then at source resolution.

The picture is moving in all 36 regions. It is moving little: a bird's-eye camera over a 640×360 frame in which the robot occupies a small fraction of the sensor. Downsampled to 64×64 grayscale, with a fixed epsilon floor of 0.10 on frame difference, that motion approaches the noise floor. The motion gate, meanwhile, is driven by the action stream, which at 30 fps is essentially never still. “Action is moving” plus “pixels are barely changing” is then scored as a frozen camera, i.e. corrupt video.

Practical consequence: the default invocation cannot be used as an acceptance gate on this dataset. It rejects 83% of valid episodes and returns exit code 2. Either pass --freeze-motion-source state, or whitelist the two static cameras, or make epsilon adaptive per camera — none of which is currently reachable from the CLI without editing code.

This section is about the tool, not the dataset. Each row is a property FineART has and current tooling has no metric for — recorded so that a clean verdict is not mistaken for full coverage.

The joint_limit row matters more than it looks. It is the only check in the entire tool that asserts a physical constraint on the data; on FineART it is silently inapplicable because the dataset does not declare joint limits and the tool has no profile for this robot.

Because no metric covers it, a standalone script was written to characterise the subtask timeline. Scope: all 242 episodes in data/chunk-000/file-000.parquet, 2,108 subtask segments. This is a larger sample than the audit run and covers a wider task range.

The annotation is internally consistent: counts agree with metadata, timestamps never invert, nothing is blank. The one number that has operational consequences is the leading gap. The dataset card describes roughly the first two seconds as having no active subtask; the measured median is 2.20 s, which matches. But the tail is longer than the median suggests — 11% of episodes start later than 5 s, the maximum is 14.10 s, and in 235 of 242 episodes the gap exceeds one second.

For a loader that resolves the active subtask by “most recent segment at or before t”, frames before the first segment have no answer. That branch is not an edge case; it fires in essentially every episode. This is a property of the data layout, not a defect, and it must be handled explicitly rather than discovered at training time.

The CLI exposes no concurrency. A full video-layer pass is therefore not a single command but a sharded job: split by chunk, run in parallel, aggregate the JSON. The parquet layer is cheap enough to run whole.

# install (video layer needs av + opencv)
pip install robot-data-audit[parquet] "av>=12" "opencv-python-headless"

# structural pass only — fast, whole corpus is feasible
rda audit /data/FineART --no-video --format json -o fineart_novideo.json

# video layer on a shard — 0.12.1 only: the state stream was needed to suppress the false positive
rda audit /data/FineART_fold0 --freeze-motion-source state --format json -o fold0.json

# video layer, 0.12.2 criterion — the re-run quoted in the note at the top of this entry
rda audit /data/FineART_fold0 --format json -o fold0_0122.json

# render a shareable report
python tools/rda_render.py fold0.json --html report.html --badge badge.svg

Reading discipline: a clean --no-video run says nothing about the video layer. On 0.12.1 the video layer had to be run with --freeze-motion-source state on FineART, or it would exclude most of the corpus — that was a defect in the freeze criterion, not a property of the data. From 0.12.2 the default criterion is correct here and the flag is no longer needed.

Raw output from the runs described above, kept unedited so that any number in this entry can be traced back to the file that produced it.