Reads the layouts FreeMoCap v1 and v2 write, dispatching on the column names rather than the file name:
<recording>_by_frame.csv: the v1 tidy export. FreeMoCap added areprojection_errorcolumn at v1.7.4, so this exists in an 8- and a 9-column form; both are read.<recording>_by_trajectory.csv: v1's one column triple per tracked point, with the camera timestamps alongside.output_data/mediapipe_*_3d_xyz.csv: v1's per-model wide files, one model each and no timestamps.output_data/freemocap_data_by_frame.csv, and the.parquetbeside it: the v2 tidy export, one row per keypoint per trajectory per frame (seetrajectory). Reading the Parquet file needs the arrow package.output_data/<tracker>_<aspect>_<trajectory>.csv: v2's per-trajectory files, such asmediapipe_body_3d_xyz.csv. They keep v1's names but are long (frame,keypoint,x,y,z), so they are told from v1's wide files by their columns. Their model is read from the file name.
Which layout was read is recorded in the source_format metadata field.
Point names are parsed the way FreeMoCap's own data saver parses them, so
the same recording gives the same model and keypoint values whichever
layout it is read from.
Arguments
- path
Path to a FreeMoCap CSV, or a v2 Parquet file.
- format
Export layout.
"auto"(default) reads it from the column names; naming one requires that layout and errors on anything else. The layout names may change while one convention for readers of a source with several export layouts is settled (#118).- trajectory
Which positions to read from the v2 tidy export:
"3d_xyz"(default), the triangulated, filtered keypoints, or"rigid_3d_xyz", the same keypoints with bone lengths held constant. FreeMoCap only computesrigid_3d_xyzfor aspects with a joint hierarchy (the body), so other aspects keep3d_xyzand a message says which. Ignored, with a warning, for every other layout.
Value
An aniframe with time, model, keypoint, confidence and
x/y/z in millimetres on a 3D cartesian coordinate system. time
is seconds elapsed from start_datetime where the layout carries
timestamps, and frames where it does not. Either way the first frame is
at 0; see "Time" in read_dataset().
Details
confidence comes from reprojection_error, which the 9-column
by_frame export and the v2 tidy export carry; every other layout gives
all-NA confidence. The two run in opposite directions: a reprojection
error is a distance in pixels, so zero is perfect and larger is worse,
whereas every other reader in aniread fills confidence from a likelihood
or a probability where larger is better. Storing the error unchanged would
make aniprocess::mask_na_across(method = "confidence") mask the best
points, so it is mapped through
$$confidence = 1 / (1 + error)$$
which is monotone decreasing onto \((0, 1]\): a zero error gives 1.
The mapping is invertible, so the original error is recoverable as
1 / confidence - 1.
FreeMoCap v2
v2 writes its output through skellyforge. Its tidy export names each model
<tracker>.<aspect> (mediapipe.body, rtmpose.left_hand); these are
read as <tracker>_<aspect> (mediapipe_body, rtmpose_left_hand), the
naming of v1's model column and of v2's own per-trajectory file names.
The hands keep separate models, since v2's hand keypoint names do not say
which hand they belong to.
Each keypoint appears once per trajectory type, so only one set of
positions is read, chosen by trajectory. The centres of mass
(total_body_center_of_mass and segment_center_of_mass) are added as
keypoints of a <tracker>_com model, as v1 kept them in mediapipe_com,
with the keypoint names v2 gives them. (Only the body has centres of mass
in FreeMoCap's models; another aspect's would go to <tracker>_<aspect>_com,
so that segment names cannot collide.) Any other trajectory type is not
read.
v2 writes no timestamps, so time is the frame number and sampling_rate
is left unset. FreeMoCap's main pipeline does not attach reprojection
errors to the skeleton it saves, so confidence is usually all NA too.
detect_source() recognises the v2 tidy CSV. It does not claim the
per-trajectory files, whose five columns any tidy export could have, or
the Parquet file; read those with read_freemocap() directly.
References
Matthis, J. S., Queen, P., Cherian, A., Wirth, T., Idehen, E., et al. FreeMoCap: a free, open-source markerless motion capture system (software). doi:10.5281/zenodo.7233713 .
Examples
path <- system.file("extdata", "freemocap.csv", package = "aniread")
read_freemocap(path)
#> # Models: mediapipe_body, mediapipe_com, mediapipe_face, mediapipe_hand
#> # Keypoints: left_ear, left_eye, left_eye_inner, left_eye_outer, left_shoulder,
#> # mouth_left, mouth_right, nose, right_ear, right_eye, right_eye_inner,
#> # right_eye_outer, right_shoulder, full_body, head, trunk, 0000, left_0000,
#> # right_0000
#> model keypoint time x y z confidence
#> <fct> <fct> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 mediapipe_body left_ear 0 -234. -539. 1592. 0.154
#> 2 mediapipe_body left_ear 1 -210. -664. 1543. 0.0482
#> 3 mediapipe_body left_ear 2 -199. -520. 1588. 0.0771
#> 4 mediapipe_body left_eye 0 -194. -616. 1599. 0.0547
#> 5 mediapipe_body left_eye 1 -224. -632. 1586. 0.152
#> 6 mediapipe_body left_eye 2 -218. -563. 1553. 0.0837
#> 7 mediapipe_body left_eye_inner 0 -153. -698. 1535. 0.167
#> 8 mediapipe_body left_eye_inner 1 -168. -673. 1584. 0.0877
#> 9 mediapipe_body left_eye_inner 2 -245. -619. 1665. 0.153
#> 10 mediapipe_body left_eye_outer 0 -243. -556. 1639. 0.120
#> # ℹ 47 more rows
# The same recording in its by_trajectory form
path <- system.file(
"extdata",
"freemocap_by_trajectory.csv",
package = "aniread"
)
read_freemocap(path)
#> # Models: mediapipe_body, mediapipe_com, mediapipe_face, mediapipe_hand
#> # Keypoints: left_ear, left_eye, left_eye_inner, left_eye_outer, left_shoulder,
#> # mouth_left, mouth_right, nose, right_ear, right_eye, right_eye_inner,
#> # right_eye_outer, right_shoulder, full_body, head, trunk, 0000, left_0000,
#> # right_0000
#> model keypoint time x y z confidence
#> <fct> <fct> <int> <dbl> <dbl> <dbl> <dbl>
#> 1 mediapipe_body left_ear 0 -234. -539. 1592. NA
#> 2 mediapipe_body left_ear 1 -210. -664. 1543. NA
#> 3 mediapipe_body left_ear 2 -199. -520. 1588. NA
#> 4 mediapipe_body left_eye 0 -194. -616. 1599. NA
#> 5 mediapipe_body left_eye 1 -224. -632. 1586. NA
#> 6 mediapipe_body left_eye 2 -218. -563. 1553. NA
#> 7 mediapipe_body left_eye_inner 0 -153. -698. 1535. NA
#> 8 mediapipe_body left_eye_inner 1 -168. -673. 1584. NA
#> 9 mediapipe_body left_eye_inner 2 -245. -619. 1665. NA
#> 10 mediapipe_body left_eye_outer 0 -243. -556. 1639. NA
#> # ℹ 47 more rows
# FreeMoCap v2's tidy export, reading the rigid-body positions
path <- system.file("extdata", "freemocap_v2.csv", package = "aniread")
read_freemocap(path, trajectory = "rigid_3d_xyz")
#> ℹ No "rigid_3d_xyz" for "rtmpose_left_hand"; read "3d_xyz" for it instead.
#> # Models: rtmpose_body, rtmpose_com, rtmpose_left_hand
#> # Keypoints: head_center, hips_center, left_ankle, left_big_toe, left_ear,
#> # left_elbow, left_eye, left_heel, left_hip, left_knee, left_shoulder,
#> # left_small_toe, left_wrist, neck_center, nose, right_ankle, right_big_toe,
#> # right_ear, right_elbow, right_eye, right_heel, right_hip, right_knee,
#> # right_shoulder, right_small_toe, right_wrist, trunk_center, head,
#> # left_foot, left_forearm, left_shank, left_thigh, left_upper_arm,
#> # right_foot, right_forearm, right_shank, right_thigh, right_upper_arm,
#> # spine, total_body_center_of_mass, forefinger1, forefinger2, forefinger3,
#> # forefinger4, hand_root, middle_finger1, middle_finger2, middle_finger3,
#> # middle_finger4, pinky_finger1, pinky_finger2, pinky_finger3, pinky_finger4,
#> # ring_finger1, ring_finger2, ring_finger3, ring_finger4, thumb1, thumb2,
#> # thumb3, thumb4
#> model keypoint time x y z confidence
#> <fct> <fct> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 rtmpose_body head_center 0 -158. -46.4 1178. NA
#> 2 rtmpose_body head_center 1 -158. -44.3 1177. NA
#> 3 rtmpose_body head_center 2 -159. -45.2 1176. NA
#> 4 rtmpose_body hips_center 0 -101. 156. 1287. NA
#> 5 rtmpose_body hips_center 1 -101. 155. 1286. NA
#> 6 rtmpose_body hips_center 2 -101. 153. 1286. NA
#> 7 rtmpose_body left_ankle 0 -197. 50.2 625. NA
#> 8 rtmpose_body left_ankle 1 -196. 48.5 624. NA
#> 9 rtmpose_body left_ankle 2 -196. 46.7 626. NA
#> 10 rtmpose_body left_big_toe 0 195. 124. 1673. NA
#> # ℹ 173 more rows