Skip to content

Fail only the video whose points are too far apart to upload - #928

Merged
caglarpir merged 1 commit into
mapillary:mainfrom
caglarpir:upload-past-points-too-far-apart
Oct 26, 2026
Merged

caglarpir merged 1 commit into
mapillary:mainfrom
caglarpir:upload-past-points-too-far-apart

Conversation

@caglarpir

Copy link
Copy Markdown
Contributor

The duration of a sample in a CAMM track is a 32-bit count of the media timescale, so two consecutive points of an uploaded video can be at most 2^32 - 1 ticks apart: 19.9 hours at a timescale of 60000. Points read by ExifTool are timed by their GPS timestamps, so a dashcam video whose first GPS timestamp is a day older than the next one has two points that far apart. Building its upload failed an assertion, which the upload does not take for an error of one video: it stopped there, the videos after it were never uploaded, and every new attempt stopped at the same video.

Raise SampleDurationOverflowError instead, which names the two points, and fail the video with GPSPointsTooFarApartError, a SequenceError, so that the upload goes on with the next video, as it does for the other errors of one video. The error is logged with the file name and counted in the upload summary, and the upload exits with 0, as for those errors. As for the other errors before a video's upload starts, no upload event is sent for it.

A dashcam video whose first GPS timestamp was moved 22 hours back, between two other videos, with process_and_upload --dry_run --video_geotag_source exiftool_runtime:

  • main: the first video is uploaded, then AssertionError: expected timedelta 4784400000 ... to be <= UINT32_MAX, exit 1; the third video is never tried.
  • this PR: GPSPointsTooFarApartError: <video>: Points at 0.000 s and 79740.000 s of the video are 79740.000 s apart, more than the 71582.788 s that can be stored between two points in the uploaded video, the other two are uploaded, exit 0.

With the default sources this video is read natively and timed by its position in the video (since #843), and uploads on main already.

Unit and integration tests, doctests, mypy --python-version 3.9, ruff check, ruff format --check and usort diff are clean.

The duration of a sample in a CAMM track is a 32-bit count of the media
timescale, so two consecutive points of an uploaded video can be at
most 2^32 - 1 ticks apart: 19.9 hours at a timescale of 60000. Points
read by ExifTool are timed by their GPS timestamps, so a dashcam video
whose first GPS timestamp is a day older than the next one has two
points that far apart. Building its upload failed an assertion, which
the upload does not take for an error of one video: it stopped there,
and the videos after it were never uploaded. Run again, it stopped at
the same video every time.

Raise SampleDurationOverflowError instead, which names the two points,
and fail the video with GPSPointsTooFarApartError, a SequenceError, so
that the upload goes on with the next video, as it does for the other
errors of one video. The error is logged and counted in the upload
summary, and the upload exits with 0, as for those errors.
@meta-cla meta-cla Bot added the cla signed label Oct 26, 2026
@caglarpir
caglarpir merged commit 88ffaae into mapillary:main Oct 26, 2026
16 checks passed
caglarpir added a commit that referenced this pull request Oct 26, 2026
#928 and #929 fail a video with "Points at 0.000 s and 79739.000 s of
the video are 79739.000 s apart, more than the 71582.788 s that can be
stored between two points in the uploaded video": accurate, but it
gives the user nothing to act on.

Say instead what is wrong, in hours: "The GPS timestamps of the video
jump by 22.1 hours between 0.000 s and 79739.000 s, more than the
uploaded video can store (19.9 hours); a GPS timestamp of the video is
probably wrong". process and the upload give the same message, the
upload after the name of the video as before.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant