Repository navigation
Fail in process a video whose points are too far apart to upload - #929
Merged
caglarpir merged 2 commits intoOct 26, 2026
Merged
Conversation
The duration of a sample in a CAMM track is a 32-bit count of the media timescale, so two consecutive points of an uploaded video can be at most 2^32 - 1 ticks apart: 19.9 hours at a timescale of 60000. Points read by ExifTool are timed by their GPS timestamps, so a dashcam video whose first GPS timestamp is a day older than the next one has two points that far apart. Building its upload failed an assertion, which the upload does not take for an error of one video: it stopped there, and the videos after it were never uploaded. Run again, it stopped at the same video every time. Raise SampleDurationOverflowError instead, which names the two points, and fail the video with GPSPointsTooFarApartError, a SequenceError, so that the upload goes on with the next video, as it does for the other errors of one video. The error is logged and counted in the upload summary, and the upload exits with 0, as for those errors.
The previous commit fails the upload of a video with two consecutive points more than 2^32 - 1 ticks of the media timescale apart, as the uploaded video cannot store the time between them. process still reported such a video as ready, and nothing in its summary said that it would not upload. Check it in process as well, after the other checks of a video, the way the upload builds the video: against the timescale in its mvhd box, raised to 1000, and leaving out the points before the start of the video. Such a video fails with MapillaryGPSPointsTooFarApartError, which names the two points, so the summary counts it with the other errors and the upload does not try it. A video whose mvhd box cannot be read is not checked here; the upload cannot read it either.
This was referenced Oct 26, 2026
caglarpir
added a commit
that referenced
this pull request
Oct 26, 2026
The ExifTool video reader times every point by its GPS timestamp, relative to the first point. A receiver at cold start, or a wrong date, can write a first timestamp far from the others: a dashcam video whose first GPS timestamp was 22 hours older than the next one had every other point 22 hours into the video. Where that overflows the sample duration of the uploaded video, #929 fails the whole video for one wrong point; below that limit (19.9 hours at a timescale of 60000, 49.7 days at a timescale of 1000), the video was processed and uploaded with its track that far past its end. Before making the times relative to the first point, split the track where two consecutive GPS timestamps are more than MAPILLARY_TOOLS_MAX_GPX_GAP_SECONDS (1 hour) apart, keep the segment with the most points (the earliest of them on a tie), and make the times relative to its first point. The extractor warns once of the points left out. Only the tracks timed by their GPS timestamps are split, the QuickTime, Insta360 and LIGO records; the GoPro tracks that ExifTool reads are timed by their samples and do not change.
caglarpir
added a commit
that referenced
this pull request
Oct 26, 2026
#928 and #929 fail a video with "Points at 0.000 s and 79739.000 s of the video are 79739.000 s apart, more than the 71582.788 s that can be stored between two points in the uploaded video": accurate, but it gives the user nothing to act on. Say instead what is wrong, in hours: "The GPS timestamps of the video jump by 22.1 hours between 0.000 s and 79739.000 s, more than the uploaded video can store (19.9 hours); a GPS timestamp of the video is probably wrong". process and the upload give the same message, the upload after the name of the video as before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #928; review the last commit.
#928 fails the upload of a video with two consecutive points more than 2^32 - 1 ticks of the media timescale apart, as the uploaded video cannot store the time between them.
processstill reported such a video as ready, and nothing in its summary said that it would not upload.Check it in
processas well, after the other checks of a video, the way the upload builds the video: against the timescale in its mvhd box, raised to 1000, and leaving out the points before the start of the video. Such a video fails withMapillaryGPSPointsTooFarApartError, which names the two points, so the summary counts it with the other errors and the upload does not try it. As for those errors,process_and_uploadwithout--skip_process_errorsnow stops before uploading (exit 6); with it, the other videos are uploaded. A video whose mvhd box cannot be read is not checked here; the upload cannot read it either. Only the videos that pass the other checks are opened again, to read that box.On the same three videos as in #928, with
--video_geotag_source exiftool_runtime,processreports 2 ready and 1MapillaryGPSPointsTooFarApartError(main: 3 ready). With the default sources all three are ready and uploaded, as on main.Compared with main on
process,video_processandprocess_and_upload --dry_run, on images and videos of many cameras with the default and other sources, the outputs and exit codes are the same.Unit and integration tests, doctests,
mypy --python-version 3.9,ruff check,ruff format --checkandusort diffare clean.