Skip to content

Leave out ExifTool GPS points timestamped far from the others - #930

Merged
caglarpir merged 1 commit into
mapillary:mainfrom
caglarpir:exiftool-leave-out-far-gps-timestamps
Oct 26, 2026
Merged

caglarpir merged 1 commit into
mapillary:mainfrom
caglarpir:exiftool-leave-out-far-gps-timestamps

Conversation

@caglarpir

Copy link
Copy Markdown
Contributor

The ExifTool video reader times every point by its GPS timestamp, relative to the first point. A receiver at cold start, or a wrong date, can write a first timestamp far from the others: a dashcam video whose first GPS timestamp was 22 hours older than the next one had every other point 22 hours into the video. Where that overflows the sample duration of the uploaded video, #929 fails the whole video for one wrong point; below that limit (19.9 hours at a timescale of 60000, 49.7 days at a timescale of 1000), the video was processed and uploaded with its track that far past its end.

Before making the times relative to the first point, split the track where two consecutive GPS timestamps are more than MAPILLARY_TOOLS_MAX_GPX_GAP_SECONDS (1 hour) apart, keep the segment with the most points (the earliest of them on a tie), and make the times relative to its first point. The extractor warns once of the points left out, e.g. "1 GPS point of VIDEO was left out: its GPS timestamp is more than 3600 s from those of the other 60 points". Only the tracks timed by their GPS timestamps are split, the QuickTime, Insta360 and LIGO records; the GoPro tracks that ExifTool reads are timed by their samples and do not change. #929's check stays, for the tracks of other sources.

  • A BlackVue video whose first GPS timestamp in its ExifTool XML was made 22.1 hours older, with process --video_geotag_source exiftool_xml: 60 points from 0 to 59 s and one warning, the same track as from the unchanged XML. Main: 61 points, 60 of them 22 hours past the end of the 59 s video, and ready. The copy of the GPS log in the embedded low resolution recording keeps its right first timestamp, and is still not doubled.
  • A GKU D900 video (timescale 60000) with the same change: ready with 59 points from 0 to 58 s. Main: MapillaryGPSPointsTooFarApartError.
  • A dashcam video whose first GPS timestamp is 22 hours old, between two other videos, with process_and_upload --video_geotag_source exiftool_runtime: all three are uploaded, one warning. Main: that video fails.
  • A track with a real gap of more than an hour keeps the larger side of it.

Read through ExifTool, the tracks of 603 videos of many cameras, timelapses and BlackVue videos included, do not change, and none has a point left out. Compared with main on process, video_process and process_and_upload --dry_run, on images and videos of many cameras with the default and other sources, the outputs and exit codes are the same.

Unit and integration tests, doctests, mypy --python-version 3.9, ruff check, ruff format --check and usort diff are clean.

The ExifTool video reader times every point by its GPS timestamp,
relative to the first point. A receiver at cold start, or a wrong date,
can write a first timestamp far from the others: a dashcam video whose
first GPS timestamp was 22 hours older than the next one had every
other point 22 hours into the video. Where that overflows the sample
duration of the uploaded video, mapillary#929 fails the whole video for one
wrong point; below that limit (19.9 hours at a timescale of 60000, 49.7
days at a timescale of 1000), the video was processed and uploaded with
its track that far past its end.

Before making the times relative to the first point, split the track
where two consecutive GPS timestamps are more than
MAPILLARY_TOOLS_MAX_GPX_GAP_SECONDS (1 hour) apart, keep the segment
with the most points (the earliest of them on a tie), and make the
times relative to its first point. The extractor warns once of the
points left out. Only the tracks timed by their GPS timestamps are
split, the QuickTime, Insta360 and LIGO records; the GoPro tracks that
ExifTool reads are timed by their samples and do not change.
@meta-cla meta-cla Bot added the cla signed label Oct 26, 2026
@caglarpir
caglarpir merged commit aa9d3f6 into mapillary:main Oct 26, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant