Repository navigation
Show progress while upload computes checksums - #942
Merged
Merged
Conversation
Before it uploads a sequence or a video, or skips it as already uploaded, upload computes the checksum of every file to look it up in the upload history. That reads every file in full, and showed nothing: for a large sequence on a slow drive, upload looked hung after "==> Uploading...", and an already uploaded sequence was skipped only after a long silence. Show a progress bar while it reads the files, as for the upload that follows, e.g. "Checksumming IMAGE (1/3)". Its speed shows how fast the drive is read.
caglarpir
added a commit
to caglarpir/mapillary_tools
that referenced
this pull request
Oct 30, 2026
- A track assumed to start with a video without GPS is never refused for its length, in process or in sampling, and the creation time is never used to place it: a MapillaryGPXAlignmentWarning says when the track and the video differ in length (mapillary#944). Only --interpolation_use_gpx_start_time still refuses a long track. - A file that a GPX or NMEA track missed fails with the miss when no later source geotags it (mapillary#938). - The process report lists the geotagging warnings (mapillary#945). - upload caches the checksums of the files, and shows progress while computing them (mapillary#942, mapillary#943).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Before it uploads a sequence or a video, or skips it as already uploaded,
uploadcomputes the checksum of every file to look it up in the upload history. That reads every file in full and showed nothing, so for a large sequence on a slow driveuploadlooked hung after==> Uploading..., and an already uploaded sequence was skipped only after a long silence.This shows a progress bar while it reads the files, like the upload bar that follows:
Its speed also shows how fast the drive reads. As with the other bars,
--verbosehides it. Checksums that the description file already gives are not computed again, and get no bar.Tests: unit tests for the new
md5sum_fpprogress callback and for_update_md5sums. Also checked by hand:processthenupload --dry_runtwice, of images and of a GoPro video.