Skip to content

README.md

Latest commit

 

History

History
217 lines (149 loc) · 5.48 KB

README.md

File metadata and controls

217 lines (149 loc) · 5.48 KB
Raw

Dataset Overview

This dataset is part of an ongoing research project focused on camera performance comparison and depth estimation. It contains recordings and logs collected across multiple environments, distances, and camera configurations.

This README provides a high-level description of the dataset structure and instructions for downloading and restoring the data from Dataverse.

Dataset Size Warning ⚠️

Before downloading or extracting the dataset, please ensure you have sufficient disk space.

  • Compressed size: approximately 140 GB
  • Uncompressed size: approximately 380 GB

Extraction will temporarily require additional space.
Running out of disk space during decompression may result in corrupted or incomplete data.

1. Top-Level Directory Structure

The root of the dataset contains two main folders:

root/
├── README.md
├── artificial/
└── natural/

These represent two types of environments:

  • artificial: controlled lighting conditions
  • natural: real-world lighting conditions

Both folders follow exactly the same internal structure.

2. Distance-Based Subdirectories

Inside each environment folder, data is organized by distance. Folder names correspond to distances in millimeters (e.g. 0400 = 400 mm):

artificial/
├── 0400
├── 0600
├── 0800
├── 1000
├── 1400
├── 1800
├── 2200
├── 2600
└── 3000

The natural/ directory contains the same set of distance folders.

Each distance directory represents a complete capture set for that environment-distance combination.

3. Dataset Structure (Uncompressed Form)

NOTE
The filename convention is: <camera_type><distance_in_mm>_<type_of_environments>.
Modalities are: hd, sd fot the realsense, and hd (or) sd followed by the neural mode for the ZED cameras.

After full restoration, each distance folder (e.g. artificial/0400/) contains:

A. Log Files

Exactly 16 .txt files, one per camera or configuration.

Examples:

rsf_0400_hd_artificial.txt
rsi_0400_sd_artificial.txt
zedx_0400_hd_neural_artificial.txt
...
zedxm_0400_sd_neural_plus_artificial.txt

These files contain metadata, calibration outputs, runtime logs, or capture notes depending on the pipeline.

When fully restored, all log files are located directly inside the distance folder.

B. Data Folders

Exactly 16 data folders, one per camera configuration.

Example:

rsf_0400_hd_artificial/
├── metadata.yaml
└── rsf_0400_hd_artificial_0.mcap

Each data folder contains:

  • ROS 2 bag metadata
  • MCAP recordings

This represents the fully expanded, analysis-ready form of the dataset.

4. Compressed Dataset Layout (Dataverse)

To reduce storage requirements, the dataset is distributed in compressed form.

Inside each distance directory, you will find:

A. Logs Archive

All .txt files are bundled into a single archive logs.tar.xz. Example: logs_0400_artificial.tar.xz

This archive contains a single folder with all log files.

B. ROS 2 Bag Archives

Each recorded bag is compressed and distributed as an archive .tar.xz.

These archives must be decompressed to restore the original folder structure. This can be automatically done using the downloading script at https://github.com/ADVRHumanoids/CameraComparison_code/dataset_downloader.sh.

5. Downloading the Dataset

The dataset is hosted on IIT Dataverse and must be downloaded via its API.

Prerequisites

  • Bash shell
  • curl
  • jq
  • tar
  • unzip

Environment Variables

Before running the downloader, set the following variables:

export API_TOKEN=your_api_token_here
export SERVER_URL=https://dataverse.iit.it/
export PERSISTENT_ID=doi:10.48557/LUONQB

Download the Downloader Script

  1. Download the dataset downloader
git clone https://github.com/ADVRHumanoids/CameraComparison_code.git
cd CameraComparison_code
chmod +x dataverse_downloader.sh
./dataverse_downloader.sh

6. Using the downloader

There are several option to use the downloader:

  1. Full Dataset Download
./dataverse_downloader.sh
  1. Log File download
./dataverse_downloader.sh --log-only

This will:

  1. Download and Decompress all archives
./dataverse_downloader.sh --decompress

This will:

  • Extract all data archives in place
  • Extract log archives
  • Move all .txt log files into their parent distance folder
  1. Download, Decompress, and Remove Archives
./dataverse_downloader.sh --decompress --clean-archives

This will:

  • Fully restore the dataset structure
  • Remove all .zip and .tar.xz files after successful extraction

⚠️ Use this option only if you are sure you no longer need the compressed files.

7. Requirements

  • Bash terminal (linux)
  • curl
  • jq
  • tar
  • unzip
  • Sufficient disk space (see size warning above)

Disclaimer

Please not that the log file provided with the Dataset contains only the log of the data collected during the experiments.
Other data presented in the paper have been analyzed using the ROS 2 bags provided with the dataset.

NOTE
The ROS 2 bag for the test ZED X HD Neural Natural at 3000 mm has been corrupted during the analysis and thus it is usable only partially.