fix(project): Bound Maven snapshot downloads and extraction

internal/documentation/docs/pages/SAPUI5.md

@@ -79,6 +79,74 @@ Please make sure that your project defines [Specification Version 2.0](./Configu
 
 For details please refer to the [framework configuration documentation](././Configuration.md#framework-configuratio).
 
+## Consuming SAPUI5 Snapshots
+
+Within SAP, in-development SAPUI5 versions can be consumed from an internal Maven registry. These snapshot versions are builds of a not-yet-released patch level that SAP does not publish to the npm registry, so UI5 CLI downloads them from the Maven registry instead. OpenUI5 has no snapshots.
+
+A framework version that ends with `-SNAPSHOT` selects the Maven path. Everything else about the project stays the same: framework libraries remain under `framework.libraries` in `ui5.yaml`, and `ui5 serve` or `ui5 build` download the required artifacts on first use.
+
+### Selecting a Snapshot Version
+
+Set the version with [`ui5 use`](./CLI.md#ui5-use), or maintain it directly in `ui5.yaml`. The version accepts a concrete snapshot, a range, or a simplified short form:
+
+```sh
+ui5 use latest-snapshot          # Latest available snapshot
+ui5 use 1.120-snapshot           # Latest snapshot of the 1.120.x line
+ui5 use sapui5@1.120.0-SNAPSHOT  # One concrete snapshot version
+```
+
+The short forms `1-SNAPSHOT`, `1.x-SNAPSHOT`, and `1.108-SNAPSHOT` resolve to the latest matching snapshot. The resulting `ui5.yaml` holds the resolved version, for example:
+
+```yaml
+framework:
+  name: SAPUI5
+  version: 1.120.0-SNAPSHOT
+  libraries:
+    - name: sap.ui.core
+    - name: sap.m
+```
+
+### Configuring the Maven Repository URL
+
+Before the first snapshot download, UI5 CLI needs the Maven repository URL. It resolves the URL from the first source that provides one, in this order:
+
+1. The `UI5_MAVEN_SNAPSHOT_ENDPOINT_URL` environment variable. This overrides any stored configuration and suits CI or one-off runs.
+2. The `mavenSnapshotEndpointUrl` UI5 CLI configuration. Set it with `ui5 config set mavenSnapshotEndpointUrl <url>`, read it with `ui5 config get mavenSnapshotEndpointUrl`, and clear it by omitting the value. See [`ui5 config`](./CLI.md#ui5-config).
+3. The `snapshot.build` repository in the Maven `~/.m2/settings.xml` file, when present. Reading `settings.xml` needs an interactive terminal, so UI5 CLI skips this source in non-interactive environments such as CI. When it does resolve a URL this way, it offers to store that URL in its configuration so later runs skip the prompt.
+
+If none of these provide a URL, snapshot resolution fails and the released-version path is unaffected.
+
+### Caching and Cache Modes
+
+Downloaded artifacts are cached in the `.ui5` directory in the user's home directory, next to the npm-based framework packages. Override the location with the `ui5DataDir` configuration or the `UI5_DATA_DIR` environment variable.
+
+Because a snapshot version points at a moving build, UI5 CLI also caches the repository metadata that maps a version to a concrete build. By default this metadata is reused for 9 hours before UI5 CLI checks the repository again. Control this with the `--snapshot-cache` option on `ui5 serve` and `ui5 build`:
+
+- `Default`: Reuse cached metadata, revalidate against the repository after 9 hours.
+- `Force`: Use the cache only, send no requests to the repository. Useful offline.
+- `Off`: Ignore the cached metadata and always update from the repository.
+
+### Resource Limits
+
+The snapshot resolver limits metadata downloads, artifact downloads, and archive extraction. Set the environment variables below to raise a limit for a large or slow trusted registry. Each value must contain decimal digits and represent a positive safe integer. The resolver reads overrides for each operation. Invalid overrides fail with `ERR_MAVEN_INVALID_LIMIT`.
+
+| Environment Variable | Unit and Scope | Default | Error Code |
+| :--- | :--- | :--- | :--- |
+| `UI5_MAVEN_MAX_METADATA_BYTES` | Bytes buffered per `maven-metadata.xml` response | `67108864` (64 MiB) | `ERR_MAVEN_METADATA_TOO_LARGE` |
+| `UI5_MAVEN_MAX_ARTIFACT_BYTES` | Bytes written per artifact response | `2147483648` (2 GiB) | `ERR_MAVEN_ARTIFACT_TOO_LARGE` |
+| `UI5_MAVEN_MAX_ARCHIVE_ENTRIES` | Entries per archive, including directories | `262144` | `ERR_MAVEN_ARCHIVE_TOO_MANY_ENTRIES` |
+| `UI5_MAVEN_MAX_ENTRY_BYTES` | Expanded bytes per archive entry | `1073741824` (1 GiB) | `ERR_MAVEN_ENTRY_TOO_LARGE` |
+| `UI5_MAVEN_MAX_EXPANDED_BYTES` | Total expanded bytes per archive | `4294967296` (4 GiB) | `ERR_MAVEN_ARCHIVE_TOO_LARGE` |
+| `UI5_MAVEN_MAX_COMPRESSION_RATIO` | Expanded bytes divided by archive bytes on disk | `200` | `ERR_MAVEN_COMPRESSION_RATIO_EXCEEDED` |
+| `UI5_MAVEN_FETCH_TIMEOUT` | Milliseconds per complete request | `1800000` (30 minutes) | `ERR_MAVEN_FETCH_TIMEOUT` |
+| `UI5_MAVEN_MAX_EXTRACTION_TIME` | Milliseconds per archive extraction | `600000` (10 minutes) | `ERR_MAVEN_EXTRACTION_TIMEOUT` |
+
+Download limits count response bytes as they arrive, including HTTP decompression. They apply even when `Content-Length` is missing or describes a smaller encoded body. Archive checks cover declared sizes before extraction and actual bytes during extraction. The declared-size checks include entries outside the `META-INF/` subtree selected for JAR extraction. Byte, entry-count, and compression-ratio limits accept values equal to the configured limit.
+
+`UI5_MAVEN_FETCH_TIMEOUT` bounds elapsed time from request start through body delivery. Redirects and any fetch retries share the same deadline. A server that keeps sending small chunks still reaches this deadline. This Maven setting is independent of npm's `fetch-timeout` configuration and does not change npm downloads.
+
+Failed downloads remove partial artifact files. Failed package installs remove staging output and the downloaded archive, then release the installation lock. A later install can download the artifact again. Limit errors name the metadata URL or artifact coordinates, the setting, and the observed value without including response bodies or file contents.
+
 ## Differences Between OpenUI5 and SAPUI5
 
 Please refer to our documentation on the [differences between OpenUI5 and SAPUI5](./FAQ.md#whats-the-difference-between-openui5-and-sapui5)

packages/project/lib/ui5Framework/maven/Installer.js

@@ -1,4 +1,5 @@
 import path from "node:path";
+import {pipeline} from "node:stream/promises";
 import {mkdirp} from "../../utils/fs.js";
 import parseJSON from "../../utils/json.js";
 import fs from "graceful-fs";
@@ -8,8 +9,13 @@ import {promisify} from "node:util";
 import Registry from "./Registry.js";
 import AbstractInstaller from "../AbstractInstaller.js";
 import SnapshotCache from "./SnapshotCache.js";
+import {
+	getMavenLimits, assertMaxBytes, assertEntryCount, assertCompressionRatio, createByteCounter, createDeadline
+} from "./limits.js";
 import {rmrf} from "../../utils/fs.js";
 const stat = promisify(fs.stat);
+const open = promisify(fs.open);
+const close = promisify(fs.close);
 const readFile = promisify(fs.readFile);
 const writeFile = promisify(fs.writeFile);
 const rename = promisify(fs.rename);
@@ -331,32 +337,31 @@ class Installer extends AbstractInstaller {
 					await rmrf(stagingDir);
 				}
 
-				await mkdirp(stagingDir);
-
-				const {artifactPath, removeArtifact} = await this.installArtifact(coordinates);
-
-				log.verbose(`Extracting archive at ${artifactPath} to ${stagingDir}...`);
-				const zip = new StreamZip({file: artifactPath});
-				let rootDir = null;
-				if (extension === "jar") {
-					rootDir = "META-INF";
-				}
-				await zip.extract(rootDir, stagingDir);
-				await zip.close();
-
-				// Check whether target dir already exists and remove it
-				if (await this._pathExists(targetDir)) {
-					log.verbose(`Removing existing target directory at ${targetDir}...`);
-					await rmrf(targetDir);
+				let removeArtifact;
+				try {
+					await mkdirp(stagingDir);
+					const artifact = await this.installArtifact(coordinates);
+					removeArtifact = artifact.removeArtifact;
+					await this._extractArchive(artifact.artifactPath, stagingDir, coordinates);
+					await removeArtifact();
+					removeArtifact = null;
+
+					if (await this._pathExists(targetDir)) {
+						log.verbose(`Removing existing target directory at ${targetDir}...`);
+						await rmrf(targetDir);
+					}
+
+					// Create only the parent to prevent EPERM in rename (https://github.com/UI5/cli/issues/487).
+					await mkdirp(path.dirname(targetDir));
+					log.verbose(`Promoting staging directory from ${stagingDir} to ${targetDir}...`);
+					await rename(stagingDir, targetDir);
+				} catch (err) {
+					await rmrf(stagingDir);
+					if (removeArtifact) {
+						await removeArtifact();
+					}
+					throw err;
 				}
-
-				// Do not create target dir itself to prevent EPERM error in following rename operation
-				// (https://github.com/UI5/cli/issues/487)
-				await mkdirp(path.dirname(targetDir));
-				log.verbose(`Promoting staging directory from ${stagingDir} to ${targetDir}...`);
-				await rename(stagingDir, targetDir);
-
-				await removeArtifact();
 			});
 		} else {
 			log.verbose(`Already installed: ${pkgName} in SNAPSHOT version ${revision}`);
@@ -418,20 +423,22 @@ class Installer extends AbstractInstaller {
 					log.verbose(`Removing existing file in staging dir at ${stagingPath}...`);
 					await rm(stagingPath);
 				}
-				await mkdirp(path.dirname(stagingPath));
+				try {
+					await mkdirp(path.dirname(stagingPath));
 
-				log.verbose(`Installing ${artifactId} in version ${version} to ${stagingPath}...`);
+					log.verbose(`Installing ${artifactId} in version ${version} to ${stagingPath}...`);
 
-				// TODO: Stream response body to installPackage and unzip directly via
-				// https://github.com/isaacs/minizlib (already in dependencies through pacote)
-				// This way we do not store the archive unnecessarily
-				const reg = await this.getRegistry();
-				await reg.requestArtifact(coordinates, stagingPath);
+					const reg = await this.getRegistry();
+					await reg.requestArtifact(coordinates, stagingPath);
 
-				await mkdirp(path.dirname(targetPath));
-				log.verbose(
-					`Promoting artifact from staging path ${stagingPath} to target path at ${targetPath}...`);
-				await rename(stagingPath, targetPath);
+					await mkdirp(path.dirname(targetPath));
+					log.verbose(
+						`Promoting artifact from staging path ${stagingPath} to target path at ${targetPath}...`);
+					await rename(stagingPath, targetPath);
+				} catch (err) {
+					await rm(stagingPath, {force: true});
+					throw err;
+				}
 			});
 		} else {
 			log.verbose(`Already installed: ${artifactId} in version ${revision}`);
@@ -444,6 +451,87 @@ class Installer extends AbstractInstaller {
 		};
 	}
 
+	async _extractArchive(artifactPath, stagingDir, coordinates) {
+		const limits = getMavenLimits();
+		const input = this._generateLogIdFromCoordinates(coordinates);
+		const deadline = createDeadline({input, limit: limits.extractionTime,
+			name: "UI5_MAVEN_MAX_EXTRACTION_TIME", code: "ERR_MAVEN_EXTRACTION_TIMEOUT"});
+		const entryLimit = {input, limit: limits.entryBytes,
+			name: "UI5_MAVEN_MAX_ENTRY_BYTES", code: "ERR_MAVEN_ENTRY_TOO_LARGE"};
+		const expandedLimit = {input, limit: limits.expandedBytes,
+			name: "UI5_MAVEN_MAX_EXPANDED_BYTES", code: "ERR_MAVEN_ARCHIVE_TOO_LARGE"};
+		let fd;
+		let zip;
+		let ready = false;
+		try {
+			log.verbose(`Extracting archive at ${artifactPath} to ${stagingDir}...`);
+			// Own the descriptor so a failed directory read can be closed before the async ZIP API is ready.
+			fd = await open(artifactPath, "r");
+			deadline.check();
+			zip = new StreamZip({fd});
+			let entryCount = 0;
+			zip.on("entry", () => {
+				deadline.check();
+				// Stop directory parsing before it retains an unbounded entry map. Count duplicate names too.
+				assertEntryCount(++entryCount, {input, limit: limits.archiveEntries});
+			});
+			const count = await deadline.wait(zip.entriesCount);
+			ready = true;
+			assertEntryCount(count, {input, limit: limits.archiveEntries});
+			const entries = Object.values(await zip.entries());
+			const {size: archiveBytes} = await stat(artifactPath);
+			let declaredBytes = 0;
+			for (const entry of entries) {
+				deadline.check();
+				assertMaxBytes(entry.size, entryLimit);
+				declaredBytes += entry.size;
+				assertMaxBytes(declaredBytes, expandedLimit);
+			}
+			const ratioLimit = {input, limit: limits.compressionRatio};
+			assertCompressionRatio(declaredBytes, archiveBytes, ratioLimit);
+
+			const root = coordinates.extension === "jar" ? "META-INF/" : "";
+			const stagingRoot = path.resolve(stagingDir);
+			let expandedBytes = 0;
+			for (const entry of entries) {
+				deadline.check();
+				if (!entry.name.startsWith(root)) {
+					continue;
+				}
+				const destination = path.resolve(stagingRoot, entry.name.slice(root.length));
+				if (destination !== stagingRoot && !destination.startsWith(stagingRoot + path.sep)) {
+					throw new Error(`Invalid archive entry path for ${input}`);
+				}
+				if (entry.isDirectory) {
+					await mkdirp(destination);
+					continue;
+				}
+				await mkdirp(path.dirname(destination));
+				const stream = await zip.stream(entry);
+				const counter = createByteCounter((bytes, chunkBytes) => {
+					deadline.check();
+					assertMaxBytes(bytes, entryLimit);
+					expandedBytes += chunkBytes;
+					assertMaxBytes(expandedBytes, expandedLimit);
+					assertCompressionRatio(expandedBytes, archiveBytes, ratioLimit);
+				});
+				await pipeline(stream, counter, fs.createWriteStream(destination), {signal: deadline.signal});
+			}
+			deadline.check();
+		} catch (cause) {
+			const err = deadline.signal.reason || cause;
+			deadline.abort(err);
+			throw err;
+		} finally {
+			deadline.clear();
+			if (ready) {
+				await zip.close();
+			} else if (fd !== undefined) {
+				await close(fd);
+			}
+		}
+	}
+
 	async _projectExists(targetDir) {
 		return this._pathExists(path.join(targetDir, "package.json"));
 	}