Repository navigation
Make the remaining pre-commit hooks pass, except markdownlint - #363
Merged
Merged
Conversation
The check-shebang-scripts-are-executable hook reported nine files
that start with `#!` but were committed without the executable bit:
- reference-apps/{fastapi,fastapi-api-first,golang}/{init,start}.sh:
container entrypoints. Each Dockerfile copies them and runs
`chmod +x` before using them as CMD, so the image is unchanged;
the repository now matches.
- tests/test-vault-function.sh: a standalone diagnostic that runs
Vault credential lookups when executed.
- scripts/add-profile-labels.py: a command-line script with a
python3 shebang and a main entry point.
- scripts/lib/service-init-common.sh: a library the init-approle.sh
scripts source. Its shebang stays because the "Check for bash
shebangs" step in .github/workflows/test.yml requires one on every
.sh file, and it marks the dialect for editors and ShellCheck.
Executing it directly only defines variables and functions and
exits 0.
.github/CONTRIBUTING.md asked for 4-space indentation in shell scripts, but the shfmt hook in .pre-commit-config.yaml runs `shfmt -i 2 -ci -sr` and every script is formatted with it. The guide now states 2 spaces and names the hook as the source of truth, with the command that applies it. wiki/Contributing-Guide.md, the wiki copy that scripts/docs-to-wiki.sh makes of this file, gets the same change.
wiki/TLS-Configuration.md showed the output of `vault write pki_int/issue/...` as comment lines, one of them `# private_key: -----BEGIN RSA PRIVATE KEY-----...`. No key follows the header, but the detect-private-key hook matches the header text and fails. That line now reads `# private_key: <PEM-encoded RSA private key>`.
Output of the trailing-whitespace and end-of-file-fixer hooks, run with the repository's configuration: - trailing-whitespace removed trailing spaces in docs/OBSERVABILITY.md and in three wiki pages (API-Development-Guide, CI-CD-Integration, Health-Monitoring). Markdown line breaks (two trailing spaces) are kept by the hook's --markdown-linebreak-ext=md setting. - end-of-file-fixer gave 22 files exactly one final newline: 15 Markdown pages under docs/ and wiki/, six Grafana dashboards under configs/grafana/dashboards/ and reference-apps/shared/openapi.json. `git diff -w --ignore-blank-lines` is empty for this commit, and scripts/wiki-mirror.sh --check still reports the mirror as current.
Output of the ruff-format hook (ruff v0.1.9, as pinned in
.pre-commit-config.yaml) over all Python files: 62 files reformatted,
17 already formatted. No other change.
Checks:
- The AST of every Python file is identical before and after, except
four docstrings in reference-apps/fastapi/app/routers/health.py
that lose a trailing space ("Check MySQL connectivity " and the
MongoDB, Redis cluster and RabbitMQ equivalents). These belong to
helper functions, not routes, so the OpenAPI output is unchanged.
- The ruff lint hook still passes.
- The FastAPI unit tests that CI runs (pytest tests/
--ignore=tests/test_integration.py -k "not integration") give
193 passed, 52 skipped, 44 deselected, and the API-first app's
tests give 12 passed, both before and after.
With only retain_line_breaks=true, yamlfmt v0.10.0 rewrote every `<<: *anchor` merge key in docker-compose.yml as `!!merge <<: *anchor` and joined the lines of folded (`>`) scalars, turning the multi-line vault entrypoint into one line. Both rewrites parse to the same data, but the explicit merge tag is unusual YAML that other readers of docker-compose.yml, such as Dependabot, are not known to accept. Two formatter options turn these rewrites off: - drop_merge_tag=true keeps merge keys as `<<:`. - scan_folded_as_literal=true keeps the line breaks inside folded scalars. The `>` indicator stays, so the value is unchanged. The hook still runs on every YAML file; nothing is excluded.
Output of the yamlfmt hook over all YAML files, plus one hand edit: - yamlfmt pads end-of-line comments with one space (`key: value # c` instead of aligned columns), drops the `---` document start in configs/prometheus/rules/devstack-alerts.yml and writes the x-init-library anchor in docker-compose.yml on one line. - configs/prometheus/prometheus.yml had two comment lines after the reference-api job's static_configs that yamlfmt re-indented under the targets list. They now sit above the job they describe. Checks: every changed file loads to the same data with PyYAML before and after; `docker compose config` prints the same output; yamllint with the configuration from .github/workflows/lint.yml reports no errors before or after (29 warnings before, 28 after).
.secrets.baseline had no results, so the detect-secrets hook (v1.4.0)
failed on every file it flagged. Each of the 108 findings in 70 files
was read; none is a credential:
- 56 are code or tests: shell checks such as
`[ "$REDIS_PASSWORD" = "null" ]`, Go and Python code that formats a
connection string from variables or reads a `secret_path`, and mock
values in unit tests (test_pass, secret123, testpass, pw).
- 46 are documentation examples: placeholder connection strings and
passwords (password, PASSWORD, USER:PASS, CHANGE_ME, your_password,
SecurePass123!, the obviously made-up Hx7kL9mNpQr2sTuVwXyZ12345 in
docs/USE_CASES.md and docs/CLI_REFERENCE.md) and the Python doctest
examples in app/utils/logging.py.
- 4 are configuration keywords: ALLOW_WEAK_PASSWORDS=true in the three
profile files and GRAFANA_ADMIN_PASSWORD=admin in
configs/profiles/full.env, Grafana's documented default for the
local stack.
- 2 are the `run: |` blocks in .github/workflows/lint.yml and
security.yml that list weak-password patterns ("password=password")
to search .env.example for.
The baseline was regenerated with `detect-secrets scan --baseline
.secrets.baseline` (1.4.0) and audited with `detect-secrets audit`,
marking 106 findings as not secrets. Audit cannot display the two
workflow findings ("Secret not found on line 5"), so those two were
marked `"is_secret": false` by hand. A rescan reproduces the same
results.
NormB
force-pushed
the
fix/pre-commit-hooks
branch
from
October 15, 2026 14:59
5dc0c5e to
dc46857
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebased onto main after #362 and #364 merged (#364 changed the two
exception_handlers.pyfiles andtest_exception_handlers_unit.py; this branch keeps #364's code, already ruff-formatted). The per-hook table below was measured on b4d4eda before the rebase.This PR fixes the other failing pre-commit hooks, except markdownlint (see "Not done" below).
Per-hook results, main → this branch (with #362)
configs/redis/scripts/init.sh(&>under#!/bin/sh)wiki/TLS-Configuration.md)--fixshifts Markdown lines first--fixrewrites 159 files and leaves 3,298--fixCommits
Mark the nine scripts with a shebang as executable: the reference-appinit.sh/start.shentrypoints (the Dockerfiles alreadychmod +xthem),tests/test-vault-function.sh,scripts/add-profile-labels.py, andscripts/lib/service-init-common.sh. The last one is a sourced library. It keeps its shebang because the "Check for bash shebangs" step intest.ymlrequires one on every.shfile; executing it only defines functions and exits 0.State 2-space shell indentation in the contributing guide: names the shfmt hook as the source of truth. The same change goes into the wiki copy.Describe the issued private key without a PEM header in the wiki: an example comment inwiki/TLS-Configuration.mdcontained-----BEGIN RSA PRIVATE KEY-----...with no key after it.Strip trailing whitespace and fix file endings: hook output only.git diff -w --ignore-blank-linesis empty, and the wiki mirror check still passes.Format Python files with the repository's ruff-format hook: the AST is identical for every file, except 4 docstrings on helper (non-route) functions that lose a trailing space. The FastAPI unit tests that CI runs give 193 passed, 52 skipped, 44 deselected before and after, and the API-first app's tests give 12 passed (after the rebase onto Return no exception text or traceback from the FastAPI error handlers #364).Keep merge keys and folded scalars intact in the yamlfmt hook: changes the hook arguments. It addsdrop_merge_tag=trueandscan_folded_as_literal=true. Without them yamlfmt rewrites every<<:merge key in docker-compose.yml as!!merge <<:and joins the vault entrypoint onto one line. Both rewrites parse to the same data, but!!mergeis unusual YAML that other readers of docker-compose.yml, such as Dependabot, are not known to accept. Nothing is excluded. Drop this commit if you prefer the default output.Format YAML files with the repository's yamlfmt hook: every changed file loads to the same data with PyYAML, anddocker compose configoutput is identical. yamllint with the lint.yml configuration reports 0 errors before and after (29 warnings before, 28 after).Record the audited detect-secrets findings in the baseline: generated withdetect-secrets scan --baseline .secrets.baseline(1.4.0) and audited withdetect-secrets audit; all 108 are marked as not secrets. Audit cannot display the two workflow findings ("Secret not found on line 5"), so those two were marked"is_secret": falseby hand. No finding was a real secret.Not done: markdownlint
markdownlint (v0.38.0 with default rules,
--fix) reports 10,286 findings on main. Its--fixrewrites 159 Markdown files and still leaves 3,298: MD013 line-length 2,061, MD040 380, MD036 311, MD029 237, MD024 215, MD051 74, MD050 16, MD033 3, MD041 1. Some of the automatic fixes also change meaning: MD026 removes trailing punctuation from headings, which changes their anchors, and MD010 replaces the tabs in the Makefile example inreference-apps/API_PATTERNS.md, where make requires tabs. The markdown-lint job inlint.ymlalready says this needs a rule policy (a markdownlint config) and one pass over the corpus before it can be a gate. That is a decision for the maintainer, so this PR leaves the hook as it is.Until then, a full
pre-commit run --all-filesalso reports detect-secrets as failed: markdownlint--fixruns first and inserts lines in Markdown files, and detect-secrets then updates the recorded line numbers in.secrets.baseline.pre-commit run detect-secrets --all-fileson its own passes.Verification
tests/test-documentation-accuracy.sh,scripts/wiki-mirror.sh --check,scripts/validate-cicd.sh,./devstack --helpand every subcommand's--help,scripts/check-markdown-links.sh,scripts/audit-capabilities.sh,scripts/docs-to-wiki.sh --dry-run,bash -nover all scripts) give the same output on main and on this branch. The one exception is the CI permissions step, which no longer lists non-executable scripts.scripts/validate-cicd.shreports 1 failed test on both: it runs ShellCheck at default severity onconfigs/pgbouncer/scripts/init.sh, which has an SC2002 style finding (useless cat) at line 209.test-documentation-accuracy.sh12/12,wiki-mirror.sh --checkcurrent (16 files),./devstack --helpexit 0,bash -non 104 scripts with no failures,docker compose configidentical to main, and every changed YAML file loads identically to main except.pre-commit-config.yaml(the intended hook arguments).detect-secrets scan --baseline(1.4.0) reproduces the same 108 findings in 70 files at the same line numbers, so the baseline commit is unchanged.chmod +xon those files.Pre-commit summary
uvx pre-commit run --all-fileson main (b4d4eda, detached):On this branch:
uvx pre-commit run detect-secrets --all-fileson this branch:Detect secrets....Passed.detect-secrets findings (108)
None is a secret. The reason for each:
.github/workflows/lint.yml:307.github/workflows/security.yml:275configs/exporters/redis/init.sh:254configs/forgejo/scripts/forgejo-bootstrap.sh:135configs/forgejo/scripts/init-approle.sh:229configs/forgejo/scripts/init.sh:92configs/mysql/scripts/init-approle.sh:72configs/mysql/scripts/init.sh:248configs/pgbouncer/scripts/init.sh:149configs/profiles/full.env:104configs/profiles/full.env:383configs/profiles/minimal.env:256configs/profiles/standard.env:394configs/redis/scripts/init-approle.sh:67configs/redis/scripts/init.sh:244configs/vector/redis-metrics.sh:46docs/CLI_REFERENCE.md:504docs/OBSERVABILITY.md:380docs/QUICK_REFERENCE.md:166docs/SERVICES.md:662docs/SERVICES.md:932docs/SERVICES.md:1040docs/TLS_CERTIFICATE_MANAGEMENT.md:369docs/USE_CASES.md:73docs/USE_CASES.md:73docs/USE_CASES.md:86docs/USE_CASES.md:172docs/USE_CASES.md:254docs/USE_CASES.md:345docs/USE_CASES.md:355docs/USE_CASES.md:367docs/USE_CASES.md:692docs/voip/ANSIBLE_DYNAMIC_INVENTORY_POSTGRESQL.md:638docs/voip/VOIP_INFRASTRUCTURE_ANALYSIS.md:1137reference-apps/fastapi-api-first/app/services/vault.py:82reference-apps/fastapi-api-first/app/utils/logging.py:37reference-apps/fastapi-api-first/app/utils/logging.py:109reference-apps/fastapi/README.md:702reference-apps/fastapi/app/services/vault.py:82reference-apps/fastapi/app/utils/logging.py:37reference-apps/fastapi/app/utils/logging.py:109reference-apps/fastapi/tests/conftest.py:49reference-apps/fastapi/tests/test_cache_demo_unit.py:193reference-apps/fastapi/tests/test_caching.py:224reference-apps/fastapi/tests/test_caching.py:259reference-apps/fastapi/tests/test_caching.py:260reference-apps/fastapi/tests/test_database_demo.py:37reference-apps/fastapi/tests/test_database_demo.py:261reference-apps/fastapi/tests/test_exception_handlers.py:43reference-apps/fastapi/tests/test_exception_handlers.py:114reference-apps/fastapi/tests/test_exceptions.py:90reference-apps/fastapi/tests/test_exceptions.py:309reference-apps/fastapi/tests/test_health_routers.py:151reference-apps/fastapi/tests/test_health_routers.py:172reference-apps/fastapi/tests/test_redis_cluster.py:34reference-apps/fastapi/tests/test_request_validation.py:33reference-apps/fastapi/tests/test_routers_unit.py:85reference-apps/fastapi/tests/test_routers_unit.py:343reference-apps/fastapi/tests/test_routers_unit.py:347reference-apps/fastapi/tests/test_routers_unit.py:358reference-apps/fastapi/tests/test_vault_service.py:28reference-apps/fastapi/tests/test_vault_service.py:297reference-apps/golang/README.md:164reference-apps/golang/internal/handlers/cache.go:42reference-apps/golang/internal/handlers/database.go:53reference-apps/golang/internal/handlers/health.go:87reference-apps/golang/internal/handlers/health.go:230reference-apps/golang/internal/handlers/messaging.go:40reference-apps/golang/internal/handlers/redis_cluster.go:41reference-apps/golang/internal/handlers/vault.go:41reference-apps/golang/internal/services/vault.go:76reference-apps/golang/internal/services/vault.go:90reference-apps/nodejs/tests/health.test.js:116reference-apps/nodejs/tests/vault-service.test.js:111reference-apps/typescript-api-first/tests/health.test.ts:137scripts/manage_devstack.py:3009tests/test-approle.sh:297tests/test-negative.sh:167tests/test-pgbouncer.sh:60tests/test-postgres-extended.sh:64tests/test-rollback-complete.sh:310tests/test-rollback-complete.sh:328tests/test-rollback-simple.sh:135tests/test-rollback.sh:290tests/test-rollback.sh:296tests/test-vault-extended.sh:114wiki/API-Development-Guide.md:424wiki/Ansible-Dynamic-Inventory-PostgreSQL.md:638wiki/Backup-and-Restore.md:1137wiki/CI-CD-Integration.md:293wiki/CI-CD-Integration.md:531wiki/CI-CD-Integration.md:533wiki/Forgejo-Setup.md:33wiki/Health-Monitoring.md:378wiki/MongoDB-Operations.md:157wiki/MongoDB-Operations.md:379wiki/MongoDB-Operations.md:396wiki/MongoDB-Operations.md:407wiki/MongoDB-Operations.md:467wiki/MongoDB-Operations.md:477wiki/MongoDB-Operations.md:539wiki/MySQL-Operations.md:1815wiki/PgBouncer-Usage.md:483wiki/PgBouncer-Usage.md:498wiki/Port-Reference.md:62wiki/Service-Overview.md:661wiki/Service-Overview.md:931wiki/Service-Overview.md:1039