Skip to content

Docs: update "Customizing library models for <lang>" to cover JSON data extensions - #22753

Merged
nickrolfe merged 9 commits into
mainfrom
nickrolfe/docs-json-data-extensions
Oct 8, 2026
Merged

nickrolfe merged 9 commits into
mainfrom
nickrolfe/docs-json-data-extensions

Conversation

@nickrolfe

@nickrolfe nickrolfe commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

The next CLI release will support JSON data extensions in addition to the existing YAML format. This PR updates the documentation to cover the JSON format and to convert all the example extensions to JSON. (JSON performs noticeably better for packs with lots of extensions).

I'll open separate PRs to convert existing models in this repository once the new CLI version is released.

Comment thread docs/codeql/codeql-language-guides/customizing-library-models-for-go.rst Outdated
Comment thread docs/codeql/codeql-language-guides/customizing-library-models-for-go.rst Outdated
@nickrolfe
nickrolfe force-pushed the nickrolfe/docs-json-data-extensions branch 2 times, most recently from be95f83 to f41d6b6 Compare October 5, 2026 14:13
@nickrolfe
nickrolfe requested a balanced review from Copilot October 5, 2026 16:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Two Go examples incorrectly target the nonexistent codeql/go pack instead of codeql/go-all.

Review effort: Balanced
Findings: 1 High severity

Open (1)
What changed in this PR

Updates library-modeling guides to document JSON data extensions while retaining YAML syntax references.

Changes:

  • Adds JSON format guidance and converts examples to JSON.
  • Updates JSON booleans and comment syntax.
  • Updates the GitHub Actions model-pack example to load JSON files.
File Description
customizing-library-models-for-rust.rst Adds Rust JSON examples.
customizing-library-models-for-ruby.rst Adds Ruby JSON examples.
customizing-library-models-for-python.rst Adds Python JSON examples.
customizing-library-models-for-javascript.rst Adds JavaScript JSON examples.
customizing-library-models-for-java-and-kotlin.rst Adds Java/Kotlin JSON examples.
customizing-library-models-for-go.rst Adds Go JSON examples.
customizing-library-models-for-csharp.rst Adds C# JSON examples.
customizing-library-models-for-cpp.rst Adds C/C++ JSON examples.
customizing-library-models-for-actions.rst Adds GitHub Actions JSON examples and configuration.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread docs/codeql/codeql-language-guides/customizing-library-models-for-go.rst Outdated
@nickrolfe
nickrolfe force-pushed the nickrolfe/docs-json-data-extensions branch from f6d9c1e to 79cefe4 Compare October 5, 2026 16:07
@nickrolfe nickrolfe changed the title Docs: update "Customizing library models for $LANG" to cover JSON data extensions Docs: update "Customizing library models for <lang>" to cover JSON data extensions Oct 5, 2026
@nickrolfe
nickrolfe marked this pull request as ready for review October 8, 2026 11:21
@nickrolfe
nickrolfe requested a review from a team as a code owner October 8, 2026 11:21
@nickrolfe
nickrolfe force-pushed the nickrolfe/docs-json-data-extensions branch from 79cefe4 to a6d96fb Compare October 8, 2026 12:27
@nickrolfe

nickrolfe commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor Author

Updated with the changes discussed offline.

@michaelnebel

Copy link
Copy Markdown
Contributor

Very nice work @nickrolfe ! Should some from the docs team review as well?
Just out of curiosity: For QL tests, it is possible to provide an .ext.yml next to a .ql test file, where the data extensions only are loaded for that particular test. Will the same feature be available for .ext.json?

@owen-mc owen-mc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't do the last 3 because I lost the will to live.

@nickrolfe

nickrolfe commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor Author

Should some from the docs team review as well?

#19190, which originally added the Actions version of this document, was reviewed only by engineers, so I'd say there's no need.

Just out of curiosity: For QL tests, it is possible to provide an .ext.yml next to a .ql test file, where the data extensions only are loaded for that particular test. Will the same feature be available for .ext.json?

Yes. I discovered the existence of that feature at the last minute, but it's been updated so you can have either <test>.ext.yml, or <test>.ext.json, or both. That will be mentioned in the CLI changelog, but I didn't find anywhere else to document it.

BTW, when I convert all the existing models in the library and query packs, I plan not to touch those .ext.yml models in tests.

@nickrolfe
nickrolfe merged commit 9e6d428 into main Oct 8, 2026
7 checks passed
@nickrolfe
nickrolfe deleted the nickrolfe/docs-json-data-extensions branch October 8, 2026 13:48
@jacknojo

jacknojo commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

I didn't do the last 3 because I lost the will to live.

I was going to suggest the include functionality of rst until I noticed that the JSON differ slightly

@owen-mc

owen-mc commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

BTW, when I convert all the existing models in the library and query packs, I plan not to touch those .ext.yml models in tests.

Why not? I think it would be confusing to use one format consistently.

I believe that this feature is not documented, and it probably should be. Bonus points if you can find somewhere to document it (though hopefully not once for each language).

@nickrolfe

Copy link
Copy Markdown
Contributor Author

Why not?

No strong reasons. I was just thinking that it wouldn't make any difference to test performance, but I hadn't considered the consistency angle. It should be easy enough to find and convert them all, so I'll be happy to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants