Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
b7cfc04
feat: foundation for the dbt Power User tools
hkaltai Nov 13, 2026
1eb722e
Merge remote-tracking branch 'origin/main' into feat/pu-mods-foundation
hkaltai Nov 13, 2026
d5e7bc8
feat: dbt_cost_estimate tool for BigQuery dry-run scan estimates
hkaltai Nov 13, 2026
d191355
test: dbt_cost_estimate is no longer a placeholder
hkaltai Nov 13, 2026
21d0494
fix: harden the dbt Power User tool foundation
hkaltai Nov 13, 2026
d90e334
fix: compile a model by name, give the scan an upper bound, stop wait…
hkaltai Nov 13, 2026
33e2655
Merge branch 'feat/pu-mods-foundation' into feat/dbt-cost-estimate-tool
hkaltai Nov 13, 2026
9f19a75
fix: start no compile for a cancelled call, and bound a rounded scan …
hkaltai Nov 13, 2026
1c23d35
fix: clean up the cancel listener when starting the work throws
hkaltai Nov 13, 2026
4fa4703
fix: a call cancelled before its queued work runs starts no compile
hkaltai Nov 14, 2026
9f4ae46
test: cover work queued before a cancellation with a signal that flip…
hkaltai Nov 14, 2026
99e953c
fix: reject a call with both model and sql, and word the read-only re…
hkaltai Nov 15, 2026
6c2ab6f
fix: name the MCP server in the read-only refusal
hkaltai Nov 15, 2026
d8c3469
fix: send falsy API bodies, follow cancellation in apiRequest, check …
hkaltai Nov 15, 2026
f2f4546
Merge remote-tracking branch 'origin/main' into feat/dbt-cost-estimat…
hkaltai Nov 17, 2026
dace1c2
Merge remote-tracking branch 'origin/main' into feat/pu-mods-foundation
hkaltai Nov 17, 2026
2d3c5fb
fix: the cost tool description names the MCP server as a way to grant…
hkaltai Nov 17, 2026
18a3fc4
fix: keep the stored API key out of API errors, bound the dbt config …
hkaltai Nov 17, 2026
6024ac9
fix: redact the API key when a refused header races the caller's abor…
hkaltai Nov 17, 2026
f5159c5
docs: say which tools use the dbt adapter helper
hkaltai Nov 17, 2026
f57de3d
docs: rewrap and correct the adapter helper comments
hkaltai Nov 17, 2026
8c8265d
Merge remote-tracking branch 'origin/feat/pu-mods-foundation' into fe…
hkaltai Nov 17, 2026
6514705
fix: escape the compiled SQL for the dbt integrations that write it i…
hkaltai Nov 17, 2026
d7938df
fix: a config that names no integration is the Python bridge
hkaltai Nov 17, 2026
aaff414
fix: never echo a send failure's text, so a trimmed key can't leak
hkaltai Nov 17, 2026
3a98dff
fix: a rejection with no reason is a send failure, not an abort
hkaltai Nov 17, 2026
34fa3dd
Merge remote-tracking branch 'origin/feat/pu-mods-foundation' into fe…
hkaltai Nov 17, 2026
29ee2cd
Merge remote-tracking branch 'origin/main' into feat/dbt-cost-estimat…
hkaltai Nov 17, 2026
ac43c53
docs: the cost estimate tool says compiling can run billed queries an…
hkaltai Nov 17, 2026
67f4dae
fix: a coded or pre-send TypeError is not a header refusal; bind the …
hkaltai Nov 17, 2026
5c61e7b
fix: the description test reads the tool the way the others do
hkaltai Nov 17, 2026
460cdc9
Merge remote-tracking branch 'origin/feat/pu-mods-foundation' into fe…
hkaltai Nov 17, 2026
3bad7cc
fix: the tool description says compiling either input can render Jinj…
hkaltai Nov 17, 2026
23dac88
Merge remote-tracking branch 'origin/main' into feat/dbt-cost-estimat…
hkaltai Nov 17, 2026
10a9f1f
fix: a zero dry-run byte count is a lower bound, not an upper bound
hkaltai Nov 18, 2026
b48df04
fix: a script's dry run is a lower bound, not an upper bound
hkaltai Nov 18, 2026
29b8f05
fix: script detection reads comments and quoted text in one pass
hkaltai Nov 18, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 144 additions & 5 deletions packages/opencode/src/altimate/tools/dbt-cost-estimate.ts
Original file line number Diff line number Diff line change
@@ -1,17 +1,156 @@
import z from "zod"
import { Tool } from "../../tool/tool"
import { dbtAdapterFor } from "./dbt-adapter-access"

/**
* Every dbt integration but the Python bridge writes the SQL into a `{{ validate_sql('...') }}` Jinja
* argument, so a quote or backslash in it must be escaped or any filter on a string breaks the dry run.
* The bridge, which a config without an integration also means, binds the SQL as a parameter and takes it as is.
*/
export function forDryRun(sql: string, integration: unknown): string {
return integration == null || integration === "core" ? sql : sql.replace(/\\/g, "\\\\").replace(/'/g, "\\'")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: the omitted-integration default assumed here is not the one the adapter applies

integration == null is treated as "the Python bridge" here, but the adapter never treats a missing integration as the bridge: packages/dbt-tools/src/adapter.ts:47 resolves cfg.dbtIntegration ?? DEFAULT_CONFIGURATION_VALUES.dbtIntegration, and this repo's own CLI default is "corecommand" (packages/dbt-tools/src/config.ts:105, packages/dbt-tools/src/commands/init.ts:19) - a value this function escapes. Nothing here ties forDryRun's fallback to the library constant the adapter uses, so a dbt.json that omits dbtIntegration gets no escaping while the adapter runs whatever the library default is; if that default is corecommand (as the two repo defaults suggest), every compiled query with a string literal breaks the Jinja argument. Resolve the default from the same source the adapter uses (or ask the adapter for its integration) instead of assuming the bridge. dbt-cost-estimate.test.ts:291 pins this assumption, not the library's default.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed: the premise does not hold. The adapter reads cfg.dbtIntegration ?? DEFAULT_CONFIGURATION_VALUES.dbtIntegration, and the library's default (@altimateai/dbt-integration 0.3.17) is "core", the Python bridge. corecommand is only what init writes into a new file. So a config naming no integration is the bridge, which is what d7938df pins and its test shows.

}

/** The units dbt-bigquery's `format_bytes` writes, smallest first; each is 1024 times the one before. */
const UNITS = ["Bytes", "KiB", "MiB", "GiB", "TiB", "PiB"] as const

/**
* The byte count in a dry run's `bytes_processed`, which dbt-bigquery formats as text ("4.2 TiB",
* "512.0 Bytes"), and the most it can be. The text carries one decimal, so a figure past a byte is
* rounded: the true count is within half a tenth of its unit. Undefined for anything else, so a changed
* format is reported, never read as a count.
*/
/** Whether the SQL is a script (more than one statement, or EXECUTE IMMEDIATE): a dry run only sees its first statements. */
Comment thread
hkaltai marked this conversation as resolved.
Resolved
Comment thread
hkaltai marked this conversation as resolved.
Resolved
export function isScript(sql: string): boolean {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: isScript sits between parseBytesProcessed's doc comment (the "rounded: the true count…" block above) and that function, so the comment now documents isScript. Moving isScript above that block would fix it.

// One pass over comments and quoted text together: a quote inside a comment, or a comment marker inside a quote, is not either.
const bare = sql
.replace(/--[^\n]*|#[^\n]*|\/\*[\s\S]*?\*\/|'(?:[^'\\]|\\.)*'|"(?:[^"\\]|\\.)*"|`[^`]*`/g, " ")
Comment thread
hkaltai marked this conversation as resolved.
Resolved
.replace(/;\s*$/, "")
return bare.includes(";") || /\bexecute\s+immediate\b/i.test(bare)
}

export function parseBytesProcessed(display: unknown): { bytes: number; maxBytes: number } | undefined {
if (typeof display === "number") {
if (!Number.isFinite(display) || display < 0) return undefined
const bytes = Math.round(display)
return { bytes, maxBytes: bytes }
}
if (typeof display !== "string") return undefined
const match = /^\s*(\d+(?:\.\d+)?)\s*(Bytes|KiB|MiB|GiB|TiB|PiB)\s*$/i.exec(display)
if (!match) return undefined
const unit = UNITS.findIndex((u) => u.toLowerCase() === match[2].toLowerCase())
const scale = 1024 ** unit
const bytes = Math.round(Number(match[1]) * scale)
if (unit === 0) return { bytes, maxBytes: bytes }
// Exact in integers: the shown figure is rounded to its last digit, so the true count is at most half
// of that digit above it. A float product is off by whole bytes past a few TiB.
const [whole, fraction = ""] = match[1].split(".")
const digits = BigInt(whole + fraction)
let upper = ((digits * 2n + 1n) * 1024n ** BigInt(unit) + 2n * 10n ** BigInt(fraction.length) - 1n) / (2n * 10n ** BigInt(fraction.length))
// Python divides in floats, so a count within a float's last place of a tie is shown rounded down.
upper += upper >> 50n
let maxBytes = Number(upper)
if (BigInt(Math.trunc(maxBytes)) < upper) maxBytes *= 1 + Number.EPSILON
return { bytes, maxBytes: Math.max(bytes, maxBytes) }
}

/** Starts `start` and waits for it, but gives up as soon as the call is cancelled; a cancelled call starts nothing. */
export function untilAbort<T>(start: () => Promise<T>, signal: AbortSignal): Promise<T> {
if (signal.aborted) return Promise.reject(signal.reason)
return new Promise<T>((resolve, reject) => {
const stop = () => reject(signal.reason)
signal.addEventListener("abort", stop, { once: true })
Promise.resolve()
.then(() => (signal.aborted ? Promise.reject(signal.reason) : start()))
.then(resolve, reject)
.finally(() => signal.removeEventListener("abort", stop))
})
}

function failure(title: string, message: string, extra: Record<string, unknown> = {}) {
return { title: `dbt_cost_estimate: ${title}`, metadata: { success: false, error: message, ...extra }, output: message }
}

export const DbtCostEstimateTool = Tool.define("dbt_cost_estimate", {
description: "Estimate the bytes a dbt model or a SQL query would scan, from a warehouse dry run (BigQuery).",
description:
"Estimate the bytes a dbt model or a SQL query would scan, from a warehouse dry run. Only BigQuery has a dry run: any other warehouse answers `supported: false` and nothing is compiled. Starts the dbt adapter (which runs `dbt deps`), so it needs write permission (`--allow-write` on `tool run` or the MCP server). The dry run never executes the final query, but compiling either input can render Jinja, and macros such as `run_query` can run real queries that are billed. Pass `model` (a model name, compiled as dbt would) or `sql` (may use Jinja).",
parameters: z.object({
model: z.string().optional().describe("Model name to compile and estimate"),
sql: z.string().optional().describe("SQL (may use Jinja) to compile and estimate"),
}),
async execute() {
async execute(args, ctx) {
if (!args.model && !args.sql) return failure("nothing to estimate", "Pass a model name or SQL.")
Comment thread
hkaltai marked this conversation as resolved.
Resolved
if (args.model && args.sql) return failure("ambiguous input", "Pass a model name or SQL, not both.")
const access = await dbtAdapterFor(ctx)
if (!access.adapter) return failure("not available", `No estimate: ${access.reason}.`, { refusal: access.refusal })
const adapter = access.adapter
const type: string | undefined = adapter.getAdapterType?.()
// Every dbt-integration refuses the dry run off BigQuery, so nothing is compiled there.
if (type !== "bigquery") {
const adapterName = type ?? "unknown"
return {
title: "dbt_cost_estimate: no estimate",
metadata: { success: true, supported: false, adapter: adapterName },
output: `No cost estimate: dry runs are only available on BigQuery (this project uses ${adapterName}).`,
}
}
const signal = ctx.abort
let compiled: unknown
try {
// A model compiles by name, so its node context (`this`, config) is kept; SQL compiles inline.
compiled = await untilAbort(
() => (args.model ? adapter.unsafeCompileNode(args.model) : adapter.unsafeCompileQuery(args.sql!, undefined)),
signal,
)
} catch (e) {
if (signal.aborted) throw e
return failure("compile failed", e instanceof Error ? e.message : String(e), { adapter: type })
}
if (typeof compiled !== "string" || !compiled.trim()) {
return failure("compile failed", "The query compiled to nothing.", { adapter: type })
}
let display: unknown
try {
const integration = await untilAbort(async () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: this reads the dbt config again instead of taking the integration from the adapter dbtAdapterFor has cached. If dbtIntegration changes mid-process, the escaping follows the new value while the adapter still runs the old integration. Rare; passing the integration through the adapter access would close it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed here. Taking the integration from the adapter would mean exposing the config from register.ts, which is the foundation's file shared by the sibling tools; the gap needs dbt.json edited mid-process after the adapter started. Recorded as a follow-up on the foundation.

const { dbtAdapterInternals } = await import("../native/connections/register")
const read = dbtAdapterInternals.readConfig ?? (await import("../../../../dbt-tools/src/config")).read
return ((await read()) as { dbtIntegration?: unknown } | null)?.dbtIntegration

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: this reads the dbt config again instead of the config the running adapter was created from

register.ts:305-307 deliberately reuses the root the settled adapter attempt read ("rather than reading the config again"), but this block re-reads ~/.altimate-code/dbt.json on every estimate, and with no (or malformed) file read() runs discoverPython()'s which/where again - up to a 5 s timeout per candidate (config.ts:82). The adapter is cached for the process while dbt.json is mutable, so an edit after the adapter started makes forDryRun's escaping decision disagree with the adapter actually in use. Consider exposing the config (or just its dbtIntegration) from native/connections/register.ts next to dbtProjectRoot, so the tool uses the adapter's own value and the readConfig ?? read seam stays in one place instead of being copied here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed here. Taking the integration from the adapter would mean exposing the config from register.ts, which is the foundation's file shared by the sibling tools; the gap needs dbt.json edited mid-process after the adapter started. Recorded as a follow-up on the foundation.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: dbtAdapterFor can return a cached adapter, but this reads the current config again; after an integration change, SQL is escaped for a different mode than the adapter uses. Keep the integration value with the adapter snapshot and use that value here.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/tools/dbt-cost-estimate.ts, line 108:

<comment>`dbtAdapterFor` can return a cached adapter, but this reads the current config again; after an integration change, SQL is escaped for a different mode than the adapter uses. Keep the integration value with the adapter snapshot and use that value here.</comment>

<file context>
@@ -93,7 +102,12 @@ export const DbtCostEstimateTool = Tool.define("dbt_cost_estimate", {
+      const integration = await untilAbort(async () => {
+        const { dbtAdapterInternals } = await import("../native/connections/register")
+        const read = dbtAdapterInternals.readConfig ?? (await import("../../../../dbt-tools/src/config")).read
+        return ((await read()) as { dbtIntegration?: unknown } | null)?.dbtIntegration
+      }, signal)
+      const result = await untilAbort(() => adapter.validateSQLDryRun(forDryRun(compiled as string, integration)), signal)
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed here, for the same reason as the thread above: it needs the foundation's adapter access to carry the integration, and the case needs dbt.json edited while the adapter is already running. Recorded as a follow-up.

}, signal)
const result = await untilAbort(() => adapter.validateSQLDryRun(forDryRun(compiled as string, integration)), signal)
if (result == null) {
return failure("dry run failed", "The dry run returned no estimate: dbt-core before 1.6 can't dry run.", { adapter: type })
}
display = (result as { bytes_processed?: unknown }).bytes_processed
} catch (e) {
if (signal.aborted) throw e
return failure("dry run failed", e instanceof Error ? e.message : String(e), { adapter: type })
}
const parsed = parseBytesProcessed(display)
if (!parsed) {
return failure("unreadable estimate", `The dry run returned an amount this tool can't read: ${JSON.stringify(display)}.`, {
adapter: type,
})
}
const name = args.model
const shown = typeof display === "number" ? `${parsed.bytes} Bytes` : String(display)
// A dry run can report 0 for a query that still reads data (BigQuery federated sources), and it skips
// what a script runs after its first statement or inside EXECUTE IMMEDIATE: those figures are lower
// bounds and carry no upper bound.
const zero = parsed.bytes === 0
const lower = zero || isScript(compiled as string)
return {
title: "dbt_cost_estimate: not implemented",
metadata: { success: false, error: "not implemented" },
output: "dbt_cost_estimate is not implemented in this build of altimate-code yet.",
title: `dbt_cost_estimate: ${shown}`,
metadata: {
success: true,
supported: true,
adapter: type,
...(lower ? { bytes: parsed.bytes, lowerBound: true } : parsed),
display: shown,
...(name && { model: name }),
},
output: `${name ? `${name} ` : "The query "}would process ${shown} ${
lower ? "(a dry run can undercount a script or a zero estimate, so this is a lower bound, not a cap)." : `(at most ${parsed.maxBytes} bytes).`
}`,
}
},
})
Loading
Loading