Skip to content

development.md

Latest commit

 

History

History
279 lines (207 loc) · 7.89 KB

development.md

File metadata and controls

279 lines (207 loc) · 7.89 KB
Raw

Development guide

Guide for setting up a local development environment.

Prerequisites

  • Python 3.14+
  • Node.js 24 and yarn — app/.nvmrc names the version to use (nvm use in app/), the same one CI runs. app/package.json enforces only the build path's floor (22.12); the dev-only test stack reaches higher (jsdom installs on ^22.22.2 || ^24.15.0 || >=26.0.0, so Node 23 and 24.0 to 24.14 are out), which the version .nvmrc names satisfies. The app image's builder stays on the 22 line.
  • PostgreSQL (or access to Cloud SQL)
  • uv - Fast Python package manager
# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

Backend setup

# Clone and install
git clone https://github.com/MarkusNeusinger/anyplot.git
cd anyplot
uv sync --all-extras

# Database configuration
cp .env.example .env
# Edit .env with your DATABASE_URL:
# DATABASE_URL=postgresql+asyncpg://user:pass@host:5432/anyplot

# Run migrations
uv run alembic upgrade head

# Start API server
uv run uvicorn api.main:app --reload
# → http://localhost:8000/docs

Frontend setup

cd app
yarn install
yarn dev
# → http://localhost:3000

For production build:

yarn build

Running tests

# All tests
uv run pytest

# Only unit tests (fast, no DB needed)
uv run pytest tests/unit

# Only integration tests (SQLite in-memory)
uv run pytest tests/integration

# Only E2E tests (requires DATABASE_URL)
uv run pytest tests/e2e

# With coverage report (sources configured in pyproject.toml)
uv run pytest tests/unit tests/integration --cov --cov-report=html

# Unit and integration suites on one worker per CPU (pytest-xdist)
uv run pytest tests/unit tests/integration -n auto

The CI job "Run Tests" (.github/workflows/ci-tests.yml) runs the unit and integration tests on the four CPUs of the hosted runner:

uv run pytest tests/unit tests/integration -n 4 --dist worksteal --durations=25 \
  --cov=core --cov=api --cov=automation --cov=agentic/workflows/modules --cov=agents

That job installs the eight lib-* extras the renderer image installs and sets ANYPLOT_REQUIRE_LIBRARIES=1. A test that needs a plotting library imports it through tests/libraries.py: on your machine it skips when the library is missing, and in CI it fails. To run those tests locally, install the same extras:

uv sync --extra test --extra agents --extra lib-matplotlib --extra lib-seaborn \
  --extra lib-plotnine --extra lib-altair --extra lib-pygal --extra lib-letsplot \
  --extra lib-plotly --extra lib-bokeh

The e2e tests run in a separate job, "Run E2E Tests", after it: that job is the only one that logs in to GCP and reaches the Cloud SQL test database, and it skips PRs by Dependabot and Google Jules, whose code nobody has read yet.

-n is not in the pytest defaults, so a single file or test still runs in one process. For a full run, add -n auto (one worker per CPU): on four CPUs the CI-equivalent run takes about 80 s instead of about 3.5 minutes in one process. Tests must not depend on the order they run in or share a fixed file path, because each worker runs a different slice of the suite in its own process.

Coverage target: 90%+ on tested modules (core/, api/, automation/, agentic/workflows/modules/, agents/). Coverage config is in pyproject.toml ([tool.coverage]) and codecov.yml. Coverage measures through sys.monitoring (core = "sysmon"), coverage's default on Python 3.14; on Python 3.13 the C tracer made a serial run about 45 % slower. branch stays off: sysmon measures branches on 3.14, but turning it on changes the reported coverage.

Code quality

Both linting and formatting must pass for CI. Ruff is in the dev extra, so the commands name it: they then work in any environment, including a fresh worktree or one that a narrower uv sync left without it.

# Check linting
uv run --extra dev ruff check .

# Auto-fix linting issues
uv run --extra dev ruff check . --fix

# Check formatting
uv run --extra dev ruff format --check .

# Auto-format
uv run --extra dev ruff format .

Always run before committing:

uv run --extra dev ruff check . && uv run --extra dev ruff format .

Database

Local PostgreSQL

Set DATABASE_URL in .env:

DATABASE_URL=postgresql+asyncpg://user:pass@localhost:5432/anyplot

Cloud SQL (development)

For Cloud SQL access, your IP must be in the authorized networks. Set:

DATABASE_URL=postgresql+asyncpg://user:pass@CLOUD_SQL_PUBLIC_IP:5432/anyplot

Migrations

# Apply all migrations
uv run alembic upgrade head

# Create new migration
uv run alembic revision --autogenerate -m "description"

# Check current version
uv run alembic current

Run these only against a database of your own, such as a local PostgreSQL. The Cloud SQL database is the shared production database, and it receives a new revision only through sync-postgres.yml after the merge.

To verify a new revision:

  1. Push the branch and open the PR. The CI job Verify Migrations (.github/workflows/ci-migrations.yml) runs on every PR that touches alembic/, alembic.ini, core/database/, pyproject.toml or uv.lock. On a throwaway PostgreSQL 18 container it checks for a single head, runs alembic upgrade head and alembic check, compares the CHECK constraints with the models, and downgrades every revision the PR adds and upgrades again. It needs nothing on your machine.
  2. Optional: to check before you push, run the same checks locally against a throwaway PostgreSQL with the /verify-migrations skill (.claude/skills/verify-migrations/SKILL.md). It starts the database with Docker, or with the pgserver wheel where Docker doesn't run, and exports the throwaway DATABASE_URL in the same command as every alembic call, so no check can reach production.

How a merged revision reaches production, and how to revert one, is in Database migrations.

Environment variables

Copy .env.example and configure:

Variable Required Description
DATABASE_URL Yes PostgreSQL connection string
GCS_BUCKET No GCS bucket for images (default: anyplot-images)
GOOGLE_APPLICATION_CREDENTIALS No Path to service account JSON
ENVIRONMENT No development or production
ORIGIN_SECRET No Production only — leave unset locally and in tests. Arms the origin gate, which refuses every request that did not come through Cloudflare

Project structure

anyplot/
├── api/                # FastAPI backend
├── app/                # React frontend
├── core/               # Shared business logic
├── plots/              # Plot specifications and implementations
├── prompts/            # AI agent prompts
├── tests/              # Test suite
│   ├── unit/           # Fast, mocked tests
│   ├── integration/    # SQLite in-memory
│   └── e2e/            # Real PostgreSQL
└── docs/               # Documentation

See Repository Structure for details.

Useful commands

# Run single test file
uv run pytest tests/unit/api/test_routers.py

# Run single test
uv run pytest tests/unit/api/test_routers.py::test_get_specs -v

# Debug test failures
uv run pytest --pdb

# Check database connection
uv run python -c "from core.database import is_db_configured; print(is_db_configured())"

Troubleshooting

Import errors

uv sync --reinstall

Database connection issues

# Test connection
psql -U anyplot -d anyplot -h localhost

# Check migrations
uv run alembic current

Test failures

  • Unit/integration tests should work without DATABASE_URL
  • E2E tests are skipped if DATABASE_URL is not set
  • Run uv run pytest tests/unit -v to isolate issues