Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md

Large diffs are not rendered by default.

44 changes: 44 additions & 0 deletions packages/agentenv-protocol/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -490,6 +490,50 @@ called with the caller's role in `ToolContext`. The site's `/mcp` is one of the
browsing route (`/website/<name>/`) never reaches. `python -m agentenv_protocol.website.conformance` checks a declared
endpoint (`mcp`): it answers `initialize` and lists the tools the card does.

## Live websites

A live site is a real one, such as github.com, that a task puts in a multi env beside synthetic sites. Its server serves
none of its pages: the real site does, and the computer reaches it over its own network. The server holds what agentenv
knows of the site: its card, a data plane that seeds and reads the task's account through the real site's API, `ready`
saying the account works, and `session`, the account's cookies and local storage, which `browser_sign_in` loads into the
browser.

```python
from pydantic import BaseModel, SecretStr
from agentenv_protocol.website import WebsiteEnvironment, ready_check, session_state, website


@website(domains=["github.com"], live=True, allow_hosts=["*.githubassets.com", "api.github.com"])
class GitHub(WebsiteEnvironment):
class Settings(BaseModel):
token: SecretStr # in env.toml as a secret:KEY or env:NAME reference

@ready_check
def account_works(self) -> bool:
... # the API answers for self.settings.token

@session_state
def account(self, user: str | None) -> dict:
... # {"cookies": [...], "local_storage": [...]} that sign a browser in
```

- **Its names:** `domains` are the real site's, where start tabs may open and `browser_sign_in` sets the session's
cookies. `allow_hosts` are the other hosts its pages reach, such as its assets and API. A computer whose network is an
allowlist may reach both; one that allows every host is left as it is. A name a synthetic site of the run also claims
is that site's.
- **Its card:** `website/v1` says `live` (`WEBSITE_LIVE_PARAM`) and `allow_hosts` (`WEBSITE_ALLOW_HOSTS_PARAM`) beside
`domains`, and names no routes. The server serves no `/api/session` page route. `@website` refuses a live site
with no domains or a static folder, and `allow_hosts` on a site that isn't live.
- **Its session:** the method marked `@session_state` answers `session {user}`. A live site must mark one, since only
the real site's cookies sign a browser in to it. Each cookie carries no domain and a `max_age`, as a synthetic
site's does.
- **Settings:** any site may declare `Settings`. A deploy hands them to its server in `WEBSITE_SETTINGS_ENV`, a JSON
object, and the site reads them as `self.settings`, unless it has a member of that name of its own. A `SecretStr`
field's value in env.toml is a `secret:` or `env:` reference, which agentenv resolves only as it writes the server's
container, and every record shows it as withheld.
- **The run:** a run that deploys a live site is marked live. The deploy record says `live`, and the run's metadata lists
its live sites with `reproducible: false`, because the real site may change between runs.

## Computer drivers

A computer driver is what runs inside a computer an agent uses through screenshots and input. Write
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,23 @@
from .clock import CLOCK_EXTENSION_URI, UNIVERSE_NOW_HEADER, ClockUnavailable, UniverseClock
from .environment import (
SESSION_PATH,
WEBSITE_SETTINGS_ENV,
WebsiteDeclaration,
WebsiteEnvironment,
host_problem,
ready_check,
session_state,
website,
)
from .errors import ApiError, BadRequest, Conflict, Forbidden, NotFound, Unauthorized
from .readiness import (
DEFAULT_HEALTH_PATH,
READY_TIMEOUT_S,
SITE_READY_TIMEOUT_S,
WEBSITE_ALLOW_HOSTS_PARAM,
WEBSITE_BROWSE_URL_PARAM,
WEBSITE_EXTENSION_URI,
WEBSITE_LIVE_PARAM,
WEBSITE_READY_OP,
WebsiteReadiness,
env_readiness,
Expand All @@ -32,11 +36,11 @@
from .session import SESSION_COOKIE, SESSION_SECRET_ENV, SessionSigner

__all__ = [
"WebsiteEnvironment", "WebsiteDeclaration", "website", "api", "ApiContext", "ApiRoute", "ready_check", "host_problem",
"WebsiteEnvironment", "WebsiteDeclaration", "website", "api", "ApiContext", "ApiRoute", "ready_check", "session_state", "host_problem",
"ApiError", "BadRequest", "Unauthorized", "Forbidden", "NotFound", "Conflict",
"UniverseClock", "ClockUnavailable", "SessionSigner", "reserved_path",
"WEBSITE_EXTENSION_URI", "CLOCK_EXTENSION_URI", "UNIVERSE_NOW_HEADER", "DEFAULT_HEALTH_PATH", "SESSION_PATH",
"SESSION_COOKIE", "SESSION_SECRET_ENV", "ROLE_KEY_ENV", "ROLE_KEY_HEADER",
"WEBSITE_READY_OP", "WEBSITE_BROWSE_URL_PARAM", "READY_TIMEOUT_S", "SITE_READY_TIMEOUT_S", "WebsiteReadiness",
"WEBSITE_READY_OP", "WEBSITE_BROWSE_URL_PARAM", "WEBSITE_LIVE_PARAM", "WEBSITE_ALLOW_HOSTS_PARAM", "WEBSITE_SETTINGS_ENV", "READY_TIMEOUT_S", "SITE_READY_TIMEOUT_S", "WebsiteReadiness",
"website_readiness", "env_readiness",
]
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,19 @@

One route is built in, replaced by a site's own ``@api`` route at its very path and answering before one whose path
parameters would also match it: ``/api/session``, which a frontend calls to learn who is signed in, signing the acting
user in when nobody is (GET), or to sign out (DELETE)."""
user in when nobody is (GET), or to sign out (DELETE).

A live site, ``@website(live=True, ...)``, is a real site, such as github.com, that its server serves no pages of: the
real site does, at its ``domains``, its pages reaching the ``allow_hosts`` too. Its server holds what agentenv knows of
it: the card, a data plane that seeds and reads its account through the real site's API, ``ready`` saying the account
works, and ``session``, the account's cookies and local storage, from the method marked ``@session_state``. The site's
``Settings``, which a deploy hands it as ``WEBSITE_SETTINGS_ENV`` (its credentials, ``SecretStr`` fields, among them), are
``self.settings``, unless the site has a member of that name of its own; any site may declare them."""
from __future__ import annotations

import asyncio
import inspect
import json
import math
import os
import re
Expand All @@ -27,6 +35,7 @@
from typing import Any, ClassVar, NamedTuple, Optional
from urllib.parse import unquote

from pydantic import BaseModel
from starlette.applications import Starlette
from starlette.datastructures import MutableHeaders
from starlette.middleware import Middleware
Expand All @@ -46,47 +55,66 @@
)
from .clock import CLOCK_EXTENSION_URI, UNIVERSE_NOW_HEADER, ClockUnavailable, UniverseClock, clock_patch_script, iso
from .errors import ApiError, NotFound
from .readiness import SITE_READY_TIMEOUT_S, WEBSITE_EXTENSION_URI, WEBSITE_READY_OP, WebsiteReadiness
from .readiness import (SITE_READY_TIMEOUT_S, WEBSITE_ALLOW_HOSTS_PARAM, WEBSITE_EXTENSION_URI, WEBSITE_LIVE_PARAM,
WEBSITE_READY_OP, WebsiteReadiness)
from .routes import (ApiContext, ApiRoute, endpoint, error_response, forwarded_prefix, openapi, openapi_path, request_role,
reserved_path, routes_of)
from .session import SessionSigner, session_cookie_name

SESSION_PATH = "/api/session"
#: The variable a deploy hands a site its settings in, a JSON object of each by its name.
WEBSITE_SETTINGS_ENV = "AGENTENV_WEBSITE_SETTINGS"
CHECK_TIMEOUT_S = 10.0
_WEBSITE_ATTR = "_aee_website"
_READY_ATTR = "_aee_website_ready"
_SESSION_ATTR = "_aee_website_session"
_HEAD = re.compile(rb"<head(?:\s[^>]*)?>", re.IGNORECASE)
_PREAMBLE = re.compile(rb"\A(?:\s|<!--.*?-->|<!doctype[^>]*>|<html(?:\s[^>]*)?>)*", re.IGNORECASE | re.DOTALL)
_HOST = re.compile(r"(\*\.)?[A-Za-z0-9]([A-Za-z0-9-]*[A-Za-z0-9])?(\.[A-Za-z0-9]([A-Za-z0-9-]*[A-Za-z0-9])?)*")


class WebsiteDeclaration(NamedTuple):
"""What ``@website`` declares: the site's domains, its static folder (relative to the class's file), how long it
may take to be ready after its server starts, and whether its HTML pages read the universe's time."""
may take to be ready after its server starts, whether its HTML pages read the universe's time, and, for a live
site, that it is one and the other hosts its pages reach."""

domains: tuple[str, ...] = ()
static: Optional[str] = None
ready_timeout_s: float = SITE_READY_TIMEOUT_S
clock_patch: bool = False
live: bool = False
allow_hosts: tuple[str, ...] = ()


def website(_cls: type | None = None, *, domains: list[str] | tuple[str, ...] = (), static: str | None = None,
ready_timeout_s: float = SITE_READY_TIMEOUT_S, clock_patch: bool = False) -> Any:
"""Declare a WebsiteEnvironment's site."""
ready_timeout_s: float = SITE_READY_TIMEOUT_S, clock_patch: bool = False, live: bool = False,
allow_hosts: list[str] | tuple[str, ...] = ()) -> Any:
"""Declare a WebsiteEnvironment's site; ``live=True`` a real one, at its ``domains``, whose pages reach the
``allow_hosts`` too."""
if _cls is not None:
raise TypeError("use @website(...) with parentheses")
if isinstance(domains, str):
raise ValueError(f"domains is a list of host names, not the string {domains!r}")
if problems := [problem for host in domains if (problem := host_problem(host))]:
raise ValueError(f"domains: {problems[0]}")
for field, hosts in (("domains", domains), ("allow_hosts", allow_hosts)):
if isinstance(hosts, str):
raise ValueError(f"{field} is a list of host names, not the string {hosts!r}")
if problems := [problem for host in hosts if (problem := host_problem(host))]:
raise ValueError(f"{field}: {problems[0]}")
if not isinstance(live, bool):
raise ValueError(f"live is True or False, not {live!r}")
if live and not domains:
raise ValueError("a live site names the domains the real site is at")
if live and static is not None:
raise ValueError("a live site serves no pages, the real site does, so it takes no static folder")
if allow_hosts and not live:
raise ValueError("allow_hosts are the other hosts a live site's pages reach; declare live=True")
if isinstance(ready_timeout_s, bool) or not isinstance(ready_timeout_s, (int, float)) or not (
0 < ready_timeout_s < math.inf):
raise ValueError(f"ready_timeout_s is a number of seconds above 0, not {ready_timeout_s!r}")

def deco(cls: type) -> type:
if not (isinstance(cls, type) and issubclass(cls, WebsiteEnvironment)):
raise TypeError(f"@website declares a WebsiteEnvironment subclass, not {cls!r}")
setattr(cls, _WEBSITE_ATTR, WebsiteDeclaration(tuple(domains), static, float(ready_timeout_s), clock_patch))
setattr(cls, _WEBSITE_ATTR, WebsiteDeclaration(tuple(domains), static, float(ready_timeout_s), clock_patch, live,
tuple(allow_hosts)))
return cls

return deco
Expand All @@ -100,6 +128,14 @@ def host_problem(host: Any) -> str | None:
return None


def session_state(fn: Any) -> Any:
"""Mark the method that answers ``website/v1``'s ``session``: it takes ``user``, the account to sign in as or
None, and returns ``{cookies, local_storage}``, each cookie with no domain and a ``max_age``. A live site marks
one, since only the real site's cookies sign a browser in to it; a site that marks none signs its own cookie."""
setattr(fn, _SESSION_ATTR, True)
return fn


def ready_check(fn: Any) -> Any:
"""Mark a method that says whether the site is usable yet: ``website/v1``'s ``ready`` says ready once every one
returns a truthy value. They run at once, each within ``CHECK_TIMEOUT_S``, a plain ``def`` in a worker thread."""
Expand All @@ -115,6 +151,10 @@ class WebsiteEnvironment(AgentEnvEnvironment):
app: ClassVar[Any] = None
asgi: Any = None

class Settings(BaseModel):
"""The site's settings, which a deploy hands it: a site declares its own; a ``SecretStr`` field is a secret,
whose env.toml value is a ``secret:`` or ``env:`` reference agentenv resolves only as it starts the server."""

def __init_subclass__(cls, **kwargs: Any) -> None:
super().__init_subclass__(**kwargs)
routes_of(cls)
Expand All @@ -124,6 +164,9 @@ def __init__(self) -> None:
self._signer = SessionSigner(cookie_name=session_cookie_name(self._build_card().name))
self._started = time.monotonic()
self._checking: dict[str, asyncio.Future] = {}
settings = type(self).Settings.model_validate(json.loads(os.environ.get(WEBSITE_SETTINGS_ENV) or "{}"))
if not hasattr(type(self), "settings"): # a site's own member named settings, an @api route say, stays its own
self.settings = settings

@classmethod
def declaration(cls) -> WebsiteDeclaration:
Expand Down Expand Up @@ -180,7 +223,7 @@ def create_app(self) -> Starlette:
else:
asgi = Starlette(middleware=middleware)
AgentEnvStarletteApplication(self._build_card(), self).add_routes_to_app(asgi)
if SESSION_PATH not in taken:
if SESSION_PATH not in taken and not declaration.live:
asgi.router.routes.append(Route(SESSION_PATH, self._session, methods=["GET", "DELETE"]))
asgi.router.routes.extend(Route(r.path, _unreserved(endpoint(getattr(self, r.fn.__name__), r, self.context)),
methods=[r.method]) for r in api_routes)
Expand Down Expand Up @@ -209,6 +252,9 @@ def serve(self) -> None:
def extension_params(self, uri: str) -> dict:
if uri == WEBSITE_EXTENSION_URI:
declaration = self.declaration()
if declaration.live:
return {"domains": list(declaration.domains), WEBSITE_LIVE_PARAM: True,
WEBSITE_ALLOW_HOSTS_PARAM: list(declaration.allow_hosts)}
return {"domains": list(declaration.domains),
"routes": [r.advertised() for r in self.routes()], "session_cookie": self.session_cookie,
"session_path": SESSION_PATH}
Expand Down Expand Up @@ -242,6 +288,18 @@ async def _website_ready(self) -> dict:

@extension(WEBSITE_EXTENSION_URI, op="session")
async def _website_session(self, user: Optional[str] = None) -> dict:
marked = [method for method, _marker in _marked_members(self, _SESSION_ATTR)]
if len(marked) > 1:
raise TypeError(f"{type(self).__qualname__} marks {len(marked)} methods @session_state; mark one")
if marked:
state = await _maybe_await(marked[0](user))
if not (isinstance(state, dict) and isinstance(state.get("cookies", []), list)
and isinstance(state.get("local_storage", []), list)):
raise TypeError(f"@session_state {marked[0].__name__} returns {{cookies, local_storage}}, not {state!r}")
return {"cookies": state.get("cookies", []), "local_storage": state.get("local_storage", [])}
if self.declaration().live:
raise TypeError(f"{type(self).__qualname__} is a live site and marks no @session_state method, so it can't "
"sign a browser in")
return {"cookies": [self._session_signer().cookie(user)], "local_storage": []}

async def _not_ready(self) -> list[str]:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@
WEBSITE_EXTENSION_URI = "urn:agentenv:website/v1"
WEBSITE_READY_OP = "ready"
WEBSITE_BROWSE_URL_PARAM = "browse_url"
#: A live site's ``website/v1`` params: that it is one, and the other hosts its pages reach.
WEBSITE_LIVE_PARAM = "live"
WEBSITE_ALLOW_HOSTS_PARAM = "allow_hosts"
READY_TIMEOUT_S = 10.0
SITE_READY_TIMEOUT_S = 300.0
DEFAULT_HEALTH_PATH = "/api/health"
Expand Down
101 changes: 101 additions & 0 deletions packages/agentenv-protocol/tests/test_website_live.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
"""A live site on WebsiteEnvironment: a real site, such as github.com, whose server serves none of its pages, only the
card, with website/v1's live params, a data plane, ready and the account's session from its ``@session_state`` method;
and a site's ``Settings``, which a deploy hands it in ``WEBSITE_SETTINGS_ENV``."""

from __future__ import annotations

import asyncio
import json

import pytest
from pydantic import BaseModel, SecretStr
from starlette.testclient import TestClient

from agentenv_protocol.website import (
WEBSITE_ALLOW_HOSTS_PARAM,
WEBSITE_EXTENSION_URI,
WEBSITE_LIVE_PARAM,
WEBSITE_SETTINGS_ENV,
WebsiteEnvironment,
session_state,
website,
)


@website(domains=["github.com"], live=True, allow_hosts=["*.githubassets.com", "api.github.com"])
class GitHub(WebsiteEnvironment):
class Settings(BaseModel):
token: SecretStr
org: str = "octo"

@session_state
async def account(self, user: str | None) -> dict:
value = f"{user}:{self.settings.token.get_secret_value()}"
return {"cookies": [{"name": "user_session", "value": value, "path": "/", "max_age": 60}], "local_storage": []}


@pytest.fixture
def github(monkeypatch) -> GitHub:
monkeypatch.setenv(WEBSITE_SETTINGS_ENV, json.dumps({"token": "t0"}))
return GitHub()


def _params(client: TestClient) -> dict:
card = client.get("/.well-known/agent-env.json").json()
return next(e for e in card["capabilities"]["extensions"] if e["uri"] == WEBSITE_EXTENSION_URI)["params"]


def test_a_live_sites_card_names_the_real_site_and_the_hosts_its_pages_reach_and_it_serves_no_pages(github) -> None:
client = TestClient(github.create_app())

params = _params(client)

assert {key: params[key] for key in ("domains", WEBSITE_LIVE_PARAM, WEBSITE_ALLOW_HOSTS_PARAM)} == {
"domains": ["github.com"], WEBSITE_LIVE_PARAM: True,
WEBSITE_ALLOW_HOSTS_PARAM: ["*.githubassets.com", "api.github.com"]}
assert not {"routes", "session_cookie", "session_path"} & params.keys()
assert params["methods"]["session"]["endpoint"] == "/agentenv/ext/website/session"
assert (client.get("/").status_code, client.get("/api/session").status_code) == (404, 404)


def test_a_live_sites_session_is_its_accounts_as_its_session_state_method_says(github) -> None:
client = TestClient(github.create_app())

state = client.post("/agentenv/ext/website/session", json={"user": "alex"}).json()

assert state == {"cookies": [{"name": "user_session", "value": "alex:t0", "path": "/", "max_age": 60}],
"local_storage": []}


def test_a_sites_settings_arrive_from_the_deploy_and_a_secret_stays_one(github, monkeypatch) -> None:
assert (github.settings.org, github.settings.token.get_secret_value()) == ("octo", "t0")
assert "t0" not in repr(github.settings)
monkeypatch.delenv(WEBSITE_SETTINGS_ENV)

@website(domains=["shop.test"])
class Shop(WebsiteEnvironment):
pass

assert Shop().settings.model_dump() == {}


def test_a_live_site_marking_no_session_state_method_cant_sign_a_browser_in() -> None:
@website(domains=["github.com"], live=True)
class Bare(WebsiteEnvironment):
pass

with pytest.raises(TypeError, match="marks no @session_state method"):
asyncio.run(Bare()._website_session("alex"))


@pytest.mark.parametrize(("kwargs", "problem"), [
({"live": True}, "names the domains the real site is at"),
({"domains": ["github.com"], "live": True, "static": "dist/"}, "serves no pages"),
({"domains": ["github.com"], "allow_hosts": ["api.github.com"]}, "declare live=True"),
({"domains": ["github.com"], "live": True, "allow_hosts": "api.github.com"}, "not the string"),
({"domains": ["github.com"], "live": True, "allow_hosts": ["https://api.github.com"]}, "not a host name"),
({"domains": ["github.com"], "live": "yes"}, "True or False"),
], ids=["no-domains", "static", "allow-hosts-not-live", "allow-hosts-string", "allow-hosts-url", "live-not-bool"])
def test_website_refuses_a_live_declaration_that_cant_be_one(kwargs, problem) -> None:
with pytest.raises(ValueError, match=problem):
website(**kwargs)
Loading
Loading