This guide covers setting up a HyperFleet environment for running E2E tests locally.
Clone the infrastructure repository:
git clone https://github.com/openshift-hyperfleet/hyperfleet-infra/
cd hyperfleet-infra/terraformChoose one of the following deployment options based on your needs:
- Kind (local): Fast setup, no cloud dependencies, uses port-forwarding
- GCP: Cloud environment, requires GCP access, slower setup, uses a LoadBalancer service
Both options deploy the desire delivery stack next to the adapters: Redis (the desire store), the applier, and the remote adapter cl-desire. The remote-transport suites need it (test cases). See Desire delivery in the hyperfleet-infra README.
The tests reach the API through the gateway, svc/hyperfleet-gateway, which listens on plain HTTP on port 8000. svc/hyperfleet-api is ClusterIP-only and serves TLS to the gateway, so do not point the tests at it.
The E2E runner defaults to AUTH_MODE=EDGE+API. Set AUTH_MODE to the same
value used to deploy HyperFleet. Supported values are NONE, API, EDGE, and
EDGE+API.
The E2E runner uses TENANT_MODEL=onprem by default. Set TENANT_MODEL to the
value that you used to deploy HyperFleet. Supported values are onprem and
oracle. The tenant model selects the tenant claims in human tokens.
In authenticated modes, normal API operations run as a human. At startup, each
test process mints a JWT for hyperfleet-e2e@example.com. The JWT comes from
the mock OIDC issuer in NAMESPACE (svc/hyperfleet-mock-oidc, release
hyperfleet-mock-oidc). The process mints a new token shortly before the old
token expires.
The suite reaches the issuer through the Kubernetes API server service proxy. Each mint is one HTTP request.
The tenant of the default human is the run ID (RUN_ID). Because of this,
reruns in one namespace do not see each other's resources.
The adapters and the machine-auth specs in e2e/auth use machine tokens. These
specs mint their own Kubernetes TokenRequest tokens for hyperfleet-e2e-runner
and for a temporary alternate ServiceAccount. For which identity a spec should
use, see API Identities.
| Mode | Token sent by E2E for API operations | Machine token sent by test adapters |
|---|---|---|
NONE |
No token | Chart default; auth is disabled |
API |
Bearer <mock OIDC JWT> |
Chart default Bearer |
EDGE |
Bearer <mock OIDC JWT> |
ServiceAccount <token> |
EDGE+API |
Bearer <mock OIDC JWT> |
ServiceAccount <token> |
The schemes select different Authorino authentication methods. ServiceAccount
triggers Kubernetes TokenReview; Bearer triggers human OIDC JWT validation.
This prevents a machine token from entering the human-token path. See the
infra authentication explanation.
The expected created_by and deleted_by value depends on the mode:
APIandEDGE+API: the email of the human.EDGE:system@hyperfleet.local. In this mode, the API does not validate tokens and records its system default.
HYPERFLEET_IDENTITY_EXPECTEDIDENTITY overrides the value.
Authenticated runs need:
- The mock OIDC issuer in
NAMESPACE. Deploy it withOIDC_ISSUER_MODE=mock. This is the hyperfleet-infra default fore2e-kindande2e-gcp. - A kubeconfig with these permissions in
NAMESPACE:- Create and get
services/proxy. The token request and the discovery check in Human token mint failed use it. - Create TokenRequests for ServiceAccounts.
- Create the temporary alternate ServiceAccount.
- Create and get
Before any spec runs, each test process also lists clusters as the default human. If the mint fails or the API does not answer 200, the suite stops. See Human token mint failed and Default human rejected.
1. Deploy HyperFleet to Kind cluster:
# option 1: Export namespace and helmfile env
export NAMESPACE=<your-dev-namespace> ; export HELMFILE_ENV=e2e-kind ; make local-up-kind
# option 2: Set in command line
NAMESPACE=<your-dev-namespace> HELMFILE_ENV=e2e-kind make local-up-kind2. Port-forward the gateway:
export API_LOCAL_PORT=8000
kubectl port-forward -n ${NAMESPACE} svc/hyperfleet-gateway ${API_LOCAL_PORT}:80003. Configure environment variables:
export HYPERFLEET_API_URL=http://localhost:${API_LOCAL_PORT}
export NAMESPACE=<your-dev-namespace>4. Verify deployment:
# Check Helm releases
helm list -n ${NAMESPACE}
# Verify all pods are running
kubectl get pods -n ${NAMESPACE}
# Test API connectivity
curl -f -X GET ${HYPERFLEET_API_URL}/api/hyperfleet/v1/clusters/1. Deploy HyperFleet to GCP cluster:
Note: Make sure your terraform files are up to date. See hyperfleet-infra/CONTRIBUTING.md for details.
- terraform/envs/gke/dev.tfbackend
- terraform/envs/gke/dev.tfvars
See hyperfleet-infra/README.md for infrastructure deployment details.
# option 1: Export namespace and helmfile env
export NAMESPACE=<your-dev-namespace> ; export HELMFILE_ENV=e2e-gcp ; make local-up-gcp
# option 2: Set in command line
NAMESPACE=<your-dev-namespace> HELMFILE_ENV=e2e-gcp make local-up-gcp2. Wait for the gateway's external IP:
In e2e-gcp, svc/hyperfleet-gateway is a LoadBalancer.
# Wait for the external IP to be assigned (may take 1-2 minutes)
kubectl get svc hyperfleet-gateway -n ${NAMESPACE} -wWith AUTH_MODE=EDGE or EDGE+API, Authorino denies requests whose host it does not know. For a dynamically assigned LoadBalancer IP, re-apply the gateway with AUTHORINO_HOSTS set to that IP before running tests. CI setup performs this second apply automatically.
3. Configure environment variables:
export API_EXTERNAL_IP=$(kubectl get svc hyperfleet-gateway -n ${NAMESPACE} -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
export HYPERFLEET_API_URL=http://${API_EXTERNAL_IP}:8000
export NAMESPACE=<your-dev-namespace>4. Verify deployment:
# Check Helm releases
helm list -n ${NAMESPACE}
# Verify all pods are running
kubectl get pods -n ${NAMESPACE}
# Test API connectivity
curl -f -X GET ${HYPERFLEET_API_URL}/api/hyperfleet/v1/clusters/Every live run starts with a read-only drift check: hyperfleet-e2e runs
helm get values hyperfleet-api --all --namespace "$NAMESPACE" and compares the
deployed config.entities with its descriptor table. Set NAMESPACE to the
namespace that holds the API release and make sure helm and KUBECONFIG can
read it. A mismatch fails the suite before any spec runs; see
debugging.md. make list-tests and
--dry-run skip the check.
If your deployment uses custom image settings, update env/env.local in this repo to match your infrastructure deployment settings:
Update env/env.local:
# env/env.local
export IMAGE_REGISTRY=<registry>
export <COMPONENT>_IMAGE_REPO=<repo>
export <COMPONENT>_IMAGE_TAG=<tag>Source the configuration:
source env/env.localThis configuration is required for running tier2 tests.
For additional help with infrastructure deployment and configuration, see:
- hyperfleet-infra README - Main infrastructure documentation
For test-specific troubleshooting (timeouts, API errors, namespace mismatches), see the Runbook Troubleshooting section.
Next Steps: Once your environment is set up, see the Runbook for running tests and troubleshooting.