Skip to content

Enhance generated HTTPException: rich context, parsed error models, configurable handling #100

Open
#124
Open
Enhance generated HTTPException: rich context, parsed error models, configurable handling#100
#124

Description

@dougborg
dougborg
opened on Aug 10, 2025

Summary

Improve the generated client's error handling around non-success HTTP responses. Currently each service method checks for a single expected status code and raises HTTPException(status_code, ' failed with status code: <code>') without context, parsed error model, or headers.

Problems With Current Behavior

  • Error message lacks HTTP method, path, expected vs actual status.
  • Response body is discarded (critical for debugging validation errors, rate limits, auth issues).
  • No attempt to parse documented error schemas (e.g. ErrorResponse).
  • Headers (rate limit, retry, correlation/request IDs) are not surfaced.
  • Only one success code is recognized (often 200/201/204 variations in real APIs).
  • No hook for custom retry / logging strategies.

Proposed Enhancements

  1. Rich HTTPException subclass (e.g. APIError) with attributes:
    • status_code: int
    • method: str
    • path: str
    • expected_statuses: list[int]
    • response_headers: dict[str, str]
    • response_text: str (raw body, truncated to e.g. 2KB)
    • error_model: Optional[BaseModel] (parsed from first matching error schema if possible)
  2. Include informative __str__ showing method path status and short body snippet.
  3. Recognize multiple success codes derived from spec (all 2XX responses listed) instead of just the first one.
  4. Attempt to parse error body:
    • Inspect documented non-2xx responses in operation.responses.
    • If JSON and schema is a $ref, import & parse into the model (pydantic).
    • Attach parsed model (or validation error info) to exception.
  5. Expose helper raise_for_status(response, operation_meta) inside generated code to centralize logic.
  6. Optional user hook: allow APIConfig to accept error_handler: Callable[[APIError], None] invoked before raising (for logging / metrics / custom retry).
  7. Backwards compatibility: keep HTTPException name but implement new fields; retain current constructor signature; old code catching HTTPException continues to work.
  8. Async parity: mirror behavior for async clients.

Stretch (Optional / Future)

  • Auto retry on 429 with Retry-After respect (configurable max attempts).
  • Deserialize XML or text/plain when declared in spec.
  • Structured rate-limit info (limit, remaining, reset) extracted into fields.

Acceptance Criteria

  • Generated services raise enriched exception containing method, path, status, and truncated body.
  • All documented 2xx codes for an operation are accepted without raising.
  • If an error schema is defined (e.g. ErrorResponse), exception contains parsed model.
  • Unit tests cover: single 200, multiple success codes (200/201), error with JSON model, error with invalid JSON, 429 rate limit header extraction.
  • Backwards compatibility test: existing code catching HTTPException still works.

Rationale

Better diagnostics accelerates integration debugging and reduces guesswork when consuming generated clients—especially for large OpenAPI 3.1 specs.

Implementation Sketch

  • Add new template snippet for a shared errors.py in output (or embed in api_config.py for simplicity).
  • During service generation, collect all success status codes (keys starting with '2').
  • Replace inline status check with call to helper that returns early if response.status_code in expected_codes else raises enriched exception.
  • Add conditional parsing of error body using the first matching documented non-2xx response schema with a JSON content type.

Let me know if any constraints/preferences before implementation and I can adjust this plan.

Activity

  1. dougborg
    added 2 commits that reference this issue on Oct 27, 2025

    Add design document for Issue MarcoMuellner#100: Enhanced error handling

    a00597e

    Add static file infrastructure for generated clients

    96bbafc

  2. dougborg commented on Oct 27, 2025

    @dougborg
    dougborg
    on Oct 27, 2025
    Author

    Implementation Roadmap

    A comprehensive design document has been created and is now under review in #124. Based on that design, here's the proposed implementation roadmap:

    🎯 Phase 0: Foundation (In Progress)

    PR #125 - Static File Infrastructure ✅

    • Add StaticFile model and ConversionResult.static_files field
    • Create static_file_generator.py module
    • Integrate static file writing into write_data()
    • Add comprehensive tests
    • Status: Ready for review

    PR #124 - Design Document 📋

    • Complete architecture specification
    • API patterns and examples
    • 10-phase implementation plan
    • Backwards compatibility analysis
    • Status: Awaiting design approval

    📋 Proposed Implementation Issues

    Once the design is approved (#124) and infrastructure is merged (#125), we'll create the following issues:

    Issue: Core Exception Hierarchy

    Estimated Effort: Small (1-2 days)
    Depends On: #125

    Implement the base exception types:

    • Create errors.py static file with APIError, ClientError, ServerError
    • Add HTTPException backwards-compatible alias
    • Implement rich __str__ with context display and body truncation
    • Write comprehensive unit tests for exception creation and formatting

    Acceptance Criteria:

    • Exception classes generated in all clients
    • Full HTTP context stored (status, method, URL, headers, body, etc.)
    • Body truncated only for display, not storage
    • All tests passing with >90% coverage

    Issue: Response Wrapper & Utilities

    Estimated Effort: Medium (2-3 days)
    Depends On: Core Exception Hierarchy

    Implement DetailedResponse and helper utilities:

    • Create response.py with DetailedResponse[T] generic class
    • Implement fluent API methods: on(), otherwise(), resolve()
    • Create utils.py with unwrap(), expect(), match()
    • Add error utilities: has_error_model(), UnexpectedStatusError
    • Write comprehensive tests for all utilities

    Acceptance Criteria:

    • DetailedResponse supports generic types and fluent API
    • Utility functions work with all response types
    • Type hints are complete and accurate
    • Documentation and examples provided

    Issue: Service Generator - Success Code Collection

    Estimated Effort: Medium (2-3 days)
    Depends On: Core Exception Hierarchy

    Update service generator to collect all success codes:

    • Implement _collect_success_codes() for exact codes and wildcards
    • Implement _collect_error_models() to extract error schemas
    • Implement _collect_response_models() for success response models
    • Update OpReturnType model with new metadata fields
    • Add unit tests for collection functions

    Acceptance Criteria:

    • Correctly handles explicit codes (200, 201, 204)
    • Correctly handles wildcards (2XX, 4XX, 5XX)
    • Extracts error model names from $ref schemas
    • All edge cases tested (missing responses, invalid refs, etc.)

    Issue: Error Creation Functions

    Estimated Effort: Medium-Large (3-4 days)
    Depends On: Service Generator - Success Code Collection

    Implement error creation with model parsing:

    • Create _create_error_from_response() for httpx/requests
    • Create _create_error_from_response_async() for aiohttp
    • Implement wildcard pattern matching (4XX, 5XX, default)
    • Parse error models using model_validate_json()
    • Extract request IDs from headers (configurable header name)
    • Handle unparseable error bodies gracefully
    • Test with all three HTTP clients

    Acceptance Criteria:

    • Works with httpx, requests, and aiohttp
    • Correctly matches status codes (exact, wildcard, default)
    • Parses error models when defined in spec
    • Falls back to raw body when parsing fails
    • Extracts metadata (headers, request IDs)

    Issue: Simple API Generation

    Estimated Effort: Large (4-5 days)
    Depends On: Error Creation Functions

    Generate simple API functions (direct model returns):

    • Update service templates to call error creation function
    • Handle single success code case (most common)
    • Handle multiple success codes with same model
    • Handle multiple success codes with different models (pick first)
    • Generate docstrings with "Raises" section for documented errors
    • Add integration tests with real specs

    Acceptance Criteria:

    • Simple functions return models directly
    • Errors raise typed exceptions with full context
    • Works for all success code patterns
    • Generated docstrings document error conditions
    • Integration tests cover major OpenAPI specs (petstore, etc.)

    Issue: Detailed API Generation

    Estimated Effort: Large (4-5 days)
    Depends On: Simple API Generation

    Generate detailed API functions with response wrappers:

    • Generate _detailed() function variants for all operations
    • Return DetailedResponse[T] for same-model responses
    • Return DetailedResponse[Union[A, B]] for different-model responses
    • Implement status-based parsing to select correct model from Union
    • Generate comprehensive docstrings with usage examples
    • Add integration tests for detailed API

    Acceptance Criteria:

    • Detailed functions return DetailedResponse wrapper
    • Metadata accessible (status, headers, request ID)
    • Union types work correctly with multiple response models
    • Fluent API (on().otherwise().resolve()) works end-to-end
    • Documentation shows both simple and detailed patterns

    Issue: Type Guards for Union Responses

    Estimated Effort: Small-Medium (2-3 days)
    Depends On: Detailed API Generation

    Generate type guard functions for Union responses:

    • Generate is_<model>() type guard for each response model in Union
    • Use TypeGuard from typing_extensions for Pydantic 1 compatibility
    • Test type narrowing with mypy/pyright
    • Add examples to generated docstrings

    Acceptance Criteria:

    • Type guards generated only for Union response operations
    • Type narrowing works in both mypy and pyright
    • Examples demonstrate usage in docstrings

    Issue: Integration Testing & Validation

    Estimated Effort: Medium (3-4 days)
    Depends On: Type Guards, Detailed API Generation

    Comprehensive end-to-end testing:

    • Test with OpenAPI 3.0 and 3.1 specs
    • Test with petstore, GitHub API, Stripe API specs
    • Test all HTTP clients (httpx sync/async, requests, aiohttp)
    • Test backwards compatibility with existing clients
    • Performance testing (error model parsing overhead)
    • Edge case testing (malformed JSON, missing headers, etc.)

    Acceptance Criteria:

    • All major public OpenAPI specs generate successfully
    • Generated clients work with real APIs (mocked)
    • Backwards compatibility maintained (existing code works)
    • Performance within acceptable bounds (<10% overhead)
    • Edge cases handled gracefully

    Issue: Documentation & Examples

    Estimated Effort: Medium (2-3 days)
    Depends On: Integration Testing

    Complete user-facing documentation:

    • Update README with error handling features
    • Create tutorial: "Simple vs Detailed APIs"
    • Create tutorial: "Handling Errors and Union Responses"
    • Document utility functions (unwrap, expect, match)
    • Create migration guide for existing users
    • Document limitations and future enhancements

    Acceptance Criteria:

    • README clearly explains error handling features
    • Tutorials cover common use cases with examples
    • Migration guide helps existing users adopt new features
    • API reference documentation complete

    📊 Overall Progress Tracker

    Phase Status PR/Issue
    Phase 0: Foundation
    Static File Infrastructure ✅ In Review #125
    Design Document 📋 In Review #124
    Phase 1-10: Implementation
    Core Exception Hierarchy ⏳ Pending TBD
    Response Wrapper & Utilities ⏳ Pending TBD
    Service Generator Updates ⏳ Pending TBD
    Error Creation Functions ⏳ Pending TBD
    Simple API Generation ⏳ Pending TBD
    Detailed API Generation ⏳ Pending TBD
    Type Guards ⏳ Pending TBD
    Integration Testing ⏳ Pending TBD
    Documentation ⏳ Pending TBD

    🎯 Next Steps

    1. Review & Approve Design (Design: Enhanced Error Handling for Generated Clients (#100) #124) - Community feedback on architecture
    2. Merge Infrastructure (Add static file infrastructure for generated clients #125) - Foundation for all subsequent work
    3. Create Implementation Issues - Once design approved, create individual issues for each phase
    4. Begin Phase 1 - Core Exception Hierarchy

    ⏱️ Estimated Timeline

    • Phase 0 (Foundation): 1 week - In Progress
    • Phase 1-3 (Core Components): 2-3 weeks
    • Phase 4-7 (Code Generation): 3-4 weeks
    • Phase 8-10 (Testing & Docs): 2 weeks

    Total Estimated Duration: 8-10 weeks for full implementation

    💬 Feedback Welcome

    Please review the design document (#124) and provide feedback on:

    • Overall architecture and design decisions
    • API ergonomics (simple vs detailed patterns)
    • Backwards compatibility approach
    • Implementation phasing and priorities

    Let's discuss any concerns or suggestions before moving forward with implementation!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions