Skip to content

test: run reading-profile SQL Server tests on Testcontainers - #1614

Draft
BenjaminMichaelis wants to merge 39 commits into
bmichaelis-reading-time-tracker-8effrom
test/reading-profile-sqlserver-testcontainers
Draft

BenjaminMichaelis wants to merge 39 commits into
bmichaelis-reading-time-tracker-8effrom
test/reading-profile-sqlserver-testcontainers

Conversation

@BenjaminMichaelis

Copy link
Copy Markdown
Member

Summary

Follow-up to #1267. Adds the real-SQL-Server integration tests for the reading profile back, now self-provisioned with Testcontainers instead of an environment variable and a CI service container.

Stacked on #1267: the base is that PR's branch, because these tests exercise code that only exists there. Retarget to main once #1267 merges.

Why a separate PR

In #1267 these tests required ESSENTIALCSHARP_TEST_SQLSERVER and failed when it was missing in CI. The SQL Server service container and that variable existed only on the push-only "Build and test (Ubuntu)" job, so all 8 tests failed on every OS in the PR's CI matrix (8 of 540 per job, nothing else failing).

What it covers

Against a real SQL Server (SQLite serializes writers, so it cannot prove these):

  • concurrent writes for a new user and for an existing user lose no updates
  • the AddUserReadingProfile migration applies, reverts and reapplies; no pending model changes; expected columns
  • foreign key violation (SQL error 547) and cascade delete on user removal
  • reset only removes the caller's row

How it works

  • Testcontainers.MsSql 4.16.0; one container per test class with a pinned image (2022-CU20-ubuntu-22.04), disposed in the class teardown.
  • No env var, no workflow change.
  • Skip vs fail: when no Linux container runtime can be used (local, and the macOS/Windows jobs, whose hosted runners cannot run Linux containers) the tests are skipped with a reason. On GitHub Linux CI (GITHUB_ACTIONS=true and RUNNER_OS=Linux) a failure to start the container fails the tests, so coverage cannot silently disappear.

Testing

  • All 8 tests ran and passed locally against a Podman-backed container (about 60s); full suite 505 passed, 0 failed.
  • Not verified: the skip path (a missing runtime could not be reproduced locally) and the first real run on GitHub Linux CI. Draft until CI confirms.

Review notes

  • Linux CI adds roughly 40-60s for the image pull and container start.
  • The loud-fail condition relies on RUNNER_OS; an explicit workflow env var would be the alternative.
  • Off Linux CI the catch is deliberately broad, so a non-Docker startup error would skip with its message shown instead of failing.

BenjaminMichaelis and others added 30 commits November 5, 2026 20:47
- WordCountService: singleton that walks all SiteMappings at startup,
  strips <pre>/<code>/<script>/<style> via HtmlAgilityPack, and caches
  per-page, per-chapter, and book-wide prose word counts + prefix sums

- ReadingActivity + UserReadingProfile EF entities with migration
  AddReadingTimeTracking; retention capped at 500 rows/user

- ReadingController (POST /session, GET /profile, GET /book-stats,
  POST /reset); WPM algorithm: >900 WPM hard-discard, slow outlier
  clamped at 0.25x referenceWpm, aggregate stored as
  {totalWords, totalActiveSeconds}

- useReadingTracker.js composable: activity state machine (READING /
  IDLE / HIDDEN), 5-min idle threshold, visibilitychange instant-pause,
  localStorage for anonymous users, flush-on-navigate, sync local→server
  on first authenticated page load

- ReadingTimeRemaining.vue: displays 'X min left in chapter / Y min left
  in book' in header; live scroll-fraction update; only renders on
  chapter pages; integrated into HeaderStatus.vue

- Reset button in Identity Manage Profile page

- Wired in HomeController (word-count ViewBag) and _Layout.cshtml
  (window.* globals: CURRENT_PAGE_KEY, PAGE_WORD_COUNT, CHAPTER_WORD_COUNT,
  WORDS_BEFORE_PAGE, CHAPTER_START_WORDS, BOOK_WORD_COUNT)

- Tests: WordCountServiceTests (prose-only counting, prefix sums,
  chapter/book totals) and ReadingControllerTests (auth guards, WPM
  algorithm constants, book-stats endpoint)
… deserialization)

SiteMapping.PrimaryKey is not serialized in sitemap.json; after deserialization
it is null, causing ArgumentNullException in WordCountService constructor and
HomeController when calling Dictionary.ContainsKey(null).

Use mapping.Keys.FirstOrDefault() ?? mapping.PrimaryKey consistently, matching
the pattern already used in BookToolQueryService.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove unused chapter offset, credit only scroll progress made during the session, validate PageKey length, and apply outlier rules to anonymous sessions.
…r with page-lifecycle flushing

Split the tracker into activity-state, readingApi and orchestrator units,
measure progress against the content element, cap credited words by active
time, remove localStorage and the duplicated outlier rules, and improve the
header widget accessibility.
…dingActivity

Replace the activity audit table with a single UserReadingProfile migration,
move business rules out of the controller into IReadingProfileService and
ReadingSpeedPolicy bound to ReadingOptions, use an atomic upsert, validate
requests, and add per-user rate limiting.
…age stats

Single immutable PageWordStats snapshot (FrozenDictionary), built in
IHostedLifecycleService.StartingAsync; logs and isolates bad content files,
decodes HTML entities, counts inline code but not code blocks. Tests rewritten
around requirements.
Removes the reading ViewBag entries and window globals; csrf-token meta is
only rendered for signed-in users.
…t, personal data export, regenerated migration

- ReadingOptions: DefaultWpm/MaxCreditWpm/IdleThresholdSeconds/PeriodicFlushActiveSeconds/MaxActiveSecondsPerInterval with validation (ValidateOnStart + cross-field validator)
- drop unused Completed from the interval DTO; per-interval active-seconds cap from options
- replace MERGE/ON CONFLICT with ExecuteUpdate + insert-on-miss + duplicate-key retry
- READING_CONTEXT carries wpm and client constants
- include UserReadingProfile in personal data download
- restore Program.cs CRLF and empty OnModelCreating; regenerate migration
… rate limit, context parsing, personal data

Co-Authored-By: Claude <noreply@anthropic.com>
…details

Data-driven speed policy and options tests, HtmlAgilityPack-based context parsing, endpoint-metadata request size check, structured log assertions, CI-failing SQL Server gating, migration lookup by name, FK-violation and UpdatedAtUtc tests.
Neither endpoint is called by the client (it reads wpm from the session
response and READING_CONTEXT). Removes the dead response types,
IWordCountService.GetChapterWordCounts, the ReadingPublic rate-limit
policy and the tests that only covered the removed endpoints.
…d error keys

Move the interval-count and active-seconds limits into IValidatableObject
on the request DTOs (reading ReadingOptions from the validation service
provider) so MVC reports keys such as Intervals[1].ActiveSeconds. The
unknown-page-key error now carries the indexed key too. The controller no
longer needs ReadingOptions.
They need a real SQL Server and failed on every PR CI job that has none. They move to a follow-up that uses Testcontainers.
…rver

Self-provisions a pinned SQL Server container per test class instead of requiring
ESSENTIALCSHARP_TEST_SQLSERVER and a CI service container. Skips when no Linux
container runtime is available; fails on GitHub Linux CI so coverage cannot silently drop.
@BenjaminMichaelis
BenjaminMichaelis force-pushed the bmichaelis-reading-time-tracker-8ef branch from ca5e847 to d02419e Compare November 14, 2026 01:20

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant