Repository navigation
Memory regression tests - #275
alan-george-lk wants to merge 14 commits into
Conversation
2a21a8a to
d638cca
Compare
…ature-memory-regression-tests
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
Note
Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.
🔍 Devin Review: 2 flags
Not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
stephen-derosa
left a comment
stephen-derosa
left a comment
There was a problem hiding this comment.
overall looks great, thanks for adding these!
| matrix.e2e-testing && | ||
| (inputs.integration_repeat > 0 || | ||
| inputs.run_stress_tests || | ||
| (matrix.name == 'ubuntu-24.04-x64' && inputs.build_type == 'release')) |
There was a problem hiding this comment.
we are only running these on ubuntu x64 release builds?
| After building, run tests using `ctest` or invoke the binaries directly: | ||
| ```bash | ||
| # Linux/macOS | ||
| <build-folder>/bin/<test binaries> |
There was a problem hiding this comment.
since there are specific examples below do we need the <build-folder>/bin/<test binaries> ?
|
|
||
| Each scenario reports process RSS and thread count at cycle 0, after warmup | ||
| cycle 20, and after final cycle 100. Checkpoint sampling waits for the thread | ||
| count to remain stable for 500 ms so short-lived teardown work does not cause a |
| // encoder so the scenario can run in CI. | ||
| void runRoomServerDeleteCycle(const RoomCredentials& credentials) { | ||
| if (std::string_view(credentials.url) != kLocalTestLiveKitUrl) { | ||
| throw std::runtime_error("room-server-delete requires LIVEKIT_URL=ws://localhost:7880"); |
There was a problem hiding this comment.
any plan to expand this to remote servers?
| delegate.requireDisconnect(livekit::DisconnectReason::ClientInitiated); | ||
| } | ||
|
|
||
| // Exercises data-track publication, payload enqueueing, explicit unpublication, |
There was a problem hiding this comment.
these doc strings are great
| return output.str(); | ||
| } | ||
|
|
||
| double kibToMib(std::uint64_t kib) { return static_cast<double>(kib) / 1024.0; } |
There was a problem hiding this comment.
for most of these helpers/utils.
Also consider moving to a test_utils or something similar. I get we dont want to depend on actual lib code, but i think we have small converters like this all over tests. If you agree, maybe just make a ticket -- no need to cram in here
2 participants
This PR introduces a new test binary called
memory_testerfor Linux, which has the following features:CI is also updated to run this tester and provide a nice markdown table report of the results, and fail jobs that introduce changes that fail this.
Documentation is also updated around testing in general, as it was a bit out of date.
Results
Here are two FFI release results with the following params:
Prior release
livekit-ffi/v0.12.81results:Current FFI release results: