You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Adds a hand-written custom test suite that raises deepgram package coverage to the ~95% target (branch coverage enabled), wires coverage reporting into CI with an automatic PR comment, and configures coverage to focus on hand-maintainable logic rather than Fern-generated boilerplate. Also freezes the new tests and coverage config in .fernignore so they survive the next SDK regeneration.
What changed
CI (.github/workflows/ci.yml)
Runs on pull_request in addition to push.
Test step now runs under coverage: pytest --cov=deepgram --cov-branch --cov-report=xml --cov-report=term-missing.
Generates a coverage summary (CodeCoverageSummary) and posts/updates a sticky PR comment with the results (Python 3.13 matrix leg only); also writes it to the job summary.
Grants the test job pull-requests: write so it can comment.
Coverage config (pyproject.toml)
New [tool.coverage.run] / [tool.coverage.report]: branch coverage on, scoped to deepgram, with pure-generated code excluded (types/, requests/, init.py, version.py, unused core/http_sse/) so the metric reflects code that actually carries logic.
New tests (tests/custom/, ~170 test functions, many parametrized) test_http_endpoints_coverage.py — table-driven sweep of every REST endpoint (success + 400/403/non-JSON error branches, sync + async). test_http_retry_coverage.py — retry/backoff behavior. test_websocket_streaming_coverage.py — websocket streaming clients. test_speak_v2_coverage.py — Speak V2 (Flux TTS): the batch REST audio.generate streaming endpoint, connect header/handshake-failure branches, and the socket-client iteration/send/recv paths. Raises speak.v2 from 44% → 97%. test_core_utilities_coverage.py, test_branch_coverage_95.py, test_misc_coverage.py, test_more_branches_coverage.py — core utilities and remaining branch gaps.
Regen safety / housekeeping .fernignore — freezes all 8 new custom test files and pyproject.toml (Fern regenerates it, so the coverage config would otherwise be stripped). Documented in AGENTS.md. .gitignore — ignores the .coverage artifact and htmlcov/.
Merged latest main (7.6.0 regen) to keep the branch current.
Testing
New tests use respx (HTTP) and in-process websocket fakes — no network / no WireMock dependency.
All Speak V2 coverage tests pass locally (50 across the v2 suite); CI runs the full suite under coverage on Python 3.10–3.13.
Hi @mgama-deepgram this is great, however some of the files you've modified are generated files based on the API specs. I'd rather we didn't freeze files unless absolutely unavoidable.
@GregHolmes I removed all the changes that were done to files under src/deepgram/core and left only the new tests under tests/custom.
The code coverage is still really high, so that did not affect it that much.
@GregHolmes I removed all the changes that were done to files under src/deepgram/core and left only the new tests under tests/custom. The code coverage is still really high, so that did not affect it that much.
@mgama-deepgram would you not want your other test files in .fernignore? otherwise when regenerations done, it'll remove them.
@GregHolmes I removed all the changes that were done to files under src/deepgram/core and left only the new tests under tests/custom. The code coverage is still really high, so that did not affect it that much.
@mgama-deepgram would you not want your other test files in .fernignore? otherwise when regenerations done, it'll remove them.
Thanks for calling this out @GregHolmes I just added the tests files to the .fernignore file. I am new to use these type of files and regenerations, thanks for explaining.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
mgama-deepgram
changed the title
Increase unit test coverage (~95%) + coverage in CI
Adds a hand-written custom test suite that raises deepgram package coverage to the ~95% target (branch coverage enabled), wires coverage reporting into CI with an automatic PR comment, and configures coverage to focus on hand-maintainable logic rather than Fern-generated boilerplate. Also freezes the new tests and coverage config in .fernignore so they survive the next SDK regeneration.
What changed
CI (
.github/workflows/ci.yml)Runs on pull_request in addition to push.
Test step now runs under coverage: pytest --cov=deepgram --cov-branch --cov-report=xml --cov-report=term-missing.
Generates a coverage summary (CodeCoverageSummary) and posts/updates a sticky PR comment with the results (Python 3.13 matrix leg only); also writes it to the job summary.
Grants the test job pull-requests: write so it can comment.
Coverage config (
pyproject.toml)New [tool.coverage.run] / [tool.coverage.report]: branch coverage on, scoped to deepgram, with pure-generated code excluded (types/, requests/, init.py, version.py, unused core/http_sse/) so the metric reflects code that actually carries logic.
New tests (
tests/custom/, ~170 test functions, many parametrized)test_http_endpoints_coverage.py— table-driven sweep of every REST endpoint (success + 400/403/non-JSON error branches, sync + async).test_http_retry_coverage.py— retry/backoff behavior.test_websocket_streaming_coverage.py— websocket streaming clients.test_speak_v2_coverage.py— Speak V2 (Flux TTS): the batch REST audio.generate streaming endpoint, connect header/handshake-failure branches, and the socket-client iteration/send/recv paths. Raises speak.v2 from 44% → 97%.test_core_utilities_coverage.py, test_branch_coverage_95.py, test_misc_coverage.py,test_more_branches_coverage.py— core utilities and remaining branch gaps.Regen safety / housekeeping
.fernignore— freezes all 8 new custom test files and pyproject.toml (Fern regenerates it, so the coverage config would otherwise be stripped). Documented in AGENTS.md..gitignore— ignores the .coverage artifact and htmlcov/.Merged latest main (7.6.0 regen) to keep the branch current.
Testing
New tests use respx (HTTP) and in-process websocket fakes — no network / no WireMock dependency.
All Speak V2 coverage tests pass locally (50 across the v2 suite); CI runs the full suite under coverage on Python 3.10–3.13.