[WIP] Added max_block and request_batch to pyfive.File and backend

[dwest77a]

Description

• Added max_block and request_batch as parameters to the pyfive.File declaration. This is relayed to the backend bulk_fsspec method which now uses the merge_requests utility from fsspec if max_block is defined. Request batch allows configuration via the fsspec cat_ranges method to restrict the number of simultaneous requests down from the default for remote connections of 1280.

Checklist

• This pull request has a descriptive title and labels
• This pull request has a minimal description (most was discussed in the issue, but a two-liner description is still desirable)
• Unit tests have been added (if codecov test fails)
• Any changed dependencies have been added or removed correctly (if need be)
• If you are working on the documentation, please ensure the current build passes
• All tests pass

[valeriupredoi]

thanks @dwest77a 🍺 Note that if you wish the PR to be truly draft, you can always convert it to a Draft PR. I approved the tests to run here, and if you don't mind, I'll also add you to PyActiveStorage 🍻

[dwest77a]

This relates to issue #257

[valeriupredoi]

@dwest77a the issue with pre-commit kicks in since you change the functional call and args:

131c139,141
<                 self._read_bulk_fsspec(fh, chunks, out, dtype)
---
>                 self._read_bulk_fsspec(
>                     fh, chunks, out, dtype, max_block=max_block, batch_size=batch_size
>                 )

posix is defined as attr only if not fh, and the same for _fh - not great in terms of coding, but also mypy is not catching that. I think you can safely ignore those, and to run the tests you can temporarily turn off the call to pre-commit in the Github Action workflow

[valeriupredoi]

@dwest77a I realized we've had quite a few of these [attr-defined] mypy barfs, that I had to manually allow - the way the code is written is a bit confusing to mypy, and having checked manually all is kosher, I told mypy to mind its own business in 702bf12 - note, though, it's still a good idea to run pre-commit install && pre-commit run -a when you done with code changes, so to catch any issues locally, and fix em 🍺

[codecov]

Codecov Report

❌ Patch coverage is 89.47368% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.46%. Comparing base (667c468) to head (702bf12).

Files with missing lines	Patch %	Lines
pyfive/h5d.py	85.71%	1 Missing and 1 partial ⚠️

❌ Your patch check has failed because the patch coverage (89.47%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #256      +/-   ##
==========================================
- Coverage   78.48%   78.46%   -0.02%     
==========================================
  Files          15       15              
  Lines        3416     3423       +7     
  Branches      546      547       +1     
==========================================
+ Hits         2681     2686       +5     
- Misses        593      594       +1     
- Partials      142      143       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[dwest77a]

@valeriupredoi thanks, I'll look into the test coverage now.

[valeriupredoi]

Good man Dan 🍺