Lookup submision IDs by strategic metadata

[jerbaroo]

This PR adds the lookup_submission_ids_by_strategic_metadata method to ProducerClient, which will return the submission IDs of in-progress subsmissions that match ALL of the provided strategic metadata key-value pairs.

    def lookup_submission_ids_by_strategic_metadata(
        self, strategic_metadata: dict[str, int]
    ) -> list[SubmissionId]:
        """Attempts to find in-progress submissions where the strategic metadata
        of that submission includes all of the key-value pairs of the given
        'strategic_metadata'. A matching submission must include all of the
        given key-value pairs, but it may also contain other key-value pairs.

        Raises:
        - `TooManyMatchingSubmissionsError` if the lookup matches more
          submissions than the server's configured maximum. Narrow the query
          with more specific strategic metadata.
        - `InternalProducerClientError` if there is a low-level internal error.

        """

[ReinierMaas]

Open question shouldn't an empty StrategicMetadataMap return all submissions instead of raising an error on that?
Also this is an easy way to DDoS OpsQueue if the number of returned submissions is large.

[jerbaroo]

Open question shouldn't an empty StrategicMetadataMap return all submissions instead of raising an error on that?

It depends on the API contract we want to provide. An argument in favour of throwing an error is that if a caller is using an API to "search for submissions matching some strategic metadata" but they haven't provided any strategic metadata, then they possibly have an issue in their code that they should be alerted to. Also, the SQLite query will need a special case to handle this, more code for something we don't have evidence for that it is a usecase we want to support.

[ReinierMaas]

I have a few comment and would like to point out that for the other queries we also assert the query plan. This allows us to spot potentially very bad execution behaviour.

[jerbaroo]

I have a few comment and would like to point out that for the other queries we also assert the query plan. This allows us to spot potentially very bad execution behaviour.

Query plan asserted in c6c9f04

[ReinierMaas]

I have a few code simplifications and one API generalisation.

[ReinierMaas]

I have some questions about the query we build-up. I don't think I fully follow without the recursive example working. And I expect that I hit on a small simplification.

[ReinierMaas]

LGTM! I was still thinking about a last small improvement to simplify the query generation code but I am not married to that.

[ReinierMaas]

The linter still failed on CI:

$ git diff
diff --git a/libs/opsqueue_python/tests/test_roundtrip.py b/libs/opsqueue_python/tests/test_roundtrip.py
index 2251c32..1f6ac50 100644
--- a/libs/opsqueue_python/tests/test_roundtrip.py
+++ b/libs/opsqueue_python/tests/test_roundtrip.py
@@ -582,13 +582,19 @@ def test_lookup_too_many_submission_ids_by_strategic_metadata() -> None:
         for _ in range(max_ + 1):
             assert (
                 inserted
-                == producer_client.lookup_submission_ids_by_strategic_metadata(strategic_metadata)
+                == producer_client.lookup_submission_ids_by_strategic_metadata(
+                    strategic_metadata
+                )
             )
             inserted.append(
-                producer_client.insert_submission([1], chunk_size=1, strategic_metadata=strategic_metadata)
+                producer_client.insert_submission(
+                    [1], chunk_size=1, strategic_metadata=strategic_metadata
+                )
             )
         with pytest.raises(TooManyMatchingSubmissionsError) as exc:
             assert len(inserted) == max_ + 1
-            producer_client.lookup_submission_ids_by_strategic_metadata(strategic_metadata)
+            producer_client.lookup_submission_ids_by_strategic_metadata(
+                strategic_metadata
+            )
         assert exc.type is TooManyMatchingSubmissionsError
         assert exc.value.max_submissions == max_