fix(api): add bulk_upsert method and unit tests for opensearch BaseOSDB class

[vkuznet]

This PR fixes issue #401 by introducing the bulk_upsert API to BaseOSDB class. It is also complement by full set of unit tests for BaseOSDB class which were missing. The unit test introduces mock client and different classes for different use cases.

Please note: it is my first PR and I'm happy to adjust it according to requirements/guidelines of DiracX community.

[read-the-docs-community]

Documentation build overview

📚 diracx | 🛠️ Build #33616932 | 📁 Comparing fe52fc2 against latest (19cb080)

  🔍 Preview build  

2 files changed

± dev/tutorials/advanced-tutorial/index.html
± dev/tutorials/advanced-tutorial/router/index.html

[fstagni]

In the original issue (#401) it was mentioned a specific use case. Can you already verify if this fix covers that? You can add it directly in this PR.

[fstagni]

What is the rationale for this?

[vkuznet]

According to aysnc_bulk documentation

... Note that by default we raise a BulkIndexError when we encounter an error so options like stats_only only+ apply when raise_on_error is set to False.

So, it is question of policy regarding errors which I frankly don't know.

[fstagni]

In https://github.com/DIRACGrid/diracx/pull/1007/changes#r3734978424 @aldbr added a DocumentUpsertError exception which I believe could be used here too

[vkuznet]

@fstagni since this PR is not yet merged and I can't yet use in my PR. Do you want me to wait until it is merged or use a different exception?

[vkuznet]

In the original issue (#401) it was mentioned a specific use case. Can you already verify if this fix covers that? You can add it directly in this PR.

@fstagni regarding issue #401. I checked the main repository code and what is described in this issue is not there the issue is based on commit outside of any branch, most likely in fork outside of the repository

diracx/diracx-routers/src/diracx/routers/jobs/status.py

Lines 174 to 180 in 7c260cc

	# TODO: can we upsert to multiple documents?
	for job_id, p_updates_ in param_updates.items():
	if p_updates_:
	await job_parameters_db.upsert(
	int(job_id),
	p_updates_,
	)

). Therefore, before adding anything to this PR with usage of bulk insert I need to know if it is in fact required by code from main repository, see utils.py from main branch

[fstagni]

Hi, since #410 was created, the mentioned code has been moved and refactored, and can now be found in https://github.com/DIRACGrid/diracx/blob/main/diracx-logic/src/diracx/logic/jobs/status.py#L643. The issue anyway still holds, can you have a look?