fix(search): report rejected documents when indexing to OpenSearch#3142
fix(search): report rejected documents when indexing to OpenSearch#3142dschmidt wants to merge 3 commits into
Conversation
ec1451b to
28e148c
Compare
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 0 |
| Duplication | 7 |
🟢 Coverage 76.47% diff coverage · +0.01% coverage variation
Metric Results Coverage variation ✅ +0.01% coverage variation (-1.00%) Diff coverage ✅ 76.47% diff coverage Coverage variation details
Coverable lines Covered lines Coverage Common ancestor commit (85e63ca) 82473 19129 23.19% Head commit (87dd68a) 82486 (+13) 19138 (+9) 23.20% (+0.01%) Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch:
<coverage of head commit> - <coverage of common ancestor commit>Diff coverage details
Coverable lines Covered lines Diff coverage Pull request (#3142) 17 13 76.47% Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified:
<covered lines added or modified>/<coverable lines added or modified> * 100%
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
140495b to
26a44bb
Compare
26a44bb to
9beb00a
Compare
| return fmt.Errorf("failed to delete by query: %w", err) | ||
| case len(resp.Failures) != 0: | ||
| return fmt.Errorf("failed to delete by query, failures: %v", resp.Failures) | ||
| return fmt.Errorf("failed to delete by query, failures: %s", resp.Failures) |
There was a problem hiding this comment.
resp.Failures is a []json.RawMessage, and json.RawMessage is a []byte without a String() method, so %v rendered the failures as byte arrays:
[[123 34 105 110 100 101 120 34 58 34 111 112 101 110 ...]]
%s prints what the API actually returned:
[{"index":"opencloud-demo-dbq","id":"1$1!3","cause":{"type":"cluster_block_exception","reason":"index [opencloud-demo-dbq] blocked by: [FORBIDDEN/8/index write (api)];"},"status":403}]
1 participant
Batch.Pushdiscarded the bulk response. The bulk API answers 200 even when it rejects individual items, so a rejected document went missing from the index without a trace.Purgealready checksresp.FailuresforDeleteByQuery, this does the same forBulk.While at it: those
DeleteByQueryfailures were printed as byte arrays,resp.Failuresis a[]json.RawMessage.This only makes the loss visible, it does not prevent it. The event is acked either way and nothing retries.