Skip to content

Fix Semantic Query Rewrite Interception Drops Boosts #129282

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
ccd64ae
fix boosting for knn
Samiul-TheSoccerFan Jun 11, 2025
9338cd5
Fixing for match query
Samiul-TheSoccerFan Jun 11, 2025
370931d
fixing for match subquery
Samiul-TheSoccerFan Jun 11, 2025
b85abda
fix for sparse vector query boost
Samiul-TheSoccerFan Jun 11, 2025
5db2686
fix linting issues
Samiul-TheSoccerFan Jun 11, 2025
2ce691e
Update docs/changelog/129282.yaml
Samiul-TheSoccerFan Jun 11, 2025
4100200
update changelog
Samiul-TheSoccerFan Jun 11, 2025
3406ae1
Copy constructor with match query
Samiul-TheSoccerFan Jun 12, 2025
d07952a
util function to create sparseVectorBuilder for sparse query
Samiul-TheSoccerFan Jun 12, 2025
f133632
util function for knn query to support boost
Samiul-TheSoccerFan Jun 12, 2025
a9048f0
adding unit tests for all intercepted query terms
Samiul-TheSoccerFan Jun 12, 2025
5a1dab9
Adding yaml test for match,sparse, and knn
Samiul-TheSoccerFan Jun 13, 2025
6cef441
Adding queryname support for nested query
Samiul-TheSoccerFan Jun 13, 2025
faa35ea
fix code styles
Samiul-TheSoccerFan Jun 13, 2025
675fb22
merge from main
Samiul-TheSoccerFan Jun 13, 2025
13e791e
Fix failed yaml tests
Samiul-TheSoccerFan Jun 13, 2025
3a5a30f
Update docs/changelog/129282.yaml
Samiul-TheSoccerFan Jun 13, 2025
016e448
update yaml tests to expand test scenarios
Samiul-TheSoccerFan Jun 16, 2025
70b228e
Merge branch 'main' into fix-semantic-query-rewrite-boost-issue
elasticmachine Jun 16, 2025
d5e7caa
Merge branch 'main' into fix-semantic-query-rewrite-boost-issue
elasticmachine Jun 16, 2025
efcf9c4
Updating knn to copy constructor
Samiul-TheSoccerFan Jun 25, 2025
00eb6ad
merge from main
Samiul-TheSoccerFan Jun 25, 2025
f449299
adding yaml tests for multiple indices
Samiul-TheSoccerFan Jun 27, 2025
6db0abf
refactoring match query to adjust boost and queryname and move to cop…
Samiul-TheSoccerFan Jun 27, 2025
daf2cb4
refactoring sparse query to adjust boost and queryname and move to co…
Samiul-TheSoccerFan Jun 27, 2025
b88b077
[CI] Auto commit changes from spotless
elasticsearchmachine Jun 27, 2025
9e725cb
Refactor sparse vector to adjust boost and queryname in the top level
Samiul-TheSoccerFan Jul 2, 2025
651ee2b
Refactor knn vector to adjust boost and queryname in the top level
Samiul-TheSoccerFan Jul 2, 2025
a356b44
merge from main
Samiul-TheSoccerFan Jul 2, 2025
71eac8d
fix knn combined query
Samiul-TheSoccerFan Jul 2, 2025
d71bf2c
fix unit tests
Samiul-TheSoccerFan Jul 2, 2025
675463c
fix lint issues
Samiul-TheSoccerFan Jul 2, 2025
201d27c
remove unused code
Samiul-TheSoccerFan Jul 2, 2025
daf2f6e
Update inference feature name
Samiul-TheSoccerFan Jul 3, 2025
2521b48
Remove double boosting issue from match
Samiul-TheSoccerFan Jul 3, 2025
61f9445
Fix double boosting in match test yaml file
Samiul-TheSoccerFan Jul 3, 2025
f4cadaa
move to bool level for match semantic boost
Samiul-TheSoccerFan Jul 3, 2025
08909de
fix double boosting for sparse vector
Samiul-TheSoccerFan Jul 3, 2025
37bfc43
fix double boosting for sparse vector in yaml test
Samiul-TheSoccerFan Jul 3, 2025
fa5cfe7
fix knn combined query
Samiul-TheSoccerFan Jul 3, 2025
0640631
fix knn combined query
Samiul-TheSoccerFan Jul 3, 2025
404efcf
fix sparse combined query
Samiul-TheSoccerFan Jul 3, 2025
f73285d
fix knn yaml test for combined query
Samiul-TheSoccerFan Jul 3, 2025
96f5aa6
refactoring unit tests
Samiul-TheSoccerFan Jul 4, 2025
3065e5b
linting
Samiul-TheSoccerFan Jul 4, 2025
828d8c2
fix match query unit test
Samiul-TheSoccerFan Jul 4, 2025
6a2e0a5
Merge branch 'main' into fix-semantic-query-rewrite-boost-issue
elasticmachine Jul 4, 2025
bde54df
Merge branch 'main' into fix-semantic-query-rewrite-boost-issue
elasticmachine Jul 7, 2025
d08dbdd
adding copy constructor for match query
Samiul-TheSoccerFan Jul 8, 2025
fa955c3
Merge branch 'main' into fix-semantic-query-rewrite-boost-issue
elasticmachine Jul 8, 2025
916b1cc
refactor copy match builder to intercepter
Samiul-TheSoccerFan Jul 8, 2025
d9ef867
Merge branch 'main' into fix-semantic-query-rewrite-boost-issue
elasticmachine Jul 8, 2025
cde55d1
resolve conflicts from main
Samiul-TheSoccerFan Jul 9, 2025
8ddda3c
[CI] Auto commit changes from spotless
elasticsearchmachine Jul 9, 2025
873efdb
Merge branch 'main' into fix-semantic-query-rewrite-boost-issue
elasticmachine Jul 9, 2025
44b8aa9
Merge branch 'main' into fix-semantic-query-rewrite-boost-issue
elasticmachine Jul 10, 2025
104f16b
Merge branch 'main' into fix-semantic-query-rewrite-boost-issue
elasticmachine Jul 11, 2025
2a96d52
Merge branch 'main' into fix-semantic-query-rewrite-boost-issue
elasticmachine Jul 11, 2025
5dcfc1b
fix unit tests
Samiul-TheSoccerFan Jul 11, 2025
469f598
update yaml tests
Samiul-TheSoccerFan Jul 11, 2025
375ae36
fix match yaml test
Samiul-TheSoccerFan Jul 11, 2025
c81f184
Merge branch 'main' into fix-semantic-query-rewrite-boost-issue
elasticmachine Jul 14, 2025
394f43a
Merge branch 'main' into fix-semantic-query-rewrite-boost-issue
elasticmachine Jul 14, 2025
768b8f6
fix yaml tests with 4 digits error margin
Samiul-TheSoccerFan Jul 16, 2025
98cba31
unit tests are now more randomized
Samiul-TheSoccerFan Jul 16, 2025
6c6e7cf
Merge branch 'main' into fix-semantic-query-rewrite-boost-issue
elasticmachine Jul 16, 2025
4b0a6fe
Merge branch 'main' into fix-semantic-query-rewrite-boost-issue
elasticmachine Jul 16, 2025
c233ee9
Merge branch 'main' into fix-semantic-query-rewrite-boost-issue
elasticmachine Jul 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ public void cleanup() {
}

public void testKnnQueryWithVectorBuilderIsInterceptedAndRewritten() throws IOException {
float boost = randomFloat() * 5;
float boost = randomFloatBetween(1, 10, true);
String queryName = randomAlphaOfLength(5);
Map<String, InferenceFieldMetadata> inferenceFields = Map.of(
FIELD_NAME,
Expand All @@ -69,13 +69,17 @@ public void testKnnQueryWithVectorBuilderIsInterceptedAndRewritten() throws IOEx
}

public void testKnnWithQueryBuilderWithoutInferenceIdIsInterceptedAndRewritten() throws IOException {
float boost = randomFloatBetween(1, 10, true);
String queryName = randomAlphaOfLength(5);
Map<String, InferenceFieldMetadata> inferenceFields = Map.of(
FIELD_NAME,
new InferenceFieldMetadata(index.getName(), INFERENCE_ID, new String[] { FIELD_NAME }, null)
);
QueryRewriteContext context = createQueryRewriteContext(inferenceFields);
QueryVectorBuilder queryVectorBuilder = new TextEmbeddingQueryVectorBuilder(null, QUERY);
KnnVectorQueryBuilder original = new KnnVectorQueryBuilder(FIELD_NAME, queryVectorBuilder, 10, 100, null);
original.boost(boost);
original.queryName(queryName);
testRewrittenInferenceQuery(context, original);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ public void testBoostAndQueryNameInMatchQueryRewrite() throws IOException {
rewritten instanceof InterceptedQueryBuilderWrapper
);
InterceptedQueryBuilderWrapper intercepted = (InterceptedQueryBuilderWrapper) rewritten;
assertEquals(5.0, intercepted.boost(), 0.0f);
assertEquals(BOOST, intercepted.boost(), 0.0f);
assertEquals(QUERY_NAME, intercepted.queryName());
assertTrue(intercepted.queryBuilder instanceof SemanticQueryBuilder);
SemanticQueryBuilder semanticQueryBuilder = (SemanticQueryBuilder) intercepted.queryBuilder;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ public void cleanup() {
}

public void testSparseVectorQueryOnInferenceFieldIsInterceptedAndRewritten() throws IOException {
float boost = randomFloat() * 5;
float boost = randomFloatBetween(1, 10, true);
String queryName = randomAlphaOfLength(5);
Map<String, InferenceFieldMetadata> inferenceFields = Map.of(
FIELD_NAME,
Expand All @@ -66,12 +66,16 @@ public void testSparseVectorQueryOnInferenceFieldIsInterceptedAndRewritten() thr
}

public void testSparseVectorQueryOnInferenceFieldWithoutInferenceIdIsInterceptedAndRewritten() throws IOException {
float boost = randomFloatBetween(1, 10, true);
String queryName = randomAlphaOfLength(5);
Map<String, InferenceFieldMetadata> inferenceFields = Map.of(
FIELD_NAME,
new InferenceFieldMetadata(index.getName(), "inferenceId", new String[] { FIELD_NAME }, null)
);
QueryRewriteContext context = createQueryRewriteContext(inferenceFields);
QueryBuilder original = new SparseVectorQueryBuilder(FIELD_NAME, null, QUERY);
original.boost(boost);
original.queryName(queryName);
testRewrittenInferenceQuery(context, original);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -322,7 +322,7 @@ setup:

- match: { hits.total.value: 1 }
- match: { hits.hits.0._id: "doc_1" }
- close_to: { hits.hits.0._score: { value: 5.700229E18, error: 1e15 } }
- close_to: { hits.hits.0._score: { value: 5.700229E18, error: 1e-7 } }
- not_exists: hits.hits.0.matched_queries

- do:
Expand All @@ -341,7 +341,7 @@ setup:

- match: { hits.total.value: 1 }
- match: { hits.hits.0._id: "doc_1" }
- close_to: { hits.hits.0._score: { value: 2.8501142E19, error: 1e15 } }
- close_to: { hits.hits.0._score: { value: 2.8501142E19, error: 1e-7 } }
- match: { hits.hits.0.matched_queries: [ "i-like-naming-my-queries" ] }

---
Expand Down Expand Up @@ -406,7 +406,7 @@ setup:
- match: { hits.total.value: 2 }
- match: { hits.hits.0._id: "doc_1" }
- match: { hits.hits.1._id: "doc_2" }
- close_to: { hits.hits.0._score: { value: 1.1140361E19, error: 1e15 } }
- close_to: { hits.hits.0._score: { value: 1.1140361E19, error: 1e-7 } }
- not_exists: hits.hits.0.matched_queries
- close_to: { hits.hits.1._score: { value: 0.2876821, error: 1e-7 } }
- not_exists: hits.hits.1.matched_queries
Expand All @@ -428,7 +428,7 @@ setup:
- match: { hits.total.value: 2 }
- match: { hits.hits.0._id: "doc_1" }
- match: { hits.hits.1._id: "doc_2" }
- close_to: { hits.hits.0._score: { value: 5.5701804E19, error: 1e15 } }
- close_to: { hits.hits.0._score: { value: 5.5701804E19, error: 1e-7 } }
- match: { hits.hits.0.matched_queries: [ "i-like-naming-my-queries" ] }
- close_to: { hits.hits.1._score: { value: 1.4384103, error: 1e-7 } }
- match: { hits.hits.1.matched_queries: [ "i-like-naming-my-queries" ] }
Original file line number Diff line number Diff line change
Expand Up @@ -291,7 +291,7 @@ setup:

- match: { hits.total.value: 1 }
- match: { hits.hits.0._id: "doc_1" }
- close_to: { hits.hits.0._score: { value: 5.700229E18, error: 1e15 } }
- close_to: { hits.hits.0._score: { value: 5.700229E18, error: 1e-7 } }
- not_exists: hits.hits.0.matched_queries

- do:
Expand All @@ -310,7 +310,7 @@ setup:

- match: { hits.total.value: 1 }
- match: { hits.hits.0._id: "doc_1" }
- close_to: { hits.hits.0._score: { value: 2.8501142E19, error: 1e15 } }
- close_to: { hits.hits.0._score: { value: 2.8501142E19, error: 1e-7 } }
- match: { hits.hits.0.matched_queries: [ "i-like-naming-my-queries" ] }

---
Expand Down Expand Up @@ -376,7 +376,7 @@ setup:
- match: { hits.total.value: 2 }
- match: { hits.hits.0._id: "doc_1" }
- match: { hits.hits.1._id: "doc_2" }
- close_to: { hits.hits.0._score: { value: 5.700229E18, error: 1e15 } }
- close_to: { hits.hits.0._score: { value: 5.700229E18, error: 1e-7 } }
- not_exists: hits.hits.0.matched_queries
- close_to: { hits.hits.1._score: { value: 1.3455845E10, error: 1e-7 } }
- not_exists: hits.hits.1.matched_queries
Expand All @@ -399,7 +399,7 @@ setup:
- match: { hits.total.value: 2 }
- match: { hits.hits.0._id: "doc_1" }
- match: { hits.hits.1._id: "doc_2" }
- close_to: { hits.hits.0._score: { value: 2.8501142E19, error: 1e15 } }
- close_to: { hits.hits.0._score: { value: 2.8501142E19, error: 1e-7 } }
- match: { hits.hits.0.matched_queries: [ "i-like-naming-my-queries" ] }
- close_to: { hits.hits.1._score: { value: 6.7279225E10, error: 1e-7 } }
- match: { hits.hits.1.matched_queries: [ "i-like-naming-my-queries" ] }
Original file line number Diff line number Diff line change
Expand Up @@ -452,7 +452,7 @@ setup:

- match: { hits.total.value: 1 }
- match: { hits.hits.0._id: "doc_1" }
- close_to: { hits.hits.0._score: { value: 0.9984111, error: 1e15 } }
- close_to: { hits.hits.0._score: { value: 0.9981499, error: 1e-7 } }
- not_exists: hits.hits.0.matched_queries

- do:
Expand Down
Loading