Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Debug tests #932

Closed
Changes from 1 commit
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
0ea7f53
added response interceptor
ronnaksaxena Aug 18, 2023
4e93b1f
Base case: Query Live and Rollup data with no overlap
ronnaksaxena Aug 18, 2023
7fd394a
finished base case and added integ test
ronnaksaxena Aug 18, 2023
f023a39
added to response interceptor
ronnaksaxena Aug 22, 2023
4aa4202
can rewrite request to bucket pipeline
ronnaksaxena Aug 22, 2023
c3b5477
trying to rewrite aggregations in a helper function
ronnaksaxena Aug 23, 2023
5d20241
able to create new aggreations but getting shardIndex is not set error
ronnaksaxena Aug 23, 2023
743263b
Can find start and end times for rollup and live index
ronnaksaxena Aug 23, 2023
1f07f21
Handles overlap between 1 live index and 1 rollup index for sum aggre…
ronnaksaxena Aug 24, 2023
b4dbc26
added min max aggregations and fixed intersection time calculation
ronnaksaxena Aug 24, 2023
1ee93c6
changed variable name in computeAggregationsWithoutOverlap
ronnaksaxena Aug 24, 2023
4203a60
Added integ tests for nonoverlapping case
ronnaksaxena Aug 28, 2023
72e0aad
added avg and value count aggregation
ronnaksaxena Aug 29, 2023
9b977d2
fixed ktlint and integ test
ronnaksaxena Aug 29, 2023
8ed31e7
changed test and build workflow
ronnaksaxena Aug 29, 2023
f0d364f
added integ test for multiple live indices
ronnaksaxena Aug 30, 2023
b52fbdc
added test case for alias live indices
ronnaksaxena Aug 30, 2023
820dc24
cleaned up code and moved functions to utils file
ronnaksaxena Aug 30, 2023
ef7682f
fixed detekt errors
ronnaksaxena Aug 30, 2023
e9fa46c
fixed ktlint error :/'
ronnaksaxena Aug 30, 2023
c8d8790
Can run all integ tests at once now
ronnaksaxena Aug 31, 2023
1e3294f
removed DateTimeFromatter
ronnaksaxena Aug 31, 2023
1a7da76
fixed inf interceptor loop, need to pass RollupInterceptorIT.test rol…
ronnaksaxena Sep 5, 2023
b145ecf
passes all integ tests
ronnaksaxena Sep 5, 2023
a9c6369
fixed detekt errors
ronnaksaxena Sep 5, 2023
495ee4d
deleted rest test
ronnaksaxena Sep 6, 2023
4dd4096
added test back
ronnaksaxena Sep 6, 2023
d94fbbd
trying a new workflow build
ronnaksaxena Sep 6, 2023
aeeccae
Merge branch 'case2' of https://github.com/ronnaksaxena/index-managem…
ronnaksaxena Sep 6, 2023
4ef546b
added stars to worklow files
ronnaksaxena Sep 6, 2023
bf04e32
added unit test
ronnaksaxena Sep 7, 2023
d618e90
resolved some PR comments from Bowen
ronnaksaxena Sep 7, 2023
9c567ee
resolved more comments on my PR
ronnaksaxena Sep 7, 2023
79f68e5
removed stars from workflows
ronnaksaxena Sep 7, 2023
768a2bc
testing time of alias test case
ronnaksaxena Sep 7, 2023
fb86758
boring, took too long
ronnaksaxena Sep 8, 2023
17df156
commented out last 2 tests
ronnaksaxena Sep 8, 2023
e496c57
removed all response interceptor tests
ronnaksaxena Sep 8, 2023
8aa3f81
added one test back
ronnaksaxena Sep 8, 2023
016d161
fixed data stream integ tests
ronnaksaxena Sep 11, 2023
96603eb
commented out breaking tests
ronnaksaxena Sep 11, 2023
35ee9e6
added tests back
ronnaksaxena Sep 11, 2023
a817ddb
changes workflow for tests
ronnaksaxena Sep 12, 2023
f9b1f9c
added debug param to workflow
ronnaksaxena Sep 12, 2023
aaeaaec
added more debug logs
ronnaksaxena Sep 12, 2023
1578d23
removed countdown latch and wrapped calls in coroutine
ronnaksaxena Sep 12, 2023
581531d
resolved detekt error
ronnaksaxena Sep 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
passes all integ tests
Signed-off-by: Ronnak Saxena <[email protected]>
ronnaksaxena committed Sep 5, 2023
commit b145ecff7fd80723a04948a59ff387a17671a9e7
Original file line number Diff line number Diff line change
@@ -135,7 +135,7 @@ class ResponseInterceptor(
// Need to avoid infinite interceptor loop
val req = SearchRequest()
.source(searchSourceBuilder)
rollupIndices.forEach { req.indices(it) }
.indices(*rollupIndices)
var res: SearchResponse? = null
val latch = CountDownLatch(1)
client.search(
@@ -191,7 +191,7 @@ class ResponseInterceptor(
// Need to avoid infinite interceptor loop
val maxRolledDateRequest = SearchRequest()
.source(searchSourceBuilder)
rollupIndices.forEach { maxRolledDateRequest.indices(it) } // add all rollup indices to this request
.indices(*rollupIndices) // add all rollup indices to this request
var maxRolledDateResponse: SearchResponse? = null
var latch = CountDownLatch(1)
client.search(
@@ -221,7 +221,7 @@ class ResponseInterceptor(
If the response shard index is a live index, need to only compute minimum value of the current shard index
*/
if (isShardIndexRollup) {
liveIndices.forEach { minLiveDateRequest.indices(it) }
minLiveDateRequest.indices(*liveIndices)
} else { // shard index is live index
minLiveDateRequest.indices(shardRequestIndex)
}
Original file line number Diff line number Diff line change
@@ -1045,21 +1045,6 @@ class RollupInterceptorIT : RollupRestTestCase() {
}
}
""".trimIndent()
// Search 1 non-rollup index and 1 rollup
val searchResult1 = client().makeRequest("POST", "/$sourceIndex2,$targetIndex2/_search", emptyMap(), StringEntity(req, ContentType.APPLICATION_JSON))
assertTrue(searchResult1.restStatus() == RestStatus.OK)
val failures = extractFailuresFromSearchResponse(searchResult1)
assertNotNull(failures)
assertEquals(1, failures?.size)
assertEquals(
"Searching multiple indices where one is rollup and other is not, didn't return failure",
"illegal_argument_exception", failures?.get(0)?.get("type") ?: "Didn't find failure type in search response"

)
assertEquals(
"Searching multiple indices where one is rollup and other is not, didn't return failure",
"Not all indices have rollup job", failures?.get(0)?.get("reason") ?: "Didn't find failure reason in search response"
)

// Search 2 rollups with different mappings
try {