Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deployment issue #818

Open
wants to merge 29 commits into
base: client-403-issue
Choose a base branch
from
Open

Deployment issue #818

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
e0b619e
reds_reverting_changes (#817)
mukeshsbbv Feb 27, 2024
fa5d3c9
branch admin -prepare release emodb-6.5.83
Feb 27, 2024
240a93d
branch admin -prepare for next development iteration
Feb 27, 2024
3a17878
branch admin -prepare release emodb-6.5.84
Mar 4, 2024
e59d42b
updated to CLTWO
mukeshsbbv Mar 5, 2024
9ba296c
5xx Exceptions: Updating CL_LOCAL_QUORUM to CL_TWO (#820)
mukeshsbbv Mar 5, 2024
ed01731
Update Snapshot Version for Read Consistency Changes (#821)
mukeshsbbv Mar 5, 2024
d8ca339
branch admin -prepare release emodb-6.5.98
Mar 5, 2024
e8a8110
branch admin -prepare for next development iteration
Mar 5, 2024
d3bacb9
Reverted CL_Local_quorum changes (#822)
VERMARAVI-BV Mar 13, 2024
68545a1
branch admin -prepare release emodb-6.5.109
Mar 13, 2024
421fb20
branch admin -prepare for next development iteration
Mar 13, 2024
c5895a7
Pd 241513 (#824)
anandujayan May 22, 2024
f10a29b
Snapshot version updated to 136 (#825)
anandujayan May 22, 2024
71f294a
branch admin -prepare release emodb-6.5.136
May 22, 2024
16992be
branch admin -prepare for next development iteration
May 22, 2024
3538bbf
SQS integration in blob and metrics for queue service migration prere…
rishuyadavbv Aug 19, 2024
032af2b
PD-257019 Changes for emodb stabilization, kafka integration and step…
rishuyadavbv Oct 28, 2024
ecda251
branch admin -prepare release emodb-6.5.204
Oct 28, 2024
a344557
branch admin -prepare for next development iteration
Oct 28, 2024
387db36
Databus Throttling Changes (#847)
nabajyotiDash-hub Nov 6, 2024
e5e5bd2
branch admin -prepare release emodb-6.5.209
Nov 6, 2024
31cea14
branch admin -prepare for next development iteration
Nov 6, 2024
bfdc7d9
PD-257039 : Fix Serialisation issue in databus (#850)
nabajyotiDash-hub Nov 7, 2024
04b4809
branch admin -prepare release emodb-6.5.211
Nov 7, 2024
9e73faa
branch admin -prepare for next development iteration
Nov 7, 2024
dfa42aa
Hotfix for EU databus throttler (#851)
nabajyotiDash-hub Nov 7, 2024
bf90025
branch admin -prepare release emodb-6.5.214
Nov 7, 2024
8184514
branch admin -prepare for next development iteration
Nov 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Empty file added abc.txt
View file
Open in desktop
Empty file.
2 changes: 1 addition & 1 deletion auth/auth-client/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion auth/auth-core/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion auth/auth-store/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion auth/auth-util/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
<parent>
<artifactId>emodb</artifactId>
<groupId>com.bazaarvoice.emodb</groupId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion blob-api/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion blob-clients/blob-client-common/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion blob-clients/blob-client-jersey2/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion blob-clients/blob-client/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion blob/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion blob/src/main/java/com/bazaarvoice/emodb/blob/core/DefaultBlobStore.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -474,4 +474,4 @@ private static void checkLegalBlobId(String blobId) {
"Blob IDs must be ASCII strings between 1 and 255 characters in length. " +
"Whitespace, ISO control characters and certain punctuation characters that aren't generally allowed in file names are excluded.");
}
}
}
2 changes: 1 addition & 1 deletion cachemgr/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion common/api/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion common/astyanax/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion common/client-jax-rs-2/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion common/client-jersey2/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion common/client/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion common/dropwizard/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion common/jersey-client/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion common/json/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion common/stash/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion common/uuid/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion common/zookeeper/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion databus-api/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../parent/pom.xml</relativePath>
</parent>

Expand Down
4 changes: 4 additions & 0 deletions databus-api/src/main/java/com/bazaarvoice/emodb/databus/api/Databus.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,10 @@ Subscription getSubscription(String subscription)
*/
long getEventCountUpTo(String subscription, long limit);

default long getMasterCount(){
return 0;
};

/** Counts the number of events with outstanding claims that cause the events to be skipped by {@link #poll}. */
long getClaimCount(String subscription);

Expand Down
2 changes: 1 addition & 1 deletion databus-client-common/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion databus-client-jersey2/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion databus-client/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion databus/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../parent/pom.xml</relativePath>
</parent>

Expand Down
5 changes: 5 additions & 0 deletions databus/src/main/java/com/bazaarvoice/emodb/databus/core/DatabusFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,11 @@ public long getEventCountUpTo(String subscription, long limit) {
return _ownerAwareDatabus.getEventCountUpTo(ownerId, subscription, limit);
}

@Override
public long getMasterCount() {
return _ownerAwareDatabus.getMasterEventCountUncached(ownerId);
}

@Override
public long getClaimCount(String subscription) {
return _ownerAwareDatabus.getClaimCount(ownerId, subscription);
Expand Down
12 changes: 12 additions & 0 deletions databus/src/main/java/com/bazaarvoice/emodb/databus/core/DefaultDatabus.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -411,6 +411,18 @@ public long getEventCount(String ownerId, String subscription) {
return getEventCountUpTo(ownerId, subscription, Long.MAX_VALUE);
}

@Override
public long getMasterEventCountUncached(String ownerId) {
_log.info("Inside getMasterEventCountUncached with _masterFanoutChannels length {}", _masterFanoutChannels.size());
long size = 0;
for(String channel : _masterFanoutChannels) {
size += _eventStore.getSizeEstimate(channel, Long.MAX_VALUE);
_log.info("From channel size {} {}:", channel, size);
}
return size;
}


@Override
public long getEventCountUpTo(String ownerId, String subscription, long limit) {
checkSubscriptionOwner(ownerId, subscription);
Expand Down
3 changes: 3 additions & 0 deletions databus/src/main/java/com/bazaarvoice/emodb/databus/core/OwnerAwareDatabus.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ Subscription getSubscription(String ownerId, String subscription)
long getEventCount(String ownerId, String subscription)
throws UnauthorizedSubscriptionException;

public long getMasterEventCountUncached(String ownerId)
throws UnauthorizedSubscriptionException;

long getEventCountUpTo(String ownerId, String subscription, long limit)
throws UnauthorizedSubscriptionException;

Expand Down
2 changes: 1 addition & 1 deletion datacenter/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion event/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../parent/pom.xml</relativePath>
</parent>

Expand Down
17 changes: 8 additions & 9 deletions event/src/main/java/com/bazaarvoice/emodb/event/db/astyanax/AstyanaxEventReaderDAO.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -340,19 +340,15 @@ public void readNewer(String channel, EventSink sink) {
*/
private Iterator<Column<ByteBuffer>> readManifestForChannel(final String channel, final boolean weak) {
final ByteBuffer oldestSlab = weak ? _oldestSlab.getIfPresent(channel) : null;
ConsistencyLevel consistency = ConsistencyLevel.CL_LOCAL_ONE ;//CL_LOCAL_QUORUM;;
final ConsistencyLevel consistency;

RangeBuilder range = new RangeBuilder().setLimit(50);

if (oldestSlab != null) {
range.setStart(oldestSlab);
consistency = ConsistencyLevel.CL_LOCAL_ONE;
} else {

try{
String _sysConsistency = System.getProperty("read-consistency");
consistency = ConsistencyLevel.valueOf(_sysConsistency);
} catch(Exception e){
_log.debug("encountered exception while parsing ", e);
}
consistency = ConsistencyLevel.CL_LOCAL_QUORUM;
}

final Iterator<Column<ByteBuffer>> manifestColumns = executePaginated(
Expand Down Expand Up @@ -404,8 +400,11 @@ private boolean readSlab(String channel, ByteBuffer slabId, SlabCursor cursor, b
// Event add and delete write with local quorum, so read with local quorum to get a consistent view of things.
// Using a lower consistency level could result in (a) duplicate events because we miss deletes and (b)
// incorrectly closing or deleting slabs when slabs look empty if we miss adds.


ColumnList<Integer> eventColumns = execute(
_keyspace.prepareQuery(ColumnFamilies.SLAB, ConsistencyLevel.CL_LOCAL_ONE)

_keyspace.prepareQuery(ColumnFamilies.SLAB, ConsistencyLevel.CL_LOCAL_QUORUM)
.getKey(slabId)
.withColumnRange(start, Constants.OPEN_SLAB_MARKER, false, Integer.MAX_VALUE));

Expand Down
2 changes: 1 addition & 1 deletion job-api/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion job/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion kafka/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../parent/pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion megabus/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.bazaarvoice.emodb</groupId>
<artifactId>emodb-parent</artifactId>
<version>6.5.73-SNAPSHOT</version>
<version>6.5.215-SNAPSHOT</version>
<relativePath>../parent/pom.xml</relativePath>
</parent>

Expand Down
Loading
Loading