Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1.20.2] Endec - Simplier Codec Alternative #184

Closed
wants to merge 31 commits into from
Closed
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
f6d935b
Initial Codeck implementation
Dragon-Seeker Oct 10, 2023
7ac9a08
Method refactors and fix errors
Dragon-Seeker Oct 11, 2023
dd47bef
Implement More Targeted approach for doing different methods of steri…
Dragon-Seeker Oct 21, 2023
e13e5c3
Implement Codeck within Owo Networking
Dragon-Seeker Oct 22, 2023
e090877
Adjust Attributes use case within De-Serilizers
Dragon-Seeker Oct 22, 2023
cb59dbc
Remove SelfDescribedSerilalizer and handle Format Specific Codecks di…
Dragon-Seeker Oct 22, 2023
8559f26
Fix incorrect constructor calls
Dragon-Seeker Oct 22, 2023
440e0be
Preallocate list and map with the given size before adding entries
Dragon-Seeker Oct 23, 2023
f6c017d
Adjust Test Command for Codeck and Replace PacketBufSeriaizer within …
Dragon-Seeker Oct 23, 2023
2a9e5bd
Resolve issue with inablity to have StructField be called with requir…
Dragon-Seeker Oct 23, 2023
7336ee7
Rename unsafe to safeHandling
Dragon-Seeker Oct 23, 2023
08cde20
Change name to Endec as the possible final name
Dragon-Seeker Oct 24, 2023
cfdfc6b
Fix Coopt Class name and add easy way of converting to DFU Codec
Dragon-Seeker Oct 25, 2023
2247a50
Add test for CooptCodec and adjust how error recover is handled with …
Dragon-Seeker Oct 27, 2023
d14078f
Implement MapCarrier with Endec
Dragon-Seeker Nov 7, 2023
3bcfaca
Implement BinaryNbtSerializer and fix class loading issue with UwUSha…
Dragon-Seeker Nov 11, 2023
342aefa
[endec] initial cleanup and first refactors
gliscowo Nov 17, 2023
ed9d552
[endec] move struct-, sequence- and map- (de)serializer types into ba…
gliscowo Nov 17, 2023
1b3b887
[endec] clean up all (de)serializer implementations, standardize (de)…
gliscowo Nov 17, 2023
7d71fec
Just going to leave this here
Dragon-Seeker Nov 19, 2023
6a7d27e
[endec] do you like electronic dance music?
gliscowo Nov 19, 2023
a6588c6
[endec] add support for assuming attributes when embedding endec into…
gliscowo Nov 19, 2023
797d059
[endec] remove reimplementations of vanilla recipes codecs and associ…
gliscowo Nov 19, 2023
7ba7a8a
[endec] quote field names in deserializer error messages
gliscowo Nov 19, 2023
5251110
[endec] superficial cleanup
gliscowo Nov 19, 2023
192dae0
Cleanup ReflectionEndecBuilder
Dragon-Seeker Nov 19, 2023
e4f8808
[endec] initial cleanup on endec interface
gliscowo Nov 20, 2023
269f57a
[endec] remove MapEndec and ListEndec classes, being adding basic doc…
gliscowo Nov 20, 2023
9508b19
[endec] refactor MapCarrier implementations and related infrastructur…
gliscowo Nov 20, 2023
9b0cc57
[endec] add Endec.forEnum
gliscowo Nov 20, 2023
9a6edbe
Merge branch '1.20.2' into 1.20.2-Codeck
gliscowo Nov 21, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 12 additions & 10 deletions src/main/java/io/wispforest/owo/network/OwoHandshake.java
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,12 @@
import io.wispforest.owo.Owo;
import io.wispforest.owo.mixin.ClientCommonNetworkHandlerAccessor;
import io.wispforest.owo.mixin.ServerCommonNetworkHandlerAccessor;
import io.wispforest.owo.network.serialization.PacketBufSerializer;
import io.wispforest.owo.ops.TextOps;
import io.wispforest.owo.particles.systems.ParticleSystemController;
import io.wispforest.owo.serialization.Endec;
import io.wispforest.owo.serialization.impl.ReflectionEndecBuilder;
import io.wispforest.owo.serialization.impl.bytebuf.ByteBufDeserializer;
import io.wispforest.owo.serialization.impl.bytebuf.ByteBufSerializer;
import io.wispforest.owo.util.OwoFreezer;
import io.wispforest.owo.util.ServicesFrozenException;
import net.fabricmc.api.EnvType;
Expand All @@ -28,16 +31,15 @@
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.CompletableFuture;
import java.util.function.ToIntFunction;

@SuppressWarnings("UnstableApiUsage")
@ApiStatus.Internal
public final class OwoHandshake {

@SuppressWarnings("unchecked")
private static final PacketBufSerializer<Map<Identifier, Integer>> RESPONSE_SERIALIZER =
(PacketBufSerializer<Map<Identifier, Integer>>) (Object) PacketBufSerializer.createMapSerializer(Map.class, Identifier.class, Integer.class);
private static final Endec<Map<Identifier, Integer>> RESPONSE_SERIALIZER =
(Endec<Map<Identifier, Integer>>) (Object) ReflectionEndecBuilder.createMapSerializer(Map.class, Identifier.class, Integer.class);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be updated to use a proper MapEndec

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly


private static final MutableText PREFIX = TextOps.concat(Owo.PREFIX, Text.of("§chandshake failure\n"));
public static final Identifier CHANNEL_ID = new Identifier("owo", "handshake");
Expand Down Expand Up @@ -118,7 +120,7 @@ private static void syncClient(MinecraftClient client, ClientConfigurationNetwor
QUERY_RECEIVED = true;

if (buf.readableBytes() > 0) {
final var serverOptionalChannels = RESPONSE_SERIALIZER.deserializer().apply(buf);
final var serverOptionalChannels = RESPONSE_SERIALIZER.decode(ByteBufDeserializer::new, buf);
((OwoClientConnectionExtension) ((ClientCommonNetworkHandlerAccessor) handler).getConnection()).owo$setChannelSet(filterOptionalServices(serverOptionalChannels, OwoNetChannel.REGISTERED_CHANNELS, OwoHandshake::hashChannel));
}

Expand All @@ -133,8 +135,8 @@ private static void syncClient(MinecraftClient client, ClientConfigurationNetwor
private static void syncServer(MinecraftServer server, ServerConfigurationNetworkHandler handler, PacketByteBuf buf, PacketSender responseSender) {
Owo.LOGGER.info("[Handshake] Receiving client channels");

final var clientChannels = RESPONSE_SERIALIZER.deserializer().apply(buf);
final var clientParticleControllers = RESPONSE_SERIALIZER.deserializer().apply(buf);
final var clientChannels = RESPONSE_SERIALIZER.decode(ByteBufDeserializer::new, buf);
final var clientParticleControllers = RESPONSE_SERIALIZER.decode(ByteBufDeserializer::new, buf);

StringBuilder disconnectMessage = new StringBuilder();

Expand All @@ -146,7 +148,7 @@ private static void syncServer(MinecraftServer server, ServerConfigurationNetwor
}

if (buf.readableBytes() > 0) {
final var clientOptionalChannels = RESPONSE_SERIALIZER.deserializer().apply(buf);
final var clientOptionalChannels = RESPONSE_SERIALIZER.decode(ByteBufDeserializer::new, buf);
((OwoClientConnectionExtension) ((ServerCommonNetworkHandlerAccessor) handler).owo$getConnection()).owo$setChannelSet(filterOptionalServices(clientOptionalChannels, OwoNetChannel.OPTIONAL_CHANNELS, OwoHandshake::hashChannel));
}

Expand Down Expand Up @@ -229,7 +231,7 @@ private static <T> void writeHashes(PacketByteBuf buffer, Map<Identifier, T> val
hashes.put(entry.getKey(), hashFunction.applyAsInt(entry.getValue()));
}

RESPONSE_SERIALIZER.serializer().accept(buffer, hashes);
RESPONSE_SERIALIZER.encode(() -> new ByteBufSerializer<>(buffer), hashes);
}

private static Pair<Set<Identifier>, Set<Identifier>> findCollisions(Set<Identifier> first, Set<Identifier> second) {
Expand All @@ -250,7 +252,7 @@ private static Pair<Set<Identifier>, Set<Identifier>> findCollisions(Set<Identif
private static int hashChannel(OwoNetChannel channel) {
int serializersHash = 0;
for (var entry : channel.serializersByIndex.int2ObjectEntrySet()) {
serializersHash += entry.getIntKey() * 31 + entry.getValue().serializer.getRecordClass().getName().hashCode();
serializersHash += entry.getIntKey() * 31 + entry.getValue().getRecordClass().getName().hashCode();
}
return 31 * channel.packetId.hashCode() + serializersHash;
}
Expand Down
135 changes: 106 additions & 29 deletions src/main/java/io/wispforest/owo/network/OwoNetChannel.java
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@

import io.wispforest.owo.mixin.ServerCommonNetworkHandlerAccessor;
import io.wispforest.owo.network.serialization.PacketBufSerializer;
import io.wispforest.owo.network.serialization.RecordSerializer;
import io.wispforest.owo.serialization.impl.RecordEndec;
import io.wispforest.owo.serialization.impl.StructEndec;
import io.wispforest.owo.serialization.impl.bytebuf.ByteBufDeserializer;
import io.wispforest.owo.serialization.impl.bytebuf.ByteBufSerializer;
import io.wispforest.owo.util.OwoFreezer;
import io.wispforest.owo.util.ReflectionUtils;
import it.unimi.dsi.fastutil.ints.Int2ObjectMap;
Expand Down Expand Up @@ -127,14 +130,14 @@ private OwoNetChannel(Identifier id, String ownerClassName, boolean required) {

ServerPlayNetworking.registerGlobalReceiver(packetId, (server, player, handler, buf, responseSender) -> {
int handlerIndex = buf.readVarInt();
final Record message = serializersByIndex.get(handlerIndex).serializer.read(buf);
final Record message = serializersByIndex.get(handlerIndex).serializer.decode(ByteBufDeserializer::new, buf);
server.execute(() -> serverHandlers.get(handlerIndex).handle(message, new ServerAccess(player)));
});

if (FabricLoader.getInstance().getEnvironmentType() == EnvType.CLIENT) {
ClientPlayNetworking.registerGlobalReceiver(packetId, (client, handler, buf, responseSender) -> {
int handlerIndex = buf.readVarInt();
final Record message = serializersByIndex.get(-handlerIndex).serializer.read(buf);
final Record message = serializersByIndex.get(-handlerIndex).serializer.decode(ByteBufDeserializer::new, buf);
client.execute(() -> clientHandlers.get(handlerIndex).handle(message, new ClientAccess(handler)));
});
}
Expand Down Expand Up @@ -164,17 +167,7 @@ private OwoNetChannel(Identifier id, String ownerClassName, boolean required) {
*/
@SuppressWarnings("unchecked")
public <R extends Record> void registerClientbound(Class<R> messageClass, ChannelHandler<R, ClientAccess> handler) {
int deferredIndex = deferredClientSerializers.removeInt(messageClass);
if (deferredIndex != -1) {
OwoFreezer.checkRegister("Network handlers");

this.clientHandlers.set(deferredIndex, (ChannelHandler<Record, ClientAccess>) handler);
return;
}

int index = this.clientHandlers.size();
this.createSerializer(messageClass, index, EnvType.CLIENT);
this.clientHandlers.add((ChannelHandler<Record, ClientAccess>) handler);
registerClientbound(messageClass, handler, () -> RecordEndec.create(messageClass));
}

/**
Expand All @@ -189,11 +182,7 @@ public <R extends Record> void registerClientbound(Class<R> messageClass, Channe
* @see PacketBufSerializer#register(Class, PacketByteBuf.PacketWriter, PacketByteBuf.PacketReader)
*/
public <R extends Record> void registerClientboundDeferred(Class<R> messageClass) {
int index = this.clientHandlers.size();
this.createSerializer(messageClass, index, EnvType.CLIENT);
this.clientHandlers.add(null);

this.deferredClientSerializers.put(messageClass, index);
registerClientboundDeferred(messageClass, () -> RecordEndec.create(messageClass));
}

/**
Expand All @@ -208,11 +197,95 @@ public <R extends Record> void registerClientboundDeferred(Class<R> messageClass
*/
@SuppressWarnings("unchecked")
public <R extends Record> void registerServerbound(Class<R> messageClass, ChannelHandler<R, ServerAccess> handler) {
registerServerbound(messageClass, handler, () -> RecordEndec.create(messageClass));
}

//--

/**
* Registers a handler <i>on the client</i> for the specified message class.
* This also ensures the required serializer is available. If an exception
* about a missing type adapter is thrown, register one
*
* @param messageClass The type of packet data to send and serialize
* @param endec The alternative Endec to Serialize the given Record
* @param handler The handler that will receive the deserialized
* @see #serverHandle(PlayerEntity)
* @see #serverHandle(MinecraftServer)
* @see #serverHandle(ServerWorld, BlockPos)
* @see PacketBufSerializer#register(Class, PacketByteBuf.PacketWriter, PacketByteBuf.PacketReader)
*/
@SuppressWarnings("unchecked")
public <R extends Record> void registerClientbound(Class<R> messageClass, StructEndec<R> endec, ChannelHandler<R, ClientAccess> handler) {
registerClientbound(messageClass, handler, () -> endec);
}

/**
* Registers a message class <i>on the client</i> with deferred handler registration.
* This also ensures the required serializer is available. If an exception
* about a missing type adapter is thrown, register one
*
* @param messageClass The type of packet data to send and serialize
* @param endec The alternative Endec to Serialize the given Record
* @see #serverHandle(PlayerEntity)
* @see #serverHandle(MinecraftServer)
* @see #serverHandle(ServerWorld, BlockPos)
* @see PacketBufSerializer#register(Class, PacketByteBuf.PacketWriter, PacketByteBuf.PacketReader)
*/
public <R extends Record> void registerClientboundDeferred(Class<R> messageClass, StructEndec<R> endec) {
registerClientboundDeferred(messageClass, () -> endec);
}

/**
* Registers a handler <i>on the server</i> for the specified message class.
* This also ensures the required serializer is available. If an exception
* about a missing type adapter is thrown, register one
*
* @param messageClass The type of packet data to send and serialize
* @param endec The alternative Endec to Serialize the given Record
* @param handler The handler that will receive the deserialized
* @see #clientHandle()
* @see PacketBufSerializer#register(Class, PacketByteBuf.PacketWriter, PacketByteBuf.PacketReader)
*/
@SuppressWarnings("unchecked")
public <R extends Record> void registerServerbound(Class<R> messageClass, StructEndec<R> endec, ChannelHandler<R, ServerAccess> handler) {
registerServerbound(messageClass, handler, () -> endec);
}

//--

@SuppressWarnings("unchecked")
private <R extends Record> void registerClientbound(Class<R> messageClass, ChannelHandler<R, ClientAccess> handler, Supplier<StructEndec<R>> endec) {
int deferredIndex = deferredClientSerializers.removeInt(messageClass);
if (deferredIndex != -1) {
OwoFreezer.checkRegister("Network handlers");

this.clientHandlers.set(deferredIndex, (ChannelHandler<Record, ClientAccess>) handler);
return;
}

int index = this.clientHandlers.size();
this.createSerializer(messageClass, index, EnvType.CLIENT, endec);
this.clientHandlers.add((ChannelHandler<Record, ClientAccess>) handler);
}

private <R extends Record> void registerClientboundDeferred(Class<R> messageClass, Supplier<StructEndec<R>> endec) {
int index = this.clientHandlers.size();
this.createSerializer(messageClass, index, EnvType.CLIENT, endec);
this.clientHandlers.add(null);

this.deferredClientSerializers.put(messageClass, index);
}

@SuppressWarnings("unchecked")
private <R extends Record> void registerServerbound(Class<R> messageClass, ChannelHandler<R, ServerAccess> handler, Supplier<StructEndec<R>> endec) {
int index = this.serverHandlers.size();
this.createSerializer(messageClass, index, EnvType.SERVER);
this.createSerializer(messageClass, index, EnvType.SERVER, endec);
this.serverHandlers.add((ChannelHandler<Record, ServerAccess>) handler);
}

//--

public boolean canSendToPlayer(ServerPlayerEntity player) {
return canSendToPlayer(player.networkHandler);
}
Expand Down Expand Up @@ -339,12 +412,12 @@ private ServerHandle getServerHandle() {
return serverHandle;
}

private <R extends Record> void createSerializer(Class<R> messageClass, int handlerIndex, EnvType target) {
private <R extends Record> void createSerializer(Class<R> messageClass, int handlerIndex, EnvType target, Supplier<StructEndec<R>> supplier) {
OwoFreezer.checkRegister("Network handlers");

var serializer = serializersByClass.get(messageClass);
if (serializer == null) {
final var indexedSerializer = IndexedSerializer.create(RecordSerializer.create(messageClass), handlerIndex, target);
final var indexedSerializer = IndexedSerializer.create(messageClass, supplier.get(), handlerIndex, target);
serializersByClass.put(messageClass, indexedSerializer);
serializersByIndex.put(target == EnvType.CLIENT ? -handlerIndex : handlerIndex, indexedSerializer);
} else if (serializer.handlerIndex(target) == -1) {
Expand All @@ -371,9 +444,7 @@ private <R extends Record> PacketByteBuf encode(R message, EnvType target) {
}

buffer.writeVarInt(serializer.handlerIndex(target));
serializer.serializer.write(buffer, message);

return buffer;
return serializer.serializer.encode(() -> new ByteBufSerializer<>(buffer), message);
}

public class ClientHandle {
Expand Down Expand Up @@ -496,14 +567,16 @@ static final class IndexedSerializer<R extends Record> {
private int clientHandlerIndex = -1;
private int serverHandlerIndex = -1;

final RecordSerializer<R> serializer;
final Class<R> rClass;
final StructEndec<R> serializer;

private IndexedSerializer(RecordSerializer<R> serializer) {
private IndexedSerializer(Class<R> rClass, StructEndec<R> serializer) {
this.serializer = serializer;
this.rClass = rClass;
}

public static <R extends Record> IndexedSerializer<R> create(RecordSerializer<R> serializer, int index, EnvType target) {
return new IndexedSerializer<>(serializer).setHandlerIndex(index, target);
public static <R extends Record> IndexedSerializer<R> create(Class<R> rClass, StructEndec<R> serializer, int index, EnvType target) {
return new IndexedSerializer<>(rClass, serializer).setHandlerIndex(index, target);
}

public IndexedSerializer<R> setHandlerIndex(int index, EnvType target) {
Expand All @@ -520,6 +593,10 @@ public int handlerIndex(EnvType target) {
case SERVER -> serverHandlerIndex;
};
}

public Class<R> getRecordClass(){
return this.rClass;
}
}
}

41 changes: 41 additions & 0 deletions src/main/java/io/wispforest/owo/serialization/Deserializer.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
package io.wispforest.owo.serialization;

import io.wispforest.owo.serialization.impl.SerializationAttribute;

import java.util.Optional;
import java.util.Set;

public interface Deserializer<T> {

Set<SerializationAttribute> attributes();

<V> Optional<V> readOptional(Endec<V> endec);

boolean readBoolean();

byte readByte();

short readShort();

int readInt();

long readLong();

float readFloat();

double readDouble();

String readString();

byte[] readBytes();

int readVarInt();

long readVarLong();

<E> SequenceDeserializer<E> sequence(Endec<E> elementEndec);

<V> MapDeserializer<V> map(Endec<V> valueEndec);

StructDeserializer struct();
}
11 changes: 11 additions & 0 deletions src/main/java/io/wispforest/owo/serialization/Endable.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
package io.wispforest.owo.serialization;

/**
* Interface that indicates that the object has an end operation to such
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO it would be nicer to just make an interface that extends AutoCloseable and just makes close not throw anything, kind of like InfallibleCloseable in gadget.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly a good idea as such exists just as a way to prevent issues with calling close method and end is just used due to what glisco used within her version.

public interface Endable extends AutoCloseable {

void end();

@Override default void close(){ end(); }
}
Loading
Loading