XTablesClient
is part of the org.kobe.xbot.Client
package, providing an interface to interact with a server for storing, retrieving, and managing data updates in various formats. This document details the methods available in the XTablesClient
class, including their usage and new enhancements related to caching and error handling.
You can access the Javadocs for XTABLES at the following link:
XTablesClient
boasts superior performance and efficiency compared to traditional FRC Network Tables. Here are some key statistics highlighting the differences:
-
Message Rate:
- FRC Network Tables: Approximately 10 to 20 thousand messages per second.
- XTablesClient: Capable of sending around ~400,000 updates per second, which translates to 25 million messages per minute with 0% loss.
-
Latency:
- FRC Network Tables: Typically higher latency due to lower message rates and less optimized data handling.
- XTablesClient: Average round-trip latency of 0.6ms in a local wireless network, ensuring almost zero latency for real-time applications.
-
Efficiency:
- FRC Network Tables: Basic data management and limited functionalities.
- XTablesClient: Enhanced with advanced features such as dynamic data compression, caching, and hybrid P2P video streaming for maximum optimization.
-
Functionality:
- FRC Network Tables: Primarily designed for basic data sharing and communication.
- XTablesClient: Includes a comprehensive suite of features such as mDNS integration, custom logging, server-side scripting, and extensive caching mechanisms, making it a versatile tool for complex robotics applications and competitions.
The latest version of XTablesClient
introduces support for mDNS (Multicast DNS) to facilitate automatic service discovery. This allows the client to locate and connect to the server using a service name, simplifying network configuration and connection management.
- Server Side: The server registers its service with mDNS, broadcasting its presence on the network with a specified name and port.
- Client Side: The client uses mDNS to discover the server by its service name, extracting the necessary connection details (such as IP address and port) to establish a connection.
This approach is particularly useful in dynamic or large-scale network environments where hardcoding server addresses is impractical.
Some operating systems prefer IPv6, which might not be fully supported by the mDNS implementation used in XTablesClient
. If you encounter issues related to java.net.SocketException: Invalid argument: no further information
, it is likely due to the system preferring IPv6.
To resolve this issue, you must set the JVM option to prefer the IPv4 stack. Add the following option when starting your Java application:
-Djava.net.preferIPv4Stack=true
This setting forces the JVM to use the IPv4 stack, which should resolve the socket exception issue. Make sure to include this option in your command line or IDE configuration when running the application.
-
XTablesClient(String name, int MAX_THREADS, boolean useCache)
- Initializes the client using mDNS service discovery to find the server by name and connect to it.
name
: The service name to look for during mDNS discovery.MAX_THREADS
: Maximum number of threads for the worker client.useCache
: Whether to enable caching.
-
XTablesClient(String SERVER_ADDRESS, int SERVER_PORT, int MAX_THREADS, boolean useCache)
- Initializes the client with server connection and optional caching.
// Initialize the client with direct server address
XTablesClient client1 = new XTablesClient("127.0.0.1", 1735, 10, true);
// Initialize the client using mDNS service discovery
XTablesClient client2 = new XTablesClient("XTablesService", 10, true);
- stopAll()
- Stops all client connections and interrupts the cache management thread.
- updateServerAddress(String SERVER_ADDRESS, int SERVER_PORT)
- Updates the server address and port for the client connection.
- getCacheFetchCooldown()
- Retrieves the cooldown time between cache fetches.
- setCacheFetchCooldown(long cacheFetchCooldown)
- Sets the cooldown time between cache fetches. This defines how frequently, in milliseconds, the client automatically fetches updates from the server to ensure the cache remains synchronized. The default value is 10000 milliseconds (10 seconds).
- isCacheReady()
- Checks if the cache is ready for use.
- getCache()
- Retrieves the current state of the cache.
The caching mechanism in XTablesClient
is designed to minimize network traffic and enhance performance by maintaining a local copy of server data. Upon initialization, the client fetches all current data from the server to populate the local cache. It then subscribes to updates from the server; any subsequent updates on the server side are automatically pushed to the client, ensuring that the local cache is always synchronized with the server.
Additionally, the client automatically fetches updates at intervals defined by the cacheFetchCooldown
to prevent any discrepancies, especially in scenarios where real-time update notifications might fail or get delayed.
This setup reduces the number of network messages needed, as the client does not have to query the server for each data retrieval and relies on real-time updates for any changes.
Note: Any updates to the local cache are local only and do not affect server-side data. This means that changes made directly to the cache will not be reflected on the server unless explicitly sent via an update method (e.g., putString
, putInteger
).
To further optimize network bandwidth usage, XTablesClient
implements data compression using Gzip, followed by Base64 encoding for network transmission. The compression level is dynamically adjusted based on the time it takes to compress data. This dynamic adjustment helps balance the trade-off between compression efficiency and processing speed, ensuring optimal performance under varying network conditions.
The compression level is adjusted as follows:
- If the compression time is faster than the set average, the compression level is increased to achieve higher compression ratios and reduce network payload size.
- Conversely, if the compression time is slower than the set average, the compression level is decreased to prioritize faster processing speed and reduce computational overhead.
You can customize the average compression speed threshold by using the setSpeedAverageMS(double milliseconds)
method in the org.kobe.xbot.Utilities.DataCompression
class. This allows you to fine-tune the compression behavior based on your specific network environment and performance requirements.
- putRaw(String key, String value)
- Stores a raw string value under a specified key.
- putString(String key, String value)
- Stores a string value after converting it to JSON.
- putInteger(String key, Integer value)
- Stores an integer value under a specified key.
- putBoolean(String key, Boolean value)
- Stores a Boolean value under a specified key.
- putObject(String key, Object value)
- Stores any serializable object under the specified key by converting it to JSON.
- putArray(String key, List value)
- Stores a list of objects under a specified key, serialized to JSON format.
- renameKey(String key, String newName)
- Renames an existing key to a new name, effectively updating the key while preserving associated data.
- getRaw(String key)
- Retrieves a raw string value using the specified key.
- getString(String key)
- Retrieves a string value from the server after converting from JSON.
- getInteger(String key)
- Retrieves an integer value using the specified key.
- getBoolean(String key)
- Retrieves a Boolean value using the specified key.
- getObject(String key, Class type)
- Retrieves an object of the specified type using the given key.
- getArray(String key, Class type)
- Retrieves an array of objects of the specified type using the given key.
- getRawJSON()
- Retrieves the raw JSON data from the server.
- delete(String key)
- Deletes the specified key and its associated data from the server.
- deleteAll()
- Deletes the entire dataset and its associated data from the server.
- subscribeUpdateEvent(String key, Class type, Consumer<SocketClient.KeyValuePair> consumer)
- Subscribes to updates for a specific key using a consumer.
- subscribeUpdateEvent(Consumer<SocketClient.KeyValuePair> consumer)
- Subscribes to any update event and returns the updates as raw JSON strings.
- unsubscribeUpdateEvent(String key, Class type, Consumer<SocketClient.KeyValuePair> consumer)
- Unsubscribes from updates for a specific key, removing the consumer.
- completeAll(List<RequestAction> requestActions)
- Completes all provided asynchronous request actions synchronously, returning a list of results.
- queueAll(List<RequestAction> requestActions)
- Queues all provided request actions for asynchronous processing without waiting for results.
-
registerImageStreamServer(String name)
- Registers a video stream with the server using the given name. Starts a local server to serve the video stream.
- Parameters:
name
: The unique name chosen by the client for the video stream.
- Returns: A
VideoStreamResponse
indicating the success or failure of the registration.
-
registerImageStreamClient(String name, Consumer consumer)
- Queries the server for the given stream name and connects to the provided IP and endpoint to receive the video stream.
- Parameters:
name
: The name of the video stream to query.consumer
: A consumer function that processes each received frame.
- Returns: A
VideoStreamResponse
indicating the success or failure of the connection.
- Description: Represents the response from the server for video stream-related operations.
- Fields:
VideoStreamServer streamServer
: The video stream server instance for Client A.VideoStreamClient streamClient
: The video stream client instance for Client B.ImageStreamStatus status
: The status of the video stream operation.String address
: The address of the video stream.
- rebootServer()
- Sends a command to reboot the server.
- sendCustomMessage(MethodType method, String message, Class type)
- Sends a custom message to the server using a specified method and expects a return type as specified.
- ping_latency()
- Measures the network and round-trip latency in milliseconds.
To ensure absolute maximum efficiency and speed, the following methods should be used as they incur no processing:
- executePutString(String key, String value): Stores a string value with no additional processing.
- executePutInteger(String key, int value): Stores an integer value with no additional processing.
- executePutBoolean(String key, boolean value): Stores a boolean value with no additional processing.
- executePutDouble(String key, double value): Stores a double value with no additional processing.
- executePutLong(String key, long value): Stores a long value with no additional processing.
- executePutArray(String key, List value): Stores an array with minimal processing, converting it to JSON.
- executePutObject(String key, Object value): Stores an object with minimal processing, converting it to JSON.
These methods directly send raw messages to the server, bypassing additional processing steps to achieve optimal performance.
The XTablesClient
allows you to run server-side scripts with custom data using the runScript
method.
The script system in XTablesClient
enables executing server-side scripts with custom data inputs. This can be particularly useful for tasks that require server-side processing or automation based on dynamic data provided by the client.
- runScript(String name, String customData)
- Executes a server-side script identified by the
name
parameter, with customData
passed as input.
- Returns a
ScriptResponse
object which provides response status and the return data from the script.
It is recommended to use the queue
method instead of complete
because server-side scripts can take time to complete, and queue
allows for non-blocking execution.
// Run a server-side script named "processData" with custom data
client.runScript("processData", "{\"key\":\"value\"}")
.queue((scriptResponse) -> {
System.out.println("Script Status: " + scriptResponse.status());
System.out.println("Script Response: " + scriptResponse.response());
});
The RequestAction
class, part of the org.kobe.xbot.Client
package, manages the asynchronous and synchronous sending of requests to a SocketClient. It provides comprehensive functionality for sending data over the network, handling responses, and managing communication errors. This class supports operations that are either queued (asynchronous) or complete (synchronous), accommodating diverse interaction needs with the server.
- Asynchronous and Synchronous Operations: Facilitates both non-blocking (asynchronous) and blocking (synchronous) ways to send requests, making it versatile for various operational contexts.
- Error Handling: Integrates robust error management during the request execution, allowing developers to handle exceptions gracefully.
- Timeout Settings: Offers the ability to set default timeout values for synchronous operations, ensuring flexibility in operation timing.
- Response Parsing: Includes mechanisms to parse and format server responses, enabling custom handling of data returned from the server.
- RequestAction(SocketClient client, String value, Type type): Initializes a new request with a specified return type.
- RequestAction(SocketClient client, String value): Constructs a request without specifying a return type, useful for simple send-and-forget operations.
- queue(Consumer onSuccess, Consumer onFailure): Sends a request asynchronously, providing handlers for both success and failure scenarios.
- complete(long timeoutMS): Sends a request and waits for the response synchronously, with a specified timeout.
- execute(): Sends a request and instructs the server to not respond back, helpful when using update methods.
// Queue a request asynchronously with handlers for success and failure
requestAction.queue(response -> System.out.println("Received: " + response),
error -> System.out.println("Error: " + error.getMessage()));
// Perform a synchronous operation with a custom timeout
String response = requestAction.complete(3000).toString();
System.out.println("Synchronous response: " + response);
// Execute a request in the current thread
requestAction.execute(false);
The RequestAction
class is a pivotal component in handling network communication efficiently, supporting a wide range of operations from simple data sends to complex handling of asynchronous processes.
This section describes the implementation and usage of a hybrid peer-to-peer (P2P) video streaming system using the XTablesClient
. The system allows Client A to register a video stream with the server and serve it, while Client B can query the server for available streams and connect directly to Client A to receive the video stream.
- Client A: Registers a unique stream name with the server and starts a local server to stream video captured using OpenCV.
- Server: Stores the stream names registered by clients and ensures no duplicate names. Provides the IP and endpoint of the stream to Client B upon request.
- Client B: Queries the server for the stream name and connects directly to Client A to receive the video stream.
Client A captures video using OpenCV and streams it to a local server. It registers the stream with the central server, providing a unique name chosen by the client.
Note: The server is started on port 4888
locally.
XTablesClient client = new XTablesClient("XTablesService", 5, false);
VideoStreamResponse response = client.registerImageStreamServer("camera1").complete();
if(response.getStatus().equals(ImageStreamStatus.OKAY)){
VideoCapture camera = new VideoCapture(0);
Mat frame = new Mat();
while(camera.read(frame)){
response.getStreamServer().updateFrame(matToByteArray(frame));
}
}
Client B queries the server for the stream name and connects directly to Client A to receive and display the video stream. The consumer is called for each received frame.
Note: Each stream client uses a worker thread so you must increase MAX_THREADS
argument to accommodate for this.
XTablesClient client = new XTablesClient("XTablesService", 5, false);
client.registerImageStreamClient("camera1", (Mat frame) -> {
opencv_highgui.imshow("Received Stream", frame);
opencv_highgui.waitKey(1);
}).complete();
This implementation provides a hybrid P2P video streaming system where:
- Client A registers a unique stream name with the central server and streams video to a local endpoint.
- Client B queries the central server for the stream name and connects directly to Client A to receive the video stream.
This setup ensures efficient video streaming with reduced latency and minimal server load by leveraging direct client-to-client connections for video data transfer.
HybridP2PDemo.mp4
The XTables local status page is hosted on port 4880. It provides real-time information about the server status and connected clients.
- Server Status: Displays the current status of the XTables server and the total number of requests received in the last minute.
- Client Management: Lists all connected clients with their IP addresses and the number of requests each client has made.
- Server Control: Includes a button to reboot the server.
- Client Control: Includes buttons to disconnect individual clients.
To access the status page, open your web browser and navigate to http://localhost:4880
.
- mDNS Server: Hosted on port
5353
. - Default XTables Server: Hosted on port
1735
. - Video Streams: Each video stream is hosted on port
4888
(Client side). - Status Page: The status page is hosted on port
4880
.
The latest version of XTablesClient
introduces custom logging capabilities, allowing users to configure logging levels for enhanced control over debugging and monitoring. To set the logging level, utilize the XTablesLogger.setLoggingLevel(Level)
method.
XTablesClient
flags keys with invalid JSON values when using the XTablesClient#putRawUnsafe
method. If a key's value is not valid JSON, it will be flagged by the server and will not be parsed. In such cases, you must use XTablesClient#getRaw(String key)
to retrieve the raw string value and parse it back yourself.
XTablesClient client = new XTablesClient("XTablesService", 10, true);
// Initialize cache and handle updates
client.subscribeUpdateEvent("session_id", Integer.class, kv -> {
System.out.println("Update received for session_id: " + kv.getValue());
});
// Synchronous operation to put a new session ID
client.putInteger("session_id", 1001).complete();
For more detailed examples, you can go to the main file of the client with many examples.
- Handling JSON parsing errors and interruptions during update processing.
- Efficient management of multiple consumers per update key using a hashmap.
This comprehensive guide ensures clarity on all functionalities and methods available in the XTablesClient
class, facilitating effective server interaction and data management.