Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LSL draft #675

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 60 additions & 1 deletion neon/data-collection/lab-streaming-layer/index.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,61 @@
# Lab Streaming Layer
This page is still a work in progress. To use LSL with Neon you need to use our [LSL Relay](https://github.com/pupil-labs/lsl-relay). More documentation is coming soon!
The Pupil Labs LSL Relay allows you to stream gaze and event data from your Pupil Labs device to [labstreaminglayer](https://github.com/sccn/labstreaminglayer).

Check warning on line 2 in neon/data-collection/lab-streaming-layer/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (labstreaminglayer)

## Installation and Usage
The LSL Relay can be installed using `pip`:
```bash
pip install lsl-relay
```

After installation, you can start it by executing:

```bash
lsl_relay
```

The LSL Relay will search for Pupil Companion device instances in the network. Available instances will be displayed in a list, with the IP address and name of the Companion device. Select the instance you want to connect with via the displayed index.

The `lsl_relay` command takes the following optional arguments:
| Argument | Description |
| :------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--time_sync_interval` | Sets the interval (in seconds) at which the relay sends events to the Pupil Companion device that can be used for time synchronization. The default is 60 seconds. |
| `--timeout` | Defines the maximum time (in seconds) the relay will search the network for new devices before returning. The default is 10 seconds. |
| `--log_file_name` | Defines the name and path of the log file. The default is `lsl_relay.log`. |
| `--device_address IP:PORT` | Connects directly to the specified device. Network device discovery and selection are skipped. The correct format to pass the device address is `device_ip:device_port`. |
| `--outlet_prefix PREFIX` | Sets the prefix of the outlet name displayed in LabRecorder. The default prefix is `pupil_labs`. Each outlet will also have a suffix, specifying the data stream type (Gaze or Event). |

## LSL Outlets

The LSL Relay creates two outlets - one streaming gaze data, the other streaming event data. Both outlets can be subscribed to independently, either through the LabRecorder App or with custom written LSL inlets.

### Gaze Data Outlet

The default name of the gaze data stream is `pupil_labs_Gaze`. This name will also show in the LabRecorder. The info of the data stream follows the recommendations of the [xdf Gaze Meta Data format](https://github.com/sccn/xdf/wiki/Gaze-Meta-Data).

The time series of the gaze data stream consists of two channels - one streaming the x- and the other streaming the y-coordinate of the gaze position in pixel, estimated based on both eyes. The gaze data may be streamed at a reduced frequency. If you want to use the 200 Hz gaze data from pupil cloud, you will need to start a recording with your companion device simultaneously with the LSL recording and use the `lsl.time_sync.*` events generated by the relay to align you data streams post-hoc.

::: tip
If you want to do the post-hoc alignment of LSL data and cloud data, you must also subscribe to the LSL event stream and make sure that at least two events are contained in your recording.
:::

### Event Data Outlet

The default name of the event stream is `pupil_labs_Event`. The event stream contains the name of each generated event as a string.

By default, the event stream will contain an `lsl.time_sync.*` event every 60 seconds. These events can be used to align the gaze and event data from pupil cloud to the lsl xdf file, or vice versa. If you want to change the rate at which the time_sync event is generated, you can use the `--time_sync_interval` argument to set the interval to a value of your choice. If you want to remove the `lsl.time_sync` events, you can set the argument to `0`.

## Timestamps

The recording of gaze data at the companion device and its transfer to LSL involves several steps of taking and aligning timestamps. This section gives you an overview of how the timestamps you’ll get out of LSL will be generated.

The original timestamp for both Gaze samples and Events is generated at the Pupil Labs Companion Device. The timestamps are generated in nanoseconds since the Unix epoch (01.01.1970).

The Gaze samples and Events are transferred to the LSL Relay via the realtime API. Gaze timestamps are converted to seconds during the transfer. Event timestamps are converted to seconds in the LSL Relay.

Upon the arrival of each sample, the LSL Relay computes the difference between the LSL local clock (in seconds) and the momentary time since Unix epoch (converted to seconds). This assumes that the time since epoch measured on the device running the relay and on the companion device are equivalent.

The offset between the LSL local clock and the time since epoch in seconds is subtracted from the sample timestamp in seconds to find the corresponding LSL time. This corrected timestamp is explicitly pushed to LSL together with the Gaze samples and the Events.

::: warning
A misalignment between the time since Unix epoch measured at the companion device and at the time since Unix epoch measured at the device running the Relay might lead to a distortion of the time series.
:::
Loading