diff --git a/README.md b/README.md index 2f21b1f3..f56353d2 100644 --- a/README.md +++ b/README.md @@ -1,66 +1,40 @@ # gamutRF -An SDR orchestrated scanner and collector. +gamutRF is a gnuradio-based SDR-based scanner, I/Q signal collector and identifier (using a image or I/Q based pyTorch model). -gamutRF is a system enabling a compact network of one or more modest machines (such as Pi4s), each with their own USB SDR (such as an Ettus -B200mini or a BladeRF XA9), to operate collectively as a configurable wideband scanner and I/Q sample recorder. +While it can run on Pi4 and Pi5 machines (and its components can be distributed over a network), it is more typically deployed on a single x86_64 machine with an nvidia GPU (see [deployment instructions](https://github.com/IQTLabs/gamutrf-deploy)). + +gamutRF's scanner container connects to a local SDR and sweeps over a configured frequency range or ranges collecting samples. When a configurable number of [valid I/Q samples](https://github.com/IQTLabs/gr-iqtlabs/blob/7990932fa871aa3f84e75771052500551f615638/lib/retune_pre_fft_impl.cc#L276) are received the SDR is retuned to a new interval (see [blocks in gr-iqtlabs](https://github.com/IQTLabs/gr-iqtlabs)). The samples are processed and sent to a waterfall container for display, and optionally to a [Torchserve](https://github.com/IQTLabs/torchserve) instance for identification. Recording, and basic parameters (such as the frequency range to scan) can be controlled from the waterfall container. -A gamutRF system comprises an "orchestrator" machine which typically runs at least one scanner service (typically scanning 0.1GHz to 6GHz in 30s) and the sigfinder service, which then -can command potentially many gamutRF "workers" to make I/Q sample recordings for later analysis. sigfinder can command multiple scanners which can be on the same machine or be connected -over a network. - -gamutRF provides tools to work with I/Q sample recordings, and to also record GPS location/compass metadata for the system itself. gamutRF typically runs on networks of Raspberry Pi4s, but can also run on x86 machines, and is based on gnuradio. - -See also [instructions on how to build a gamutRF system](docs/3-BUILD.md). - -## Scanner theory of operation - -gamutRF's scanner container connects to a local SDR and sweeps over a configured frequency range or ranges collecting samples. The samples are sent an FFT block, and then to a [streaming retuning gnuradio block](https://github.com/iqtlabs/gr-iqtlabs), which aggregates and then serves the frequency-annotated FFT points as JSON objects over ZMQ to the "sigfinder" container (see below) and also commands the source SDR block to retune to a new frequency in the range when enough FFT points have been accumulated. - -The sigfinder container consumes these FFT points over ZMQ (potentially from many scanners), does some noise processing (correcting FFT points to be in frequency order, computing mean power over 10kHz, and then a rolling mean over 1MHz) and then submits them to [scipy.signals.find_peaks](https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.find_peaks.html). - -If workers have been provisioned, the sigfinder will then command the workers to make an approximately 10 second I/Q recording at approximately 20Msps of each signal. Each signal peak is assigned a 20MHz bin, which means that if a signal is repeatedly detected with some frequency variation, the assigned recording bin will be constant, and if multiple signals are detected within 20MHz they can be collected simultaneously. A worker by default records at a higher sample rate than the bin size, so that 20MHz signal margins can be recorded. - -As there will almost certainly be more signals than workers available, sigfinder will prioritize signals that it least often observed over a configurable number of scanner cycles. It is possible to configure this to `off` so that the recording choice will be random. It is also possible to configure the workers to tell the sigfinder to exclude that worker from certain frequency ranges (if for example the worker SDR cannot handle some part of the frequency spectrum scanned). - -## Build and Operating gamutRF +## License -See the [build doc](docs/3-BUILD.md) for instruction on setting up a GamutRF system. Next see the [operating instructions](docs/4-OPERATION.md) for instructions on operating GamutRF. +Distributed under the [Apache 2.0](./LICENSE). See [LICENSE](./LICENSE) for more information. -## Documentation +## Contact IQTLabs -[1 - Introduction](docs/1-INTRODUCTION.md) -[2 - System Overview](docs/2-SYSTEM_OVERVIEW.md) -[3 - Build](docs/3-BUILD.md) -[4 - Operation](docs/4-OPERATION.md) -[5 - ML Tooling](docs/5-ML_TOOLING.md) -[6 - Testing](docs/6-TESTING.md) -[7 - Troubleshooting](docs/7-TROUBLESHOOTING.md) -[AIRT Support](docs/README-airt.md) +- Twitter: [@iqtlabs](https://twitter.com/iqtlabs) +- Email: info@iqtlabs.org -## Contributing +See our other projects: [https://github.com/IQTLabs/](https://github.com/IQTLabs/) -1. Fork the Project -2. Create your Feature Branch (`git checkout -b dev`) -3. Commit your Changes (`git commit -m 'adding some feature'`) -4. Run (and make sure they pass): +## AIR-T support -``` -black --diff --check gamutrf -``` +GamutRF has legacy support for the [Deepwave AIR-T](docs/README-airt.md) -5. Push to the Branch (`git push origin dev`) -6. Open a Pull Request +## Development -See `CONTRIBUTING.md` for more information. +Development with GamutRF requires familiarity with gnuradio, an SDR, a x86_64 host running Ubuntu 24.04 with Docker installed (and ideally an nvidia GPU, though this is not required). -## License +### Local development -Distributed under the [Apache 2.0](./LICENSE). See [LICENSE](./LICENSE) for more information. +* Install [gnuradio](https://wiki.gnuradio.org/index.php/InstallingGR), 3.10 or later +* Install [gr-iqtlabs](https://github.com/IQTLabs/gr-iqtlabs) +* Make modifications, and install with ```poetry install``` +* Run tests with ```pytest``` -## Contact IQTLabs +### Docker development -- Twitter: [@iqtlabs](https://twitter.com/iqtlabs) -- Email: info@iqtlabs.org +Follow above local development instructions, and then build containers (tests will be run inside the containers). -See our other projects: [https://github.com/IQTLabs/](https://github.com/IQTLabs/) +* ```docker build -f docker/Dockerfile.base docker -t iqtlabs/gamutrf-base:latest``` +* ```docker build -f Dockerfile . -t iqtlabs/gamutrf:latest``` diff --git a/docs/1-INTRODUCTION.md b/docs/1-INTRODUCTION.md deleted file mode 100644 index 5168740a..00000000 --- a/docs/1-INTRODUCTION.md +++ /dev/null @@ -1,7 +0,0 @@ -# Introduction to GamutRF - -GamutRF is a combination of open-source software based on the GNU radio framework and low-cost software defined radios, that together implements radio spectrum scanning, finding, and recording raw signals, with signal identification and geolocalization using a variety of machine learning approaches. GamutRF can be run on a set of Raspberry Pi4s working together in a mobile/battery powered system, can scale up to more powerful compute platforms, can offload processing to GPUs, and can support multiple radios and distributed signal finding and recording. - -At its core, the project harnesses SDR platforms that are dynamically retuned across a predefined bandwidth and collating the disparate signals into one cohesive picture, thereby achieving ultrawide spectrum awareness. By giving an ultra-wide stare at the entire usable spectrum, we are better able to understand spectrum utilization, and both common and uncommon RF transmissions. This combination of classification and localization provides numerous opportunities for better understanding the RF spectrum in both time and space. - -The signals of interest are discerned through a two-pronged approach: energy spectra analysis and RF spectrogram-based machine learning classification. The energy spectra analysis involves identifying unusual spikes or patterns in signal energy across the spectrum. This aids in quickly flagging known signals that warrant further investigation. Concurrently, the RF spectrogram-based machine learning classification brings a new dimension to signal identification. By generating and analyzing spectrograms, which represent the spectral content of a signal over time, the system employs machine learning models to classify and categorize signals. This fusion of techniques not only expedites the identification process but also ensures a comprehensive understanding of the RF environment. \ No newline at end of file diff --git a/docs/2-SYSTEM_OVERVIEW.md b/docs/2-SYSTEM_OVERVIEW.md deleted file mode 100644 index 161de777..00000000 --- a/docs/2-SYSTEM_OVERVIEW.md +++ /dev/null @@ -1,125 +0,0 @@ -# System Overview - -The GamutRF system is a software stack that runs on low-cost consumer hardware. To simplify deployment and dependency management GamutRF relies on a series of Docker containers to provide an array of functionalities. Docker is a containerization technology that allows developers to isolate computing environments while only incurring minimal overhead when compared to virtual machines. - -![](imgs/gamutrf-system-software.png) -*Software system of GamutRF* - -For GamutRF, each of these containers provide a specific service required for the collection system to operate. The services are generally a combination of python and C++ software developed by IQT Labs which heavily interfaces with GNU Radio (GR). The containers are distributed across different machines, depending on the requirements of a given deployment. For example, multiple worker nodes for RSSI calculation or signal recording, or multiple scanner nodes. There is always at least one orchestration node, which may also be a scanner node. - -## GamutRF Orchestration - -One of the most important features of GamutRF is its ability to detect energy and task sensors for recording. This is generally referred to as orchestration and is key to optimizing the utilization of constrained resources as in this use case of spectral sensing. Since the sensor cannot be always consuming the entire spectrum, it must be able to rapidly scan, detect, and re-tune other sensors to record on bands and signals of interest. Successful orchestration requires the ability to perform fast and reliable scans on the spectrum to be observed. The faster this can be done, the better the spectrum can be accurately characterized. - -GamutRF achieves optimal scanning by minimizing the time needed for the radio to re-tune to a different band. This varies between radios and will determine how quickly a chunk of spectrum can be scanned. There are other factors such as instantaneous bandwidth (IBW), the maximum bandwidth a radio can see at one time, which determines the size of each re-tune step. The bigger the IBW, the faster a radio can scan since it requires fewer steps to traverse the spectrum. High IBW combined with fast re-tune time is optimal for fast scanning. - -While scanning, after each re-tune of the radio, the GamutRF orchestrator will report any energy above a user-specified threshold of the current band it is in, then it will re-tune to the next band and repeat the process. GamutRF can be configured to task another dedicated radio, called a worker, to collect if it detects energy or a certain signal in any of the bands it is scanning. This form of tasking is often referred to as tip and cue. After a “tip” has been generated by the energy detector, a “cue” is generated to task a dedicated sensor to get a better view of the signal in question. - - -![](imgs/gamutrf-orchestration.png) -*Tip and Cue Example of Orchestrator and Workers* - -## Scanner Theory - -Efficient scanning is key to achieving our goal of sensing and identifying signals across a wide band of RF spectrum in the fastest possible time. The heart of the GamutRF system is the scanner which is based on the open-source GNU Radio SDR framework (see section on GNU Radio). The scanner is a GNU radio flowgraph containing several blocks that sweep across the spectrum at a configurable rate, while also storing the raw I/Q sample data for analysis and producing a spectrogram. The scanner also performs peak detection in which the center frequency, bandwidth and magnitude of energy exceeding a certain threshold is noted. - -Over the course of the project GamutRF’s scanner implementation was improved to enable it to scan more quickly. The first implementation of the scanner used host-based tuning in which the computer running the scanner sent commands to the SDR at fixed time intervals to achieve the sweep. This approach proved not to work well because tuning commands could not be issued at a consistent rate resulting in a variable sweep duration. - -The current GamutRF implementation now uses sample counting. When a pre-determined number of samples have been received by the SDR, GamutRF sends a tuning command to move to a new frequency while continuing to process samples. Samples arrive at the SDR at a fixed rate which eliminates need for a host-based timer and results in more consistent scan times. This technique was a significant improvement to data quality and scan stability. - -In addition to the SDR retune/command function, the other core component of the scanner is Fast Fourier Transform (FFT) processing. The FFT is used to generate spectrograms and spectral plots of the energy in the scan across frequency. Computation of the FFT is CPU intensive and can slow the system down. We were able to improve scanning performance by offloading FFT calculations from the CPU to a GPU using VkFFT1. VkFFT is a Fast Fourier Transform library optimized for GPUs and supports several platforms including the Raspberry Pi-4 and Intel NUC computers. - -Our GNU radio design also enabled us to use the same scanner implementation on a separate platform, the Deepwave AIR-T. - -## System Components - -GamutRF has several components and services that are required for successful operation. The key components and services are described below. These are grouped into hardware and software components. - -### GamutRF Hardware - -GamutRF supports a variety of different hardware and radios ranging from inexpensive to moderately priced. Two of these systems are examples of different hardware price-points and are discussed next. - -#### Ettus B200mini/Raspberry Pi-4 - -The “Pi Rig” is a version of the GamutRF platform built around commodity single board computers and commercial SDRs and is the least expensive of the options tested. The use of all consumer grade components keeps the project accessible. This compute platform paired with the Ettus B200 mini are a capable collection platform providing 20Msps across the 300MHz to 6GHz frequency spectrum. - -![](imgs/gamutrf-pi-rig.png) -*Raspberry Pi based GamutRF System* - -The following COTS equipment is used to build the Pi Rig: -- 3x Raspberry Pi 4 -- 3x Ettus B200mini -- 1x POE ethernet switch -- 1x Adafruit Ultimate GPS Board -- 1x Adafruit Compass Board - -As described above, a Raspberry Pi node can be deployed as either an orchestrator/scanner or as a worker. Our standard build includes one orchestrator node and two worker nodes. More workers could be added as needed. - -While the Pi Rig is a capable demonstration unit, system performance is limited by the compute power available in such a small form factor. The data velocity in the RF space is extremely high and often reaching the limit of what the Pi is capable of handling. In our testing we have identified optimization to maximize the performance achievable by the Pi 4. This bottleneck is caused by frequency of accessing the USB controller to receive I/Q data and the FFT transform from time to frequency space. FFT is a computationally expensive task which has been offloaded to the onboard GPU using the VkFFT framework. This allows the CPU to focus on compression and disk I/O to avoid dropping data frames. While performance is limited, the Pi system is still a performant tool when used with precision. We believe the Pi form factor is best used for narrow band scanning or in non-linear scan mode focusing only on frequency bands of interest. Because the cost of the platform is less than $5,000, it is a competitive target for orchestrator – worker workflows. - -#### AIR-T - -The Deepwave Artificial Intelligence Radio Transceiver (AIR-T)2 contains SDRs integrated with processing and deep learning inference hardware. The AIR-T also incorporates a graphics processing unit (GPU) to enable real-time wideband digital signal processing (DSP) algorithms to be executed in software. The AIR-T utilizes direct memory access (DMA) for extremely fast and efficient data transfers. These features combined with the large instantaneous bandwidth of the radio make it one of the higher performing systems for RF based machine learning applications. - -The AIR-T performed well in our use-cases and has a good application programming interface (API) for integration into custom software. The disadvantage of the AIR-T is that it is highly dependent on the NVidia GPU, in this case the Jetson TX2. This can be problematic for lifecycle support as this GPU relies on the Ubuntu 18.04 operating system, which has been deprecated as of April 2023. While this currently does not present major issues preventing the use of this system, this may not necessarily be the case in the future as this operating system ages and loses community support. Unfortunately, the AIR-T is not able to be upgraded with newer hardware. - -### GamutRF Software - -GamutRF uses several applications and services to perform the various tasks associated with spectral awareness and geolocalization. The key applications and services are described below. - -- gamutrf-scan: Interfaces with SDR hardware to scan and dynamically retune. -- gamutrf-sigfinder: Uses peak detection to find signals of interests, tasks workers. -- gamutrf-waterfall: ingests FFT values, creates GUI to visualize data. -- Compass: Interfaces with the compass hardware to make heading accessible to api. -- gpsd: systemd process to provide endpoint for accessing GPS information. -- API: container executed by a worker, receives tasking via sigfinder or manually, listens to narrow spectrum, publishes RSSI and heading info to MQTT. -- Birdseye: signal localization service, ingests RSSI and heading from worker. -- ZMQ: High-speed message broker, used to pass IQ / FFT data/metadata to services. -- MQTT: Low-speed message broker for passing RSSI, GPS, and heading to Birdseye. - -The generic route for data during operation starts with the gamutrf-scan container reading I/Q data directly from the SDR. This I/Q is then processed into FFT values by CPU or GPU. The FFT values are passed to the ZMQ broker and optionally the I/Q is stored on the local file system. ZMQ then passes the FFT data to both sigfinder and the waterfall (GUI). The GUI simply visualizes spectrum, while sigfinder identifies signals of interest which should be further investigated. - -Sigfinder can task the api container which is running on a worker node to investigate a signal. The worker tunes to listen to the tasked frequency and either calculates the associated RSSI or makes a recording of the signal. In RSSI mode, the worker interrogates the gpsd service and compass container running on the orchestrator to get position and heading, then publishes location, heading, and RSSI to the MQTT broker to be consumed by Birdseye. Birdseye then uses this information to geolocate the target signal. - -Orchestrator.yml and worker.yml are the main control points for docker-compose to start the GamutRF system. These scripts spin up the iqtlabs/gamutrf Docker container and run the appropriate gamutrf- python command. These files should be modified for the specific orchestrator and worker combination. Some common parameters for the gamut-scan and gamutrf-sigfinder are in the GamutRF [operation guide](./4-OPERATION.md).md. The repository also provides a few tools such as freespacer, compress_dirs, etc that can be used for post processing of data. These tools are included in the GamutRF python module so it is suggested to use them via iqtlabs/gamutrf docker container. - -## Graphical User Interface - -While not needed for running, the GUI offers users an intuitive means of visualizing and comprehending the collected data. The process ingests FFT samples being streamed from a ZeroMQ (ZMQ) broker where sampled values are collated and plotted through a Python Matplotlib script. This plot showcases both the immediate spectral content of the received signals and frequency-time spectrogram where the intensity corresponds to the signal energy at that given time and frequency, granting users a live view of the spectrum's fluctuations. - -![](imgs/gamutrf-gui.png) -*Waterfall GUI of GamutRF* - -The final output of the data visualization process is wrapped in a Flask-based webpage which can auto configure based on the metadata provided with the FFT values from the ZMQ broker. All that is needed is to point the gamutrf-waterfall container to the gamutrf-scanner. This step can be done locally on the GamutRF system or on a remote computer such as a laptop or phone. GamutRF also provides a webform for configuring the parameters of a worker. These parameters include the frequency, bandwidth, and duration of recording. The webform offers a manual way to submit API requests to the worker, like tasking via sigfinder. - -## Data Transport - -Aside from the control APIs, GamutRF’s primary data transport between components is Java-Script Object Notation (JSON)5 over ZMQ6, with the scanner acting as a publisher, supporting potentially many clients such as user interface instances, or signal finders. This transport is unidirectional going from scanner to client and provides FFT, time, frequency, power, and scan metadata to include scanner configuration information such as frequency range, sample rate, and the current state of the scan. GamutRF includes a client library that implements this API and can be used by third party systems to receive GamutRF scanner data. The library supports aggregation, such that a client may attach to multiple scanners (which could have their own radios and may be scanning different ranges at different rates). - -## GNU radio - -GNU Radio7 was used throughout the course of development for this effort. GNU Radio is an open-source SDR framework used for developing prototypes and signal processing blocks. It supports a variety RF hardware like the radios previously mentioned to create software-defined radio-based systems, or without hardware to simulate or evaluate a waveform or signal flow. GNU Radio provides pre-defined signal processing blocks that have been thoroughly tested which can be used to build waveforms. It also lets you easily build custom blocks that can be integrated into a signal flow. GNU Radio has a well-documented and supported API and has a large user-base with a lot of community support. - -### GNU Radio Out-Of-Tree Modules - -One advantage of using the GNU Radio framework is that is offers a great deal of flexibility if you need to extend GNU Radio with your own functions and blocks that are not in the main source tree. These custom blocks and functions are called Out-Of-Tree (OOT) modules8. We developed several OOT modules for this work and they are contained in gr-iqtlabs which is described in more detail below. - -Part of the GamutRF system consists of GNU radio software blocks written by IQT Labs, that have been made open source at http://github.com/iqtlabs/gr-iqtlabs. We intend to add more blocks over time as we continue to evolve GamutRF and our work in SDR generally. The repository also allows the wider community to benefit from our work by using the modules separately from GamutRF if need be. - -The blocks are divided into two main categories; blocks that directly provide GamutRF scanner functionality, chief among them the retune_fft, image_inference, and VkFFT blocks. These blocks provide the scan-and-retune sweep, production of spectrograms for ML inference, and CPU offload of FFT calculations respectively, all functions which are not provided by the core GNU radio framework. - -The other blocks provide supporting functionality. The block tuneable_test_source simulates a retuneable SDR device with predictable simulated simple data output so that the scanner code can be automatically tested with best practice continuous integration techniques. The write_freq_samples block implements compressed sample archiving using the zstandard9 compression algorithm. The vector_roll and vector_to_json blocks enable third party FFT output to be rearranged into a more convenient layout and allow GNU radio raw data to be represented as JSON for easier integration with third party software, respectively. - -## Data Compression - -Digital signal processing with SDR involves generating large volumes of data in producing, processing, and storing millions of I/Q samples per second. With frameworks such as GNU radio, the samples must be transformed into complex, 32-bit floating point numbers before further processing. As an example, data can be produced at rates of up to 0.75GB per second in extreme cases. This data needs to be consumed, processed, and sometimes stored to disk. This presents storage capacity and I/O performance challenges on mobile platforms. For example, on the Raspberry Pi4, the USB3 controller hardware can struggle to handle very high incoming data rates from SDRs and outgoing data rates to USB3 hard disk simultaneously. Even when cloud storage is available and plentiful, network transfer speeds to and from cloud storage can be a bottleneck. - -We observed that many commonly available SDRs were based on similar ADC hardware, and that hardware has only 12-bit resolution, which lead to the insight that the raw data should be highly compressible since not all the full 32 bits of floating-point resolution are needed. We benchmarked several compression approaches and found that zstandard achieved 30-40% compression at very low CPU cost. zstandard is intended specifically to optimize CPU cost and performance. This serves to relieve USB3 bus contention between SDR and hard disks on the more resource constrained platforms such as the Raspberry Pi4. We implemented libraries that allow ML pipelines to read zstandard compressed data which saves cloud storage space. - -## Signal Strength - -Initially we used the received signal strength along with simple propagation models to estimate distance and track the signals we were interested in. The received signal strength (RSS) is a measurement of the power of a detected signal at the receiver. The RSS indicator or RSSI is a value or index that is proportional to the received power of the signal. There is generally no set standard range for RSSI, different equipment manufacturers define their own range scale. For example, Cisco uses 0-100, while Atheros uses 0-60 and others use 0-255. RSSI is usually measured at the front end of the radio as close to the antenna terminal as possible to get the optimal measurement of the received signal power. - -We take a moving average of this value to smooth it out as it can jump around quite a bit and we are most interested in the slower moving trend of the value. This helps to estimate distance from the target. Figure 7 below shows the portion of the GNU Radio flow graph that estimates the RSSI value. - -Birdseye (RF geolocalization tool) is also located on the IQT Labs Github at https://github.com/iqtlabs/birdseye. The repository contains instructions for using the Birdseye tool as well as different methodologies for ML-enabled localization. Birdseye can be used in conjunction with GamutRF or as a standalone tool. When using with the GamutRF system, the instructions for integration are in the Build.md file which detail how to run Birdseye as a systemd service. diff --git a/docs/3-BUILD.md b/docs/3-BUILD.md deleted file mode 100644 index fe18d3df..00000000 --- a/docs/3-BUILD.md +++ /dev/null @@ -1,382 +0,0 @@ -# Building a GamutRF System - -## Prerequisites - -- 1 x Raspberry Pi4 running Raspberry Pi OS (64-bit), A port of Debian Bullseye with the Raspberry Pi Desktop (Orchestrator) -- 1 x Raspberry Pi4 running Ubuntu 22.04.1 LTS Server (64-bit) -- 1 x [PoE Switch](https://www.amazon.com/gp/product/B087QQ46K4) -- 1 x [GPS module](https://www.adafruit.com/product/746) -- 1 x [7" Touchscreen](https://www.amazon.com/dp/B09X2N9C5V) -- 1 x [Triple Axis Compass](https://www.amazon.com/gp/product/B07PP67N9Q) -- 2 x [Ettus B200-mini](https://www.ettus.com/all-products/usrp-b200mini/) -- 1 x [USB 3.1 Flash Drive](https://www.amazon.com/dp/B07857Y17V) -- 1 x [Antenna Splitter](https://www.amazon.com/gp/product/B07STYNB6V) -- 1 x [Directional Antenna 850-6500Mhz](https://www.wa5vjb.com/products1.html) - -## Installation overview - -These instructions are for 2 different machines: - - One that is the [orchestrator](#orchestrator) which will do signal scanning, serve up GPS and heading, send out requests to workers to collect I/Q or RSSI sample for a given frequency, and run the [BirdsEye](https://github.com/IQTLabs/BirdsEye) interface. - - One that is the [worker](#worker) which will do collection as instructed via API from the orchestrator. There can be `n` number of workers assigned to a given orchestrator. - - For instructions on builing the AIR-T system see the [AIR-T README](./README-airt.md). - -## Orchestrator - -### Quick Installation - -If you installing on x86 or Arm without need for GPIO or GPS. - -1. Either clone the repo or copy the `gamutRF/bin/install-deps.sh` install script. -```bash -git clone https://github.com/IQTLabs/gamutRF -``` - -2. Run install script -```bash -cd gamutRF -chmod +x bin/install-deps.sh -./bin/install-deps.sh -``` - -3. Pull/Update GamutRF containers -```bash -chmod +x bin/update.sh -./bin/update.sh -``` - -4. Check install was completed -```bash -chmod +x bin/check-install.sh -./bin/check-install.sh -``` - -5. Start the Orchestrator containers -```bash -VOL_PREFIX=/flash/gamutrf/ FREQ_START=70e6 FREQ_END=6e9 docker compose -f orchestrator.yml -f torchserve.yml up mqtt gamutrf waterfall -d -``` -If successful, you should see the containers running with `docker ps`. - -6. Navigate to localhost:9003 to see GUI. - -### Advanced Installation - -If you are installing on a Raspberry Pi, need access to GPIO, or you will need a few additional steps for installation. - -1. Install Raspberry Pi4 to 7" Touchscreen - -2. Install GPS module to the 7" Touchscreen 40-pin header -``` -PPS -> PIN12 (GPIO18) -VIN -> PIN17 (3.3V) -GND -> PIN25 (GND) -RX -> PIN8 (TXD) -TX -> PIN10 (RXD) -``` - -3. Install Triple Axis Compass to the 7" Touchscreen 40-pin header -``` -VCC -> PIN1 (3.3V) -GND -> PIN6 (GND) -SCL -> PIN5 (SCL) -SDA -> PIN3 (SDA) -``` - -4. Plug in an Ettus B200 mini into a USB3 port on the Pi4. - -5. Install Raspberry Pi OS (64-bit), a port of Debian Bullseye with the Raspberry Pi Desktop to the micro SD card. - -6. Install dependencies: -``` -sudo apt-get update -curl -fsSL https://get.docker.com -o get-docker.sh -sudo sh get-docker.sh -sudo usermod -aG docker $USER -sudo apt install -y git libjpeg-dev python3 python3-pip python3-tk uhd-host gpsd gpsd-clients chrony pps-tools onboard at-spi2-core tmux -sudo /usr/libexec/uhd/utils/uhd_images_downloader.py -t "b2|usb" -git clone https://github.com/IQTLabs/gamutRF -git clone https://github.com/IQTLabs/BirdsEye -sudo su - -mkdir -p /flash/gamutrf -echo "dtoverlay=vc4-kms-dsi-7inch" >> /boot/config.txt -echo "dtoverlay=vc4-kms-v3d" >> /boot/config.txt -echo "hdmi_cvt=1024 600 60 3 0 0 0" >> /boot/config.txt -echo "hdmi_group=2" >> /boot/config.txt -echo "hdmi_mode=87" >> /boot/config.txt -echo "hdmi_drive=2" >> /boot/config.txt -``` - -7. Set options in raspi-config -``` -sudo raspi-config --> Interface Options - -> Serial Port - -> Disable serial login - -> Enable serial hardware - -> I2C - -> Enable -``` -Reboot - -8. Install GamutRF and BirdsEye -``` -cd gamutRF && docker compose -f orchestrator.yml pull && cd .. -cd BirdsEye && pip3 install -r requirements.txt && cd .. -``` - -9. Enable GPSD to listen on the external interface - -Change `/lib/systemd/system/gpsd.socket` to match the following: -``` -[Unit] -Description=GPS (Global Positioning System) Daemon Sockets - -[Socket] -ListenStream=/run/gpsd.sock -#ListenStream=[::1]:2947 -#ListenStream=127.0.0.1:2947 -# To allow gpsd remote access, start gpsd with the -G option and -# uncomment the next two lines: -ListenStream=[::]:2947 -ListenStream=0.0.0.0:2947 -SocketMode=0600 -#BindIPv6Only=yes - -[Install] -WantedBy=sockets.target -``` - -10. Add BirdsEye systemd service - -Create `/etc/systemd/system/birdseye.service` to contain the following: -``` -[Unit] -Description=BirdsEye - -[Service] -Type=simple -Restart=always -User=pi -Group=pi -WorkingDirectory=/home/pi/BirdsEye -Environment=DISPLAY=:0 -ExecStart=/usr/bin/python3 sigscan.py - -[Install] -WantedBy=multi-user.target -``` - -11. Add GPSD systemd service - -Create `/etc/systemd/system/gpsd.service` to contain the following: -``` -[Unit] -Description=GPSD - -[Service] -Type=simple -Restart=always -ExecStart=/usr/sbin/gpsd -G -N -n /dev/serial0 -F /var/run/gpsd.sock - -[Install] -WantedBy=multi-user.target -``` - -12. Add to the end of `/etc/chrony/chrony.conf`: -``` -# SHM0 from gpsd is the NMEA data at 9600, so is not very accurate -refclock SHM 0 delay 0.5 refid NMEA -refclock PPS /dev/pps0 refid PPS - -# SHM1 from gpsd (if present) is from the kernel PPS_LDISC -# module. It includes PPS and will be accurate to a few ns -#refclock SHM 1 offset 0.0 delay 0.1 refid NMEA+ -# -allow 192.168.111.0/24 -``` - -13. Add to the following sections in `/etc/systemd/system/chronyd.service`: -``` -[Unit] -StartLimitIntervalSec=30 -StartLimitBurst=5 - -[Service] -Restart=on-failure -``` - -14. Enable the PPS GPIO: -``` -sudo su - -echo 'pps-gpio' >> /etc/modules -``` - -15. Enable services -``` -sudo systemctl enable birdseye.service -sudo systemctl enable gpsd.service -sudo systemctl enable gpsd.socket -sudo systemctl daemon-reload -``` - -16. Set static IP address for wired connection (plug ethernet into the PoE switch - non-PoE port) -``` -sudo su - -echo 'interface eth0' >> /etc/dhcpcd.conf -echo 'static ip_address=192.168.111.10/24' >> /etc/dhcpcd.conf -``` - -17. Disable bluetooth and wifi (note you'll want to be wired into the switch and have an IP on the `192.168.111.0/24` subnet to maintain remote access to the Orchestrator after you do this) -``` -sudo rfkill block bluetooth -sudo rfkill block wlan -``` - -18. Set any [keyboard configuration changes](https://docs.sunfounder.com/projects/ts-7c/en/latest/settings_for_raspberry_pi.html#install-virtual-keyboard-on-raspberry-pi) -``` -Raspberry Pi Icon -> Preferences -> Onboard Settings -``` - -19. Reboot -``` -sudo reboot -``` - -20. Start the orchestrator containers -``` -cd gamutRF -UHD_IMAGES_DIR=/usr/share/uhd/images uhd_find_devices && VOL_PREFIX=/flash/gamutrf/ FREQ_START=70e6 FREQ_END=6e9 docker compose -f orchestrator.yml up -d -``` -(optionally add `-f monitoring.yml` if you want additional monitoring containers) - -Additionally, if you want to use the workers as recorders you'll want to update `orchestrator.yml` (before running the `docker compose` command above) under the gamutRF directory to include it. Multiple workers can be assigned to be recorders. Here's an exmaple with two: -``` - sigfinder: - restart: always - image: iqtlabs/gamutrf:latest - networks: - - gamutrf - ports: - - '80:80' - - '9002:9000' - volumes: - - '${VOL_PREFIX}:/logs' - command: - - gamutrf-sigfinder - - --scanners=sigfinder:8001 - - --log=/logs/scan.csv - - --fftlog=/logs/fft.csv - - --fftgraph=/logs/fft.png - - --width=10 - - --prominence=2 - - --threshold=-25 - - '--freq-start=${FREQ_START}' - - '--freq-end=${FREQ_END}' - - --recorder=http://192.168.111.11:8000 - - --recorder=http://192.168.111.12:8000 -``` - -### Worker - -1. Plug in an Ettus B200 mini into a USB3 port on the Pi4. - -2. Plug in the USB3.1 Flash drive into a USB3 port on the Pi4. - -3. Install Ubuntu 22.04.1 LTS Server (64-bit) to the micro SD card (NOTE: Raspbian should also work, but has not been tested). - -4. Install dependencies and configuration. -``` -sudo apt-get update -sudo apt-get -y upgrade -curl -fsSL https://get.docker.com -o get-docker.sh -sudo sh get-docker.sh -sudo usermod -aG docker $USER -sudo apt install -y git python3 python3-pip uhd-host -sudo /usr/lib/uhd/utils/uhd_images_downloader.py -t "b2|usb" -git clone https://github.com/IQTLabs/gamutRF -sudo su - -echo "dtoverlay=vc4-kms-v3d-pi4" >> /boot/firmware/config.txt -``` - -5. Setup Flash drive - -Verify the device name using `lsblk -f` and create an ext4 filesystem on it: -``` -sudo mkfs -t ext4 /dev/sda1 -``` - -Copy the UUID of the device from `lsblk -f` (note it will have changed after running `mkfs`). Add the following (replacing the UUID value with your own) to `/etc/fstab`: -``` -UUID=a04e77e2-772e-45b0-8590-bfb0741d855d /flash ext4 defaults,auto,users,rw,nofail 0 0 -``` - -6. Set static IP address for wired connection (plug ethernet into the PoE switch - PoE port). Add the following to the end of `/etc/netplan/50-cloud-init.yaml`: -``` - ethernets: - eth0: - addresses: - - 192.168.111.11/24 -``` - -7. Reboot -``` -sudo reboot -``` - -8. Install GamutRF -``` -cd gamutRF && docker compose -f worker.yml pull && cd .. -sudo mkdir -p /flash/gamutrf -``` - -9. Choose what type of worker you want: - -Each worker can be run in either `recorder` mode or `RSSI` mode. - -If run in `recorder` mode (the default) no changes on the worker are needed, but the recorder needs to be added to the orchestrator as described above. In `recorder` mode the worker will capture full I/Q samples in `s16` format, and write it out to `/flash/gamutrf` as `.zst` compressed files. - -If run in `RSSI` mode the `worker.yml` file under the gamutrf directory needs to be updated to include the following options: -``` - gamutrf-worker: - restart: always - image: iqtlabs/gamutrf:latest - networks: - - gamutrf - ports: - - '8000:8000' - cap_add: - - SYS_NICE - - SYS_RAWIO - privileged: true - devices: - - /dev/bus/usb:/dev/bus/usb - volumes: - - '${VOL_PREFIX}:/data' - environment: - - 'WORKER_NAME=${WORKER_NAME}' - - 'ORCHESTRATOR=${ORCHESTRATOR}' - - 'CALIBRATION=${CALIBRATION}' - - 'ANTENNA=${ANTENNA}' - command: - - nice - - '-n' - - '-19' - - gamutrf-worker - - --no-agc - - --rxb=62914560 - - '--gain=${GAIN}' - - --rssi - - --rssi_threshold=-110 - - --rssi_throttle=10 -``` -RSSI mode will only record signal strength in the form of float. - -10. Start GamutRF -``` -cd gamutRF -UHD_IMAGES_DIR=/usr/share/uhd/images uhd_find_devices && VOL_PREFIX=/flash/ ORCHESTRATOR=192.168.111.10 WORKER_NAME=worker1 ANTENNA=directional docker compose -f worker.yml up -d -``` - -## Initiating an API request - -If running the orchestrator as a scanner with recorder workers, it will automatically make requests via the API for you based on signals it detects and the workers will start capturing I/Q. However if you want to control what gets recorded or are using workers in RSSI mode you'll need to make manual requests. Browse to the IP of the orchestrator via your browser and make a request with the specifications you desire. diff --git a/docs/4-OPERATION.md b/docs/4-OPERATION.md deleted file mode 100644 index da45974f..00000000 --- a/docs/4-OPERATION.md +++ /dev/null @@ -1,238 +0,0 @@ -# Operating GamutRF - -This section details the operation of a GamutRF system and how to configure the system (both orchestrator and worker) to achieve common use cases. - -Table of Contents: -- [CLI Options](#sdrscannersigfinder-command-line-options) -- [Common Use Cases](#common-operation-use-cases) -- [Running multiple radios on single maching](#running-multiple-radios-on-the-same-machine) -- [Manually initiating worker actions](#manually-initiating-worker-actions) -- [Utility Functions](#utility-functions) - -## SDR/scanner/sigfinder command line options - -While there are other options, these options primarily influence gamutRF's scanner functionality. These CLI args are generally changes in the docker compose files, but can be used in a standard terminal as well if GamutRF is installed. - -### gamutrf-scan - -| Option | Description | -| -- | -- | -| --freq-start and --freq-end | Start and end of frequency range to scan in Hz (if --freq-end=0, will run in "stare" mode, static scanning from --freq-start to --freq_start + (--tuneoverlap * --samp_rate))| -| --tuning_ranges | Overrides --freq-start and --freq-end if present. A comma separated list of ranges in Hz to scan, for example ```2.2e9-2.6e9,5.1e9-5.9e9``` | -| --igain | SDR input gain in dB | -| --samp-rate | Number of samples/sec | -| --tuneoverlap | For each retune interval, advance center frequency by N * --samp_rate | -| --sweep-sec | Attempt to sweep frequency range within N seconds (effectively, best guess for --tune-step-fft) | -| --tune-step-fft | Overrides --sweep-sec if present. Retune to next interval every N FFT points | -| --skip-tune-step | Discard N FFT points after retuning | -| --nfft | Number of FFT points | -| --write_samples | If > 0, write N samples to disk after retuning, in --sample_dir as a zstd compressed file | -| --description | Optionally provide text description along with streaming scanner updates, to the sigfinder | - -### sigfinder - -| Option | Description | -| -- | -- | -| --bin_width | Bin width in MHz | -| --history | Number of scanner cycles over which to prioritize recording of least often observed peaks | -| --record_bw_msps | Number of samples per second in units of 1024^2 (generally larger than bin size to record signal margins) | -| --record_secs | Length of time to record I/Q samples in seconds | -| --fftlog | Log raw output of CSV to this file, which will be rotated every --rotatesecs | -| --scanners | Comma separated list of scanner ZMQ endpoints to use, for example --scanners=127.0.0.1:8001,127.0.0.1:9002 | - -## Common Operation Use Cases - -Below are a list of common use cases and the configuration required. - -Use Cases: -- [Scan a single frequency](#scan-a-single-frequency) -- [Scan across a frequency range](#scan-across-a-frequency-range) -- [Scan across a non-linear range](#scan-across-a-non-linear-range) -- [Deploy signal detector](#deploy-signal-detector) -- [Deploy Birdseye](#deploy-birdseye) -- [Scan a single frequency with model](#scan-a-single-frequency-with-model) -- [Scan across a frequency range with model](#scan-a-single-frequency-with-model) -- [Scan across a non-linear range with model](#scan-across-a-non-linear-range-with-model) -- [Scan across freq with model feeding to Birdseye](#scan-across-freq-with-model-feeding-to-birdseye) - -The main components are located in -- `orchestrator.yml`: main GamutRF SDR orchestration and message passing -- `torchserve*.yml`: ML inference server (`torchserve.yml` for CPU inference or `torchserve-cuda.yml` for GPU inference) -- `utils/mavlink-api/mavlink-api.yml`: Interface to MAVLINK based GPS unit (for Birdseye only) -- `worker.yml`: when orchestrating multiple units in orchestrator-worker configuration - -After linking the relevant docker-compose file(s), you can call the needed services or use all services by not specifying individual containers. - -Notes: The `VOL_PREFIX` is the location all output files will be saved. Check and/or comment out the `deploy` and `depends_on` sections of `gamutrf` in `orchestrator.yml` if you are using a non-x86 architecture or not using torchserve respectively. If needed, run `uhd_find_devices` to check if SDR can be found. It is recommended to include `-f docker/freespacer.yml` and `freespacer` to avoid filling up your disk. - -### Scan a single frequency -```bash -VOL_PREFIX=/flash/gamutrf FREQ_START=2.4e9 FREQ_END=0 docker compose -f orchestrator.yml up mqtt gamutrf waterfall -d -``` -### Scan across a frequency range -```bash -VOL_PREFIX=/flash/gamutrf FREQ_START=2.4e9 FREQ_END=5.8e9 docker compose -f orchestrator.yml up mqtt gamutrf waterfall -d -``` -### Scan across a non-linear range -Uncomment `--tuning_range` in gamutrf service in `orchestrator.yml` and adjust range as needed. `tuning_range` will override `freq_start` and `freq_end` -```bash -VOL_PREFIX=/flash/gamutrf docker compose -f orchestrator.yml up mqtt gamutrf waterfall -d -``` - -### Deploy signal detector -```bash -VOL_PREFIX=/flash/gamutrf FREQ_START=2.4e9 FREQ_END=5.8e9 docker compose -f orchestrator.yml -f sigfinder.yml up mqtt gamutrf waterfall sigfinder -d -``` - -### Deploy Birdseye -```bash -VOL_PREFIX=/flash/gamutrf FREQ_START=2.4e9 FREQ_END=0 docker compose -f orchestrator.yml -f ../Birdseye/geolocate.yml up mqtt gamutrf waterfall geolocate -d -``` - -### Scan a single frequency with model -Ensure `torchserve*.yml` is configured with model and placed in the correct `$VOL_PREFIX/model_store` directory - -```bash -VOL_PREFIX=/flash/gamutrf FREQ_START=2.4e9 FREQ_END=0 docker compose -f orchestrator.yml -f torchserve-cuda.yml up mqtt gamutrf waterfall torchserve -d -``` - -### Scan across a frequency range with model -Ensure `torchserve*.yml` is configured with model and placed in the correct `$VOL_PREFIX/model_store` directory - -```bash -VOL_PREFIX=/flash/gamutrf FREQ_START=2.4e9 FREQ_END=5.8e9 docker compose -f orchestrator.yml -f torchserve-cuda.yml up mqtt gamutrf waterfall torchserve -d -``` - -### Scan across a non-linear range with model -Ensure `torchserve*.yml` is configured with model and placed in the correct `$VOL_PREFIX/model_store` directory - -Uncomment `--tuning_range` in gamutrf service in `orchestrator.yml` and adjust range as needed. `tuning_range` will override `freq_start` and `freq_end` - -```bash -VOL_PREFIX=/flash/gamutrf docker compose -f orchestrator.yml -f torchserve-cuda.yml up mqtt gamutrf waterfall torchserve -d -``` - -### Scan across freq with model feeding to Birdseye -```bash -VOL_PREFIX=/flash/gamutrf FREQ_START=2.4e9 FREQ_END=0 docker compose -f orchestrator.yml -f torchserve-cuda.yml -f ../Birdseye/geolocate.yml up mqtt gamutrf waterfall torchserve geolocate -d -``` - -## Running multiple radios on the same machine - -gamutRF supports the use of multiple radios on the same machine, whether for scanning or recording. When using multiple Ettus SDRs, assign a specific radio to a specific container by specifying the ```serial``` UHD driver argument. To list all connected Ettus radios, run ```uhd_find_devices```. Then add the ```--sdrargs``` argument to the specific container. For example, ```--sdrargs num_recv_frames=960,recv_frame_size=16360,type=b200,serial=12345678```. - -To use multiple radios in docker compose: in `orchestrator.yml` create a gamutrf container (gamutrf1, gamutrf2, etc.) for each radio. Each container should contain the `sdrargs` with the appropriate serial and incremented port values as seen below. -```bash -gamutrf1: -... - command: - - gamutrf-scan - - --sdrargs=num_recv_frames=500,recv_frame_size=16360,type=b200,serial=XXXXXXX - - --fft_zmq_port=10000 - - --inference_port=10001 - ... - ports: - - '9001:9001' - - '10000:10000' - - '10001:10001' - ... -gamutrf2: -... - command: - - gamutrf-scan - - --sdrargs=num_recv_frames=500,recv_frame_size=16360,type=b200,serial=XXXXXXX - - --fft_zmq_port=11000 - - --inference_port=11001 - ... - ports: - - '9004:9001' - - '11000:11000' - - '11001:11001' - ... -waterfall: - command: - - gamutrf-waterfall - - --scanners=gamutrf1:10000,gamutrf2:11000 -``` - -gamutRF also supports the KrakenSDR (which presents as five librtlsdr radios). You can run a scanner container for each radio, by adding a serial number - for example, ```--sdr=rtlsdr,serial=1000```. Use ```SoapySDRUtil --find``` to check the radios are correctly connected. - -## Manually initiating worker actions - -The orchestrator has a web interface on port 80. You can use this to command a worker to start an I/Q sample recording or start a RSSI stream. - -You can also call the worker API directly (for example, to use a stand-alone worker to make a recording or stream RSSI). - -For example, make a recording with a sample rate of 20.48e6, for 2 seconds (409600000 samples worth), at 100MHz: - -``` -$ wget -nv -O- localhost:8000/v1/record/100000000/409600000/20480000 -``` - -To stream RSSI values instead, call: - -``` -$ wget -nv -O- localhost:8000/v1/rssi/100000000/409600000/20480000 -``` - -If the sample count parameter is 0, the stream will not end -until a new RPC (whether rssi or record) is received. - -## Utility Functions - -Below are some utility functions for working with GamutRF and the collected IQ files. - -### Running GamutRF pipeline from IQ file - -gamutrf-offline can be used to run a previously recorded IQ file through GamutRF to simulate expected behavior of FFT, spectrogram generation, and model performance. To run in offline mode you will need to supply the relevant IQ file as either .sigmf-data file and the relevant .sigmf-meta file or the .zst or raw IQ file with the appropriately formatted naming convention. - -```bash -docker run -v $(pwd):/gamutrf/data -ti iqtlabs/gamutrf gamutrf-offline --tune-step-fft=512 --db_clamp_floor=-150 --fft_batch_size=256 --nfft=1024 "data/{source_file}.sigmf-meta" -``` - -### Working with worker I/Q recordings - -Workers make recordings that are compressed with zstandard, and are typically in complex number, int16 format, and include the center frequency and sample rate that the recording was made with. gamutRF tools can generally work with such files directly, but other tools require the recordings to be converted (see below). - -### Generating a spectrogram of a recording - -gamutRF provides a tool to convert a recording or directory of recordings into a spectrogram. For example, to convert all I/Q recordings in /tmp: - -```docker run -ti -v /tmp:/tmp iqtlabs/gamutrf gamutrf-specgram /tmp``` - -Use the ```--help``` option to change how the spectogram is generated (for example, to change the sample rate). - -### Reviewing a recording interactively in gqrx - -[gqrx](https://gqrx.dk/) is a multiplatform open source tool that allows some basic SDR operations like visualizing or audio demodulating an I/Q sample recording (see the [github releases page](https://github.com/gqrx-sdr/gqrx/releases), for a MacOS .dmg file). To use gqrx with a gamutRF recording, first translate the recording to gnuradio format (see above). Then open gqrx. - -* Select ```Complex Sampled (I/Q) File``` -* Set ```Input rate``` to be the same as the gamutRF sample rate (e.g. from the recording file name, -```gamutrf_recording_ettus_directional_gain70_1234_100000000Hz_20971520sps.raw```, set ```Input rate``` to 20971520, and also edit ```rate=``` in ```Device string``` to be 20971520) -* Set ``Bandwidth`` to 0 -* Edit ```Device string``` to set the ```file=``` to be the path to the recording. -* Set ```Decimation``` to None. -* Finally select ```OK``` and then ```play``` from the gqrx interface to watch the recording play. - -### Demodulating AM/FM audio from a recording - -gamutRF provides a tool to demodulate AM/FM audio from a recording as an example use case. - -* Use the ```airspyfm``` tool to demodulate audio to a WAV file. - -For example, to decode an FM recording which must be at the center frequency of a recording: - -```docker run -v /tmp:/tmp -ti iqtlabs/gamutrf-airspyfm -m fm -t filesource -c filename=/tmp/gamutrf_recording_gain70_1234_100000000Hz_2097152sps.raw,raw,format=FLOAT,srate=2097152 -F /tmp/out.wav``` - -Run: - -```docker run -ti iqtlabs/gamutrf-airspyfm -h``` - -To view other options. - -### API access - -gamutRF supports two separate APIs - for receiving scanner updates, and making scanner configuration changes. - -* scanner update API: described in [zmqreceiver.py](gamutrf/zmqreceiver.py). Receives real time scanner updates and config. -* scanner config API: allows RESTful updates to any CLI argument. Eg, ```wget -O- "http://localhost:9001/reconf?freq_start=1e9&freq_end=2e9"``` causes the scanner to reset and scan 1e9 to 2e9Hz. Any config change causes the scanner's gnuradio flowgraph to restart. diff --git a/docs/5-ML_TOOLING.md b/docs/5-ML_TOOLING.md deleted file mode 100644 index 382fa093..00000000 --- a/docs/5-ML_TOOLING.md +++ /dev/null @@ -1 +0,0 @@ -# RFML Tooling \ No newline at end of file diff --git a/docs/6-TESTING.md b/docs/6-TESTING.md deleted file mode 100644 index f6e1aeaa..00000000 --- a/docs/6-TESTING.md +++ /dev/null @@ -1,21 +0,0 @@ -# GamutRF Testing - -## Scanner testing - -Currently, the scanner ```gain``` must be set manually for the general RF environment (e.g. noisy/many signals versus quiet/few signals). -To establish the correct values and to confirm the scanner is working, initiate a scan over the 2.3-2.6GHz range. As the 2.4GHz spectrum is very busy with legacy WiFi -and BlueTooth, the probability of seeing signals is high. If in an environment without BlueTooth or WiFi, an alternative is the FM broadcast band (88MHz to 108MHz). - -To begin, commence scanning with just the scanner and waterfall containers: - -``` -VOL_PREFIX=/tmp FREQ_START=2.3e9 FREQ_END=2.6e9 docker compose -f orchestrator.yml up gamutrf waterfall -``` - -Browse the waterfall container's webserver port (by default, localhost:9003). It should contain strong signals similar to this example: - -![2.4G example](imgs/fft24test.png) - -If no or only small peaks appear which are not marked as peaks, increase ```gain``` (e.g., from 40 to 45) until peaks are detected. - -If no peaks appear still, check antenna cabling, or choose a different scan range where signals are expected in your environment. \ No newline at end of file diff --git a/docs/7-TROUBLESHOOTING.md b/docs/7-TROUBLESHOOTING.md deleted file mode 100644 index 77537862..00000000 --- a/docs/7-TROUBLESHOOTING.md +++ /dev/null @@ -1,38 +0,0 @@ -# Troubleshooting - -Instructions for troubleshooting common errors on GamutRF. If your error is not covered here please create an issue so the team can address it and update the documentation. - -## Common Errors - -### SoapySDR errors allocating buffers - -Run ```echo 0 > /sys/module/usbcore/parameters/usbfs_memory_mb``` as root before starting the scanner(s). - -### Containers won't start using Ettus SDRs - -#### ```[ERROR] [USB] USB open failed: insufficient permissions``` - -Ettus SDRs download firmware and switch USB identities when first powered up. Restart the affected container to work around this (if run with docker compose, restart will happen automatically). - -#### ```[ERROR] [UHD] An unexpected exception was caught in a task loop.The task loop will now exit, things may not work.boost: mutex lock failed in pthread_mutex_lock: Invalid argument``` - -UHD driver arguments ```num_recv_frames``` or ```recv_frame_size``` may be too high. The defaults are defined as ETTUS_ARGS in [utils.py](gamutrf/utils.py). Try reducing one or both via ```--sdrargs```. For example, ```--sdrargs num_recv_frames=64,recv_frame_size=8200,type=b200```. - -#### ```[ERROR] [UHD] EnvironmentError: IOError: usb rx6 transfer status: LIBUSB_TRANSFER_OVERFLOW``` - -Stop containers, and reset the Ettus as follows: - -``` -$ /usr/lib/uhd/utils/b2xx_fx3_utils -D -$ /usr/lib/uhd/utils/b2xx_fx3_utils -U -$ /usr/lib/uhd/utils/b2xx_fx3_utils -S -``` - -### Scanner with Ettus SDR shows implausible low power at approx 100MHz intervals - -Ettus radios periodically need extra time to produce good data when being retuned rapidly by the scanner. Increasing the value of ```--db_clamp_floor``` will cause the scanner to discard windows after retuning (effectively waiting for the retune command to be executed and produce good data before proceeding). - -### "O"s or warnings about overflows in SDR containers - -* Ensure your hardware can support the I/Q sample rate you have configured (gamutRF has been tested on Pi4 at 20Msps, which is the default recording rate). Also ensure your recording medium (e.g. flash drive, USB hard disk) is not timing out or blocking. -* If using a Pi4, make sure you are using active cooling and an adequate power supply (no CPU throttling), and you are using a "blue" USB3 port. \ No newline at end of file diff --git a/docs/8-DEVELOPMENT.md b/docs/8-DEVELOPMENT.md deleted file mode 100644 index 324075e4..00000000 --- a/docs/8-DEVELOPMENT.md +++ /dev/null @@ -1,197 +0,0 @@ - -# Development - -The best approach for doing development is to make changes within the repo and then build a new version of the GamutRF Docker container. To do that, run this docker command in the main directory of the repo: - -```bash -docker build -t iqtlabs/gamutrf:latest . -``` - -Since it is tagged with **iqtlabs/gamutrf:latest** the docker-compose file (orchestrator.yml) will use the local image you just built instead of the one from docker hub. If you want to revert back to using the docker hub version simply run: - -```bash -docker rmi iqtlabs/gamutrf:latest -``` - -This will of course generate lots of container images. To clean up old images, occasionally run: - -```bash -docker system prune -``` - -### gr-iqtlabs Development - -If you are looking to make changes to [gr-iqtlabs](https://github.com/IQTLabs/gr-iqtlabs) and then test them inside GamutRF, the following approach should work. The gr-iqtlabs libraries are pulled in and built in a base Dockerfile, so some changes are needed. - -First, go and clone a local copy of the **gr-iqtlabs** repo. Make any changes you want. Because the **gr-iqtlabs** folder needs to be in the build context for the GamutRF base Dockerfile, you need to do some weird stuff. - -In the **gamutrf** repo open the `docker/Dockerfile.base` file. Line 31 `RUN git clone https://github.com/iqtlabs/gr-iqtlabs -b 1.0.76` can be commented out. - -Line 32 `COPY --from=iqtlabs/gamutrf-vkfft:latest /root /root/gr-iqtlabs` needs to be updated to copy the files from your local copy of the **gr-iqtlabs** repo instead. The build will actually be done in the **gr-iqtlabs** folder, so make this change. -```docker -COPY . /root/gr-iqtlabs -``` - -Now make the following call while in the **gr-iqtlabs** repo folder, updated to reflect the correct path to the **Dockerfile.base** file: - -```bash -docker build -f ../gamutRF/docker/Dockerfile.base -t iqtlabs/gamutrf-base:latest . -``` - -This will create a local version of the **gamutRF** base Dockerfile. - -After this is done you will need to create a new version of the local gamutrf:latest image. Go back to the **gamutrf** repo directory and run the following command: - -```bash -docker build -t iqtlabs/gamutrf:latest . -``` - -Startup the `docker compose up` command and your changes to **gr-iqtlabs** should be included. - - - -## Dockerfile structure - -- Dockerfile: builds the **gamutrf** container -- docker/: The Dockerfiles used to **build** the containers used in GamutRF - - Dockerfile.base: creates the build containers that are used along with the base container. - - Dockerfile.driver: build container for the different Soapy Drivers used to support the SDRs - - Dockerfile.uhd-sr: build container for the UHD sample recorder - - - -# Code Structure - -- gamutrf Docker Container (./Dockerfile) - - gamurtrf/scan.py -> gamutrf-scan (/gamutrf/scan.py): parses all of the arguments - - gamutrf/grscan.py: - - -# grscan - - - __init__() - - find freq_range (end - start) - - find fft_rate (samp_rate / nfft) - - sets the *tune_step_fft*, which will default to *nfft* which is 2048 - - creates the source by calling **get_sources()** which is in **gamutrf/grsource.py** - - sets up different SDR sources - Soapy, Ettus, file, etc - - calls **get_fft_blocks()** - - **get_offload_fft_blocks()** - - loads one of 3 FFT blocks: wavelearner for AIR-T, IQT Labs vkFFT block or GR FFT_VCC block - - **get_pretune_block()** - - if pretuning is used it will use the IQT Labs **retune_pre_fft()** block, otherwise it will use the GR stream_to_vector block - - Adds **get_dc_blocks()** before the FFT block - - if the *dc_block_len* variable is set, it will return the GR **dc_blocker_cc** block - - Adds **get_db_blocks** after the FFT block - - Adds a series of GR blocks to scale the output of the - - if **write_samples** variable is set, it will add the IQT Labs **write_freq_samples()** - - Adds the IQT Labs **retune_fft()** block from the **gr-iqtlabs** repo to the end of the FFT set of blocks - - Adds a ZeroMQ sink to the end of the FFT set of blocks - - If the *inference_output_dir* variable is set it creates an IQT Labs **image_inference** block from the **gr-iqtlabs** repo - - If the *mqtt_server* variable is defined it will add an **inference2mqtt** block from the **gamutrf/grinference2mqtt.py** file - - if *pretune* variable is defined it will wire up the message ports - - if *pretune* is enabled, the **retune_pre_fft** block will be connected to the source and the **retune_fft** block - - if not, the **retune_fft** block is connected to the source - - it then connects the blocks together using a mix of standard GR and a function which pulls from list of blocks **connect_blocks** - - The construction flow graph should be: - - - SOURCE **Tuneable File Source** or **Ettus** or **Soapy Source** - - *DC Block* - - FFT **wavelearner** or **VKFFT** or **FFT_VCC** - - *retune_pre_fft()* pretuning - - DB Block - FFT scaling - - **retune_fft** IQT Labs gr-iqtlabs - - Attached to **retune_fft** - - - Inference Blocks - - **image_inference** - - *inference2mqtt* - - - - Zero MQ Pub Sink - - **zeromq.pub_sink()** - - - Writing the captured samples to file - - *write_freq_samples()* - - -# retune_pre_fft - -**part of gr-iqtlabs repo** - -## Code Structure - -- **general_work()** - - **get_tags()** is in base_impl.cc, it goes through the received tags and pulls the tags that match along with the time and puts them in the array - - if no tags were received, goto **process_items** - - else pull out the first Freq/Time from the tags and then goto **process_items** - - if you are not in *low_power_hold* or *stare_mode* - - if *pending_retune_* then reduce it by 1, store some state stuff - - -- **process_items()** - - loop through all of the samples, incrementing by the size of the FFT (*nfft_*) - - if you are getting non-zero packets, and you have not hit the target number of packets, copy 1 NFFT worth of packets to output - - call **need_retune_()** in **retuner_impl** to keep track of how many FFTs have been processed is greater than *tune_step_fft_* - - if it is, signal that a retune is needed - -# retune_fft - -**part of gr-iqtlabs repo** - -- **general_work()** - - - check if the output buffer is NOT empty, move what ever is there to a leftover buffer, then delete stuff 🤷 - - call **process_tags_** - - get all the tags, pull out the ones we like by calling **get_tags()** from **base_impl.cc** - - pull info from tags and propigate if opproriate - - call **process_items_()** - - since this is FFT values, collect the max, to find if the Ettus radio is in a low power mode after retuning. This lets you know if the retuning has completed and you do not have to wait a static amount of time ( this was previously done with the *skip_fft_count* ) - - loop through an copy over *nfft* worth of samples at a time - - once we hit the target number of FFTs (the amount we should dwell at that tuning) tune to the next increment - - then call **process_buckets_** - - sees if there is a new freq, and sets up a new file - - calls **write_bukets_** which writes all of the FFTs? 🤷 to a file - - -# image_inferece - -**part of gr-iqtlabs repo** - -## Code Structure - -- **general_work()** - - get all the tags, pull out the ones you want - - propigate tags if you have them - - if there are any JSON objects from processed inference, pop them from the queue and write them to *output* - - call **process_items_()** - - go through all of the items and generate rows of values - - when you have the targetted number of rows call **create_image()** - - if there are enough points above a certain level then add the image/meta to *inference_q_* - -- **background_run_inference_** - - This is runs in a thread that gets started up when the block is created - - it is a simple loop that sleeps and calls **run_inference_()** - - while the *inference_q_* is not empty go through this: - - pop an item from the queue - - build a JSON object with the metadata - - do some light flips on the image to get it in the right orientation - - send an HTTP request to Torch Serve - - get the result, which should have the detections - - call **parse_inference_()** to overlay the detections - - call **write_image_** to create a file with the detections overlayed - - - - - - -# Notes -- wavelearner: this is the FFT library used for the AIR-T SDR -- low_power_hold_down: -This is the infamous ettus low power workaround. The ettus driver immediately acks and tags a retune command (well almost immediately) - -However even after the tag it continues to emit samples that are for the old frequency. Then, the ettus enters a state where it outputs all zeros, then it starts emitting samples again at the new frequency - -We send a retune and immediately enter holddown, discarding samples. We wait until we see all zeros. Then we resume sending samples again \ No newline at end of file