Resumen de Towards large-scale and collaborative spectrum monitoring systems using iot devices
The Radio Frequency (RF) Electromagnetic (EM) spectrum is a scarce, precious and widely used resource for many different tasks and purposes in our society. For instance, the communication between the robots on Mars and the Earth is performed over the radio EM spectrum by sending radio waves. Location systems such as Global Positioning System (GPS), that is extensively used in aircraft, cars or smartphones, make use of the radio EM waves. Public broadcast transmissions such as radio/television or even everyday devices as Bluetooth peripherals, wearables and surveillance cameras use the radio EM waves to transmit and exchange information. Mobile data traffic, that increases every day, normally uses wireless communication technologies to provide content to the users. In other words, EM spectrum is used everywhere.
In the 1980s, the only concern for spectrum management was mostly about radio/television broadcasting and military communications. This is rapidly hanging today. We are in the age of the Internet-of-Things (IoT) where more and more devices are connected to the Internet sending and exchanging information using the radio waves.
We expect to see more than 20 billion inter-connected devices by 2020 and more than 75 billion by 2025. All of them will make use of the EM spectrum by using different RF technologies such as WiFi, Long Term Evolution (LTE), 5G, Bluetooth, Long-Range (LoRa), SigFox, etc. in order to be connected to the Internet and send information. In addition, mobile phones are becoming the main device used for multimedia content consumption which wireless data usage has increased very quickly. These mobile devices will take ad vantage of the fifth generation (5G) of cellular mobile communication, which is expected to provide high wireless network capacity, up to 100 times more than actual networks.
RF spectrum is becoming a limited resource and its use is fragmented, bursty and very diverse. Monitoring the spectrum becomes a complex task, as it requires a very dense information in frequency, time and space. Radio communications are and will remain essential for society, yet the traditional approaches for monitoring the spectrum use very expensive and bulky equipment. The latter disables the possibility to deploy large-scale deployments running 24/7 to get a better understanding of the EM space. A new spectrum monitoring paradigm is needed to sense the EM waves, keep safe radio communications and make sure that RF technologies are regulation compliance.
Traditionally the governmental agencies and international organizations (such as the International Communication Unit, ITU) are responsible for regulating the usage of the radio EM spectrum. Each country creates and maintains its own national frequency allocation plan which describes how the EM spectrum shall be used. Despite the EM spectrum is well-organized in terms of frequency allocation, technologies and organizations which can use it, its actual usage in different geographical places and times is not well-known at all. Today’s spectrum measurements are mainly performed by governmental agencies and telecommunication companies which drive around using expensive and specialized hardware. This spectrum monitoring approach does not allow to create a well scaled infrastructure to cover the pervasive deployment of wireless networks. Therefore the research community has recently focused on low-cost Software Defined Radio (SDR) for sensing the spectrum, such as the RTL-SDR. A large-scale RF spectrum monitoring system in real time is desirable for a multitude of practical reasons. Storing spectrum data at large scale can provide a historical spectrum database that allows to get a better understanding and knowledge about how the EM spectrum is used. Analyzing the EM spectrum in real-time at large-scale and 24/7 is challenging, but can offer a wide-range of possibilities to build smart applications to detect anomalies or non-authorized transmitters in specific bands. The upcoming smart and agile radios, cognitive radios could use spectrum data knowledge to detect what spectral resources are unused and exploit them to provide high throughput and better services. From the cyber-security point of view, it is important to monitor the spectrum and protect it from attacks whose aim is to destabilize the communications in a country and negatively impact its economical opportunities.
In the recent years, the idea of distributed spectrum monitoring has gained attention to monitor and capture the real-time usage of the RF spectrum at large geographical scale. Several platforms use expensive and specialized hardware for sensing the spectrum at large scale such as DARPA’s Spectrum Challenge and Microsoft Spectrum Observatory. Other platforms such as Google Spectrum Database, BlueHorizon or KiwiSDR are application oriented (e.g. TV white spaces, amateur radio 0-30 MHz) and cannot be used for other purposes. Other works such as SpecNet, SpectrumSense or RadioHound propose the use of commodity hardware for sensing the spectrum to analyze its usage, transmitter coverage estimation, etc. However, the previous works make use of expensive equipment to monitor the spectrum, and they are all limited to applications in which power spectrum measurements or spectrum usage are sufficient.
This thesis presents ElectroSense, a crowdsourcing and collaborative system that enables large scale deployments using IoT spectrum sensors to collect EM spectrum data which is analyzed in a big data infrastructure. The ElectroSense platform seeks a more efficient, safe and reliable real-time monitoring of the EM space by improving the accessibility and the democratization of spectrum data for the scientific community, stakeholders and the general public. In this work, we present the ElectroSense architecture, and the design challenges that must be faced to attract a large community of users and all potential stakeholders. ElectroSense also allows users to remotely decode specific parts of the radio spectrum using low-cost IoT devices as radio sensors. It provides a peer-to-peer communication among clients and sensors to exchange information and make the system scalable (a centralized approach would cause large latency and high load resources in the backend). The system architecture allows to decode any wireless signal that is within range of the sensors. ElectroSense contains several publicly available decoders that are not intrusive to the privacy of the wireless users. Our decoders operate on the sensor-side and have optimized their computational performance to run in embedded and low-cost IoT devices. The decoders that are currently implemented are Amplitude Modulation (AM) and Frequency Modulation (FM) for radio; Automatic Dependent Surveillance - Broadcast (ADS-B), Aircraft Communication Addressing and Reporting System (ACARS) and Automatic Identification System (AIS) for tracking systems; and LTECell for LTE signals.
ElectroSense is a crowdsourcing network that measures and analyzes the EM spec trum in real time using small-factor and low-cost IoT RF sensors, yet also supporting more expensive spectrum sensors such as Ettus boards. In this part, ElectroSense network and its capabilities are introduced in detail. ElectroSense architecture allows to collect spectrum information from the sensors and send it to the backend. Several algorithms are executed in the backend on the collected spectrum information providing data already processed to the user. Users can remotely control IoT spectrum sensors of the ElectroSense network to decode specific parts of the radio spectrum (broadcast and control signals) in real time through the Internet. We also propose the use of a virtual accounting system to first incentivize users to host ElectroSense sensors and second, regulate the access to the sensors in a fair manner. We propose a fast and precise frequency offset estimation (integrated in ElectroSense), for SDR platforms that makes use of LTE signals as a reference to determine the inaccuracies of the low-cost SDR receivers.
It is envisioned that a large number of sensors deployed in ElectroSense will be at affordable cost, supported by more powerful spectrum sensors when possible. Although low-cost RF sensors have an acceptable performance for measuring the EM spectrum, they present several drawbacks (e.g. frequency range, Analog-to-Digital Converter (ADC), maximum sampling rate, etc.) that can negatively affect the quality of collected spectrum data as well as the applications of interest for the community. In order to counteract the above-mentioned limitations, we propose to exploit the fact that a dense network of spectrum sensors will be in range of the same transmitter(s). We envision to exploit this idea to enable smart collaborative algorithms among IoT RF sensors. In this thesis we identify the main research challenges to enable smart collaborative algorithms among low-cost RF sensors. We explore different crowdsourcing and collaborative scenarios where low-cost spectrum sensors work together in a distributed manner.
Monitoring the spectrum at large-scale, in real time and making use of the crowdsourcing approach is challenging for several reasons ranging from the use of non-expensive and inaccurate hardware to fuse and reconstruct spectrum information in the backend. Main challenges are described below:
- Real-time spectrum acquisition with low-cost IoT RF sensors. Commodity and low-cost hardware has much more constraints in terms of computational calculation, memory and their RF front-end are limited in terms of sampling rate, frequency bandwidth, dynamic range which restrict the effectiveness of a spectrum monitoring system.
- The need for a fine time and frequency synchronization. A time/frequency fine synchronization is required in order to use In-phase & Quadrature (I/Q) data from multiple RF sensors for collaborative applications. This is not trivial to achieve with low-cost spectrum sensors that are distributively deployed and connected over the Internet.
- Control of the uplink network bandwidth of the sensors. In a crowd-sourced platform, the participation of the users is essential. Users may deploy the sensors at a location with a limited network bandwidth. The collection of I/Q (raw) spectrum information imposes a very large volume of data that most of the Internet connections may not handle properly. Smart algorithms and techniques are required to alleviate the network capacity needed for collaborative applications, and balance the uplink network load among RF sensors.
- Spectrum data fusion using low-cost IoT RF sensors. A large-scale deployment of sensors is an essential requirement for investigating collaborative strategies among them. Since the low-cost spectrum sensing devices have important hardware and computational limitations, fusing and reconstructing signals in the backend using spectrum data received from different RF sensors becomes challenging.
In this work, we propose novel solutions for the research challenges to enable the collaborative signal monitoring and decoding using commodity hardware as sensing receivers. We propose a fast and precise frequency offset estimation method for low-cost spectrum receivers that makes use of LTE signals broadcasted by the base stations. We have introduced a precise and fast frequency offset estimator for low-cost SDR devices connected to embedded boards. LTESS-track exploits the synchronization signals broadcasted by LTE base stations to determine the LO offset of the RTL-SDR devices. LTESS-track implements several key mechanisms not presented in other methods such as initial frequency offset compensation, up-sampling, sampling of data only in time proximity to the expected synchronization signal to reduce the computational cost and linear regression of samples. Our method is 10 times faster than the best open-source tools currently available, and is able to provide a new estimate every second.
We also focus on new Time-of-Arrival (ToA) estimation methods that can run on low-cost SDR receivers which are used for widely deployed sensor networks for collecting air traffic control messages such as the OpenSky Network, FlightAware or FlightRadar24. We compare our proposed methods with the state-of-the-art using real scenarios and real air traffic signals. We have shown that such algorithms can achieve ToA estimates with nanosecond-level precision even with real-world signals captured with the cheapest SDR hardware that is currently available, namely RTL-SDR. A closer look at the test results reveals that the main limiting factor for the achievable ToA precision with RTL-SDR is the limited dynamic range, resulting in a large fraction of packets being clipped or drowned into quantization noise.
We propose the collaborative approach among IoT spectrum sensors in order to enable collaborative narrowband decoding capabilities considering the limited network uplink bandwidth of the sensor’s connection. We describe a distributed system architecture for collaborative radio signal monitoring and decoding that builds on SDR receivers, propose a time multiplexing approach for sampling the spectrum and address the strict time synchronization required among sensors to efficiently optimize the network bandwidth usage and reconstruct the signal. We have proposed sampling methods on the sensor side and synchronization techniques on the backend side in order to align, at the sub-microsecond level, the signals received from multiple sensors connected over the Internet when traditional approaches, such as discipline the Local Oscillator (LO) of the RF frontend, are not an option due to the hardware limitations. Our approach can reconstruct the signal based on raw I/Q samples received by different low-cost sensors, all in range of the same transmitter. We have provided an evaluation with real LTE and Mode S signals and shown the feasibility to reconstruct and decode signals in a crowdsourcing scenario with low-cost sensors.
In the last part of the thesis we present a methodology to enable the wideband signal reconstruction in the backend using non-coherent receivers. We propose a method for collaborative wideband signal decoding that exploits the idea of multiplexing in frequency a certain number of non-coherent receivers in order to cover a higher signal bandwidth that would not otherwise be possible using a single SDR receiver. Our results show that is feasible to use low-cost and distributed non-coherent spectrum sensors for decoding wideband signals, getting more than 80% of packets correctly decoded, and reaching nearly 100% of packets decoded when receivers share the local oscillator. Collaborative signal decoding using low cost IoT spectrum sensors has been successfully proved and can enable multiple applications for those existing systems that deploy low-cost receivers such as RTL-SDR at large scale.
In this thesis, we have presented ElectroSense, a collaborative and crowd-sourcing initiative for collecting and analyzing the EM spectrum by using low-cost RF IoT sensors as well as more expensive RF spectrum devices to monitor the spectrum at large scale. We have identified the main research problems to enable smart and collaborative algorithms among low-cost spectrum sensors, which are challenging to solve due to the limitations of their components, both in the RF front-end and in the embedded board capabilities. At the time of writing we are the first looking at the problem of collaborative signal reconstruction and decoding using I/Q data received from low-cost RF sensors. Our results reported in this thesis and obtained from the experiments made in real scenarios, suggest that it is feasible to enable collaborative spectrum monitoring strategies and signal decoding using commodity hardware as RF sensing sensors.
We have proposed ElectroSense, a crowdsourcing framework for spectrum monitoring based on 3 main ideas: i) a sensing architecture that provides different spectrum data pipelines, ii) low-cost and software-defined IoT spectrum sensors to enable large-scale deployments, and iii) signal processing performed in the big data architecture. By using ElectroSense, we have successfully proved that complex tasks such as ToA estimation or collaborative signal decoding are affordable using very simple and low-cost RF receivers thanks to the signal synchronization mechanisms and algorithms proposed in this thesis.
