EPIC: Refactoring low-level GR 4.0 API

[ivan-cukic]

• [0pt,18pt] New GR 4.0 low-level API design #43
• [0pt,6pt] Clean up the prototype code for presenting to wider GR audience #45
• [0pt] intermediate step: make graph-prototype pre OP-usable for FAIR/OpenDigitizer-UI #46
• [0pt] Core Implementation
• [0pt] Benchmarks Implementation

[RalphSteinhagen]

Elimination of RxCpp as a Core OpenCMW/OpenDigitizer Dependency

The original OpenDigitizer concept involved two types of signal processing:

1. sample-by-sample-based, predominantly in the low-level front-end/services, and
2. processing of chunked-data, which was done both in the low-level service and notably also at the UI level.

GNU Radio (GR) has always been a good match for the first type of signal processing because it supports a feature-rich suite of high-performance C++-based signal processing blocks and algorithms that can be both programmatically and graphically defined through a flow-graph-type representation that nicely bridged the gap between high-performance C++ developers and graphical interaction for domain experts with insufficient C++ expertise. However, with GR 3.10 and earlier versions, this type of interaction was limited to sample-by-sample processing.

Therefore, the second type of post-processing related to chunked data (e.g., transient-recording and/or multiplexed signal like) needed to be implemented through a 'DataSet'-based processing/transform akin to those found in modern I&M devices. This has been prototyped in the Java-based Chart-FX and OpenCMW projects. Initial investigations found that the reactive pattern and it's C++ implementation RxCpp was a good match for this purpose, but required a graphical graph-based representation component similar to what already existed for GR. However, RxCpp became increasingly stale and not actively maintained nor modernised.

Meanwhile, FAIR was contacted by the GR leadership, who were open to accepting new ideas and requirements for the upcoming GR 4.0 version. This jump-started a productive conversation and a graph prototype, and eventually, a commitment to integrating these as well as modern C++ concepts as part of the upcoming GR 4.0.

The proposed improvements to GR, notably the new type-strict circular buffer, unlock a slimmer API due to the use of modern C++20 concepts. This, in turn, improved the performance of the buffers by 10 to 100-fold and the basic block processing through the introduction of SIMD-based vectorization and parallelization, compile-time merging of nodes, design/preference of 'composition to inheritance,' and minimization of unnecessary virtual inheritance to the absolute necessary. Notably, the type-safe buffers also enable arbitrary types, including the DataSet concept, to be transmitted via these buffers.

As a result, the need for the flow-graph API for RxCpp and a lot of other related boilerplate code, and compatibility with gr companion for both sample-by-sample and chunked-data-based processing were eliminated and are now covered by the targeted new GNU Radio 4.0 implementation.

This thus closes/modifies the following issues:

• to be closed [total: 15SP]:
  [2pt] Investigate suitability of the RxCPP testing framework for use in OpenCMW opencmw-cpp#195
  [8pt] Implement RxCPP type-erased nodes and the dynamic pipeline loader opencmw-cpp#196
  [5pt] Implement compile-time meta-information for RxCPP pipelines opencmw-cpp#198

• to be modified: replace RxCpp through GR 4.0 in the following issues [total: 8SP]:
  [3pt,0pt] Evaluate approaches for GR4-flowgraph integration with the OpenCMW Disruptor opencmw-cpp#199
  [5pt, 0pt] Implement gr4-flowgraph integration with the OpenCMW Disruptor opencmw-cpp#200