Special features for the Earth-Venus-Earth experiment

Author: Wolfgang Büscher, DL4YHF .
Thanks to Karl Meinzer (DJ4ZC), Achim Vollhard (DH2VA), and Markus Vester (DF6NM) for their explanations.

Date: 2007-07-26

< Still Under Construction ! >

Contents

  1. Introduction
  2. Setting up SpecLab for the experiment
  3. Synchronisation of receive / transmit periods
  4. Theory of operation
  5. Generation of a "test signal"
  6. Analysing I/Q wave files (recorded with SpectraVue)
  7. Real-time signal analysis with SDR-IQ and Spectrum Lab
  8. RX/TX control (inside the Spectrum Lab application)

Introduction

In 2007, some new features were implemented in Spectrum Lab for an experiment with the 20-meter parabolic dish antenna at the IUZ in Bochum. The principle, as explained by DH2VA (refined by Karl Meinzer a bit later) :

  1. Transmit a signal on 10 GHz towards Venus for 5 minutes. Estimated parameters:
    Antenna gain = 63.5 dBi, transmitter output power =  600 W (on 10 GHz),
    EIRP = 600 W * 10^(63.5/10) = 1.34 GW .
  2. Switch from transmit to receive, and wait for the (theoretic) arrival of the signal. Parameters:
    noise temperature = 120 K, antenna gain = same as on transmit .
  3. Receive for 5 minutes, analyzing the signal with an FFT bandwidth of 40 Hz (or so), and add all FFTs within this period to reduce the noise (and increase the signal-to-noise ratio),
  4. Plot a single line on the waterfall screen (spectrogram), representing the spectrum of the 5-minute reception cycle,
  5. repeat the above steps over and over, until a sufficient number of FFTs is captured.

In addition to the above, the average of ALL lines in the spectrogram buffer (= ALL FFTs since the start of the experiment) is calculated, and displayed as a curve in the spectrum graph window. This should result in an additional noise reduction (or, more precisely, flattening of the noise floor in the spectrum), so we hope (!) that, after a sufficient integration time, the reflected signal will become detectable in the long-term spectrum graph. The expectable signal-to-noise ratio -under optimum conditions- is explained in a later chapter.

There are, of course, a few obstacles :

The following chapter describes how Spectrum Lab could (!) be used for this experiment.

Setting up Spectrum Lab for the experiment (initial version, already superseeded)

There will be a special configuration file in the CONFIGURATIONS directory, called EVE-IQ-44kHz.usr (for the soundcard sampling at 44.1 kHz). To analyze I/Q samples directly received with SDR-IQ, or audio files saved with SDR-IQ / SpectraVue, use EVE-IQ-5kHz.usr instead.

There is also a  fast-running "simulator" for this experiment (EVE_Simulator.usr ). It uses a synthetic test signal instead of the microwave receiver which will be used in the experiment later. While the simulator runs, look at the red line in the spectrum graph - it is the long-term average spectrum, which should (if all works as planned) slowly creep out of the noise, as more and more signal energy is accumulated. Details about the theory, and the test signal generator settings follow in the chapter Generation of a Test Signal.

To load one of these files, use "File".."Load Settings" in the main menu. Alternatively, you can add this configuration to the "Quick Settings" menu ("Quick Settings"..."Load and Create User Defined Entries"). This allows you to switch quickly between different configurations. All configurations for the Earth-Venus-Earth experiment begin with the prefix "EVE_" .

If you want to modify the configuration, or write your own configuration instead of using one of the preconfigured "EVE"-settings, here are the necessary steps:

  1. Set the display in the main window to combined display mode (which means, the spectrum graph in the upper part, and the waterfall (spectrogram) in the lower part of the window: Right-click into the waterfall or spectrum graph area to open a context menu, and turn on "Waterfall" and "Spectrum Graph" if not already active.
  2. Turn on the "Long-term Average Graph" in the Spectrum Graph window:
    Right-click into the Spectrum Graph area to open a context menu, point at "Long term average spectrum", and click on "show Long-term average graph" if not already checked (there is a checkmark near this menu item, which is set if the option is already active). On this occasion, you can also turn on the Auto Range function (in the same menu under "Spectrum Graph Options". This will automatically switch the amplitude scale in the spectrum graph for best visibility, which helps a lot as the long-term integration proceeds.
  3. Select a suitable sampling rate for the soundcard, at least two times the maximum audio frequency of the radio. For a typical "SSB" receiver, 11 kHz is sufficient. If you plan to record the "raw audio" while the experiment is running, don't make the sampling rate larger than required because you may get increadibly large wave files ;-). How to:
    In the main menu, select "Options"..."Audio Settings"..."Sample Rate (nominal)".
    (ToDo: Modify this step as soon as the SDR-IQ downconverter is directly supported)
  4. Select a suitable FFT size, which results in an effective receiver bandwidth of 10...100 Hz (see considerations above; we may want to tell "stable carriers" from "Venus reflections" by looking at the spectrum a bit closer). Example: 11 kHz sampling rate, 1024-point FFT -> Width of one FFT-bin = 10.7 Hz, Equivalent noise bandwidth = 16.15 Hz (due to the Hann FFT window). The FFT settings can be accessed through the main menu, select "Options"..."FFT Settings".
  5. Check the "Display Settings, part 1":
    Set the option "Optimum Waterfall Average". This causes SpecLab to calculate a new FFT every few hundred milliseconds, which will be added to the first averaging stage. The internal FFT calculation interval is automatically set for a 50 percent overlap between to FFTs (if an FFT windowing function other than the "rectangle" window is used). For example, at 11 kHz sample rate, and an FFT length of 1024 points, a new FFT will be calculated every 0.5 * 1024 / 11025 Hz = 46 ms. The "momentary" FFTs will be frequently shown as a blue curve in the Spectrum Graph window (more precisely, this curve is drawn with "Pen number 2", if you have modified the colours...). More info about spectrum averaging can be found here (in the SpecLab help system).
  6. Next, set the Waterfall Scroll Interval to 30 seconds. This will produce about 10 lines in the waterfall during the 5-minute reception period, which allows us to quickly spot "problems" in the waterfall. This setting does not affect the SNR of the finally calculated spectrum (long term average), because the final result is the average of ALL calculated FFT's.

During an initial test in Bochum, it turned out that the soundcard wasn't up to the job as explained later . The soundcard was replaced with a digital downconverter called SDR-IQ (details in one of the next chapters).

Synchronisation of the transmit/receive periods

There are different possible ways to achieve synchronisation:

Theory of operation

As explained in the introduction, the expected SNR (signal-to-noise ratio) will be very low; for example -21.6 dB (*) in a 40 Hz (**) bandwidth. But how can such a weak signal be made visible, especially since it is incoherent, "40-Hz wide or more" ?

(*) estimated by DJ4ZC for an albedo of 1 percent (albedo = measure of diffuse reflectivity).
(**) 40 Hz is the expected bandwidth if most of the signal is reflected on the surface.

Longer FFTs longer to get additional gain (gc/dB = 10*log10 of the bandwidth ratio) doesn't help a lot here, because the reflected signal is not coherent (it is possible to use a longer FFT, but in that case the "FFT Smoothing" option must be turned on to match the overall filter to the signal). A 40-Hz receiver bandwidth already matches the expected signal bandwidth. The only chance is integrating the signal energies (in the FFTs) over a long time, to reduce the standard deviation in the "long term average spectrum" (which is the red graph displayed in SpecLab). Increasing the integration time (or "number of FFTs") reduces the standard deviation (sigma) by the square root of the number of FFTs.

This "gain" (gi, gain from incoherent integration) can be calculated in decibel as :
gi/dB = 10 * log10 ( sqrt( number_of:_FFTs ) )
Example: 1 million FFTs :
gi = 10 * log10 ( sqrt( 1000000 ) ) dB = 30 dB .

With the FFT window set to "Rectangular" in SpecLab, there is no overlap between the FFTs, and collecting data for an FFT with 40 Hz bin width takes 25 milliseconds.

Note: A soundcard running at 11025 samples/second, and a 256-point "real" FFT, give a 43 Hz FFT bin width, and 23 milliseconds time to acquire the samples for every FFT. The output of the simulation after three hours of integration is shown here . If the SDR-IQ is used (with 8138 samples/second), each of the 256 FFT bins will be 32 Hz wide, but the principle remains the same.

Aquiring the data for 500000 FFTs (gi=28.5 dB) with 40 Hz FFT bin width requires 3.5 hours, and 300000 FFTs (gi=27.4 dB) 2 hours of signal integration. This doesn't take the 50 percent RX/TX duty cycle into account yet !

Generation of a test signal

To check the proper function of the long-term spectrum average, a sufficiently "weak" test signal was created using SL's built-in test signal generator. The output "amplitude" of the noise generator in SL was set to -50 dB / sqrt(Hz), and one of the sine wave outputs to -55 dB (Note: all these dB-values are relative to "full scale"; 0 dB is the clipping point of the soundcard).

-50 dB noise in a 1 Hz bandwidth is equivalent to  (-50 + 10*log10(40Hz/1Hz) ) dB = (-50 + 16) dB = - 34 dB noise power in a 40 Hz bandwidth; so the sine wave is (55-34=) 21 dB "below the noise" in a 40 Hz receiver bandwidth. So far, so good... but how does it work in practise ? The following diagram shows the resulting spectrum after three hours of averaging:

The red diamond marks the frequency of the sine wave generator (in the simulator configuration).

Note: The "real" noise from the receiver will not look as good ("flat") as this one.. there will be some passband ripple which is not considered in this simulation !

Analysing I/Q wave files (recorded with SpectraVue or Spectrum Lab)

In older SpecLab versions, the SDR-IQ downconverter was not directly supported by Spectrum Lab. So the 5-minute receive periods had to be saved as stereo wave files (*); so they can be post-processed with Spectrum Lab. How to do that ....

  1. Load the configuration "EVE-SDR-IQ-5kHz" as explained in a previous chapter (for simplicity, add this configuration to one of the entries in the "Quick Settings"-menu).
  2. Stop the "Sound Thread" (in the main menu: Start/Stop) if the audio processing still runs in real time (using input from the soundcard, or the SDR-IQ converter).
  3. Clear the "Long Term Average Spectrum": right-click into the spectrum graph area, and select "Long-term average spectrum"..."Clear.." in the popup menu. This is important, otherwise you may be fooled by old data in the average buffers !
  4. Select all recorded files (or the "next recorded" file) for analysis:
    In the "File" menu, select "Analyse Audio File (without DSP)". There is an important trick how to select multiple files in a windows file selector box:
    - Click on the first file, then press and hold the SHIFT key ("Umschalttaste"), and select the last file .  
    OR :
    - Select multiple individual files while holding the CONTROL key ("Strg").
    Click "OK" in the file selector box when ready.
  5. A special dialog titled "File Analysis" is displayed, which shows some parameters of the first analysed wave file. Make sure the option "play/analyse this file in an endless loop" is not set. Then click "OK".
  6. The program will analyse the wave file now, much faster than it was recorded (that's the difference between the functions "Analyse File (without DSP)" and "Analyse and play file (with DSP)" ).
  7. When the new file (or all files) has been analysed, the analysis stops.
    You can select a new file now (as soon as SpectraVue has recorded a new one), or try the entire analysis again with modified parameters (different FFT size, with or without automatic gain control, with or without passband ripple equalisation, etc etc).
(*) Why use SDR-IQ, and not the soundcard ?
During a test in Bochum, we found that there were a lot of "spikes" in the 10 kHz region, caused by the frequency converter which drives the azimuth / elevation rotors. A typical EMC problem... which seriously affected the signal entering the soundcard. The SDR-IQ (an interesting and affordable software defined receiver / downconverter by RFSPACE) did not suffer from this problem, because it directly processes the 10.7 MHz IF and sends it via USB to the PC. So we decided to use it instead of the soundcard). At that time, we used the SpectraVue software to record the SDR-IQ's output in blocks of 5 minutes, with 37 kHz sampling rate.
In the meantime, Spectrum Lab directly supports SDR-IQ so the data can be analysed in real time. Also, the passband ripple at 5 kHz audio bandwidth (8.1 kHz sampling rate) can be used, which greatly reduces the size of the recorded audio files, and the CPU time required to analyse the data.

After analysing 3 hours of samples, or about 340000 256-point FFTs with 32 Hz bin width, the long-term average spectrum looked as shown below. Data were collected with SDR-IQ running at 8138 Hz sampling rate, fed with "real noise" from the microwave transverter. Only the Venus-signal was "simulated" with a stable RF generator; 21.6 dB below the noise in a 32-Hz FFT bin.

The red curve in this diagram is the long-term average. The readout cursors (red and green cross in the graph) mark the generator signal. The difference between (signa+noise) and noise in an FFT frequency bin (32 Hz) is 0.03 dB after three hours of integration. The standard deviation (sigma, here indicated as 0.0102 dB) is slightly higher than the theoretic value; but this is caused by the remaining passband ripple (which could be reduced further by using an improved compensation table).

Real-time signal analysis with SDR-IQ and Spectrum Lab

To reduce the CPU load, the passband ripple, and the file size, we will let the SDR-IQ run at f_sample = 8.1 kHz (which results in an observed bandwidth of 5 kHz). The optimum parameters were unknown at the time of this writing, so just a few short notes:

(*) Why should the USB communication "break down", and how to notice that ?
The reason may be a severe, but temporarily overload of the analysing PC. If that happens, the SDR-IQ stops flashing it's yellow LED rapidly, and the "input"-block in SL's circuit window turns red instead of green.
The SDR-IQ seems to monitor if data from the USB arrives periodically (not sure how yet, but there seems to be a watchdog-like "timeout monitor" in it).
Spectrum Lab alsoy monitors if data from the USB port arrive in regular intervals. If that doesn't work, it tries to establish communication again by sending a special command to the radio. If that fails, too, the "input-block" in SL's circuit window turns red instead of green. BTW, the colours in the circuit have the following meaning:
GREEN=active/ok; YELLOW=busy/waiting/being configured;  RED=error;  GRAY=passive.

RX / TX Control (inside the Spectrum Lab application)

To analyse (and record!) only the receive-phases during the experiment, one of the input signal on the RS-232 interface is used to sense the current RX- / TX state of the equipment. Inside the Spectrum Lab application, the programmable 'conditional actions' are used for that purpose. They permanently work "in the background" to control the following:

To see or modify the "conditional actions" which perform the tasks mentioned above, select "File".."Conditional Actions" in SpecLab's main menu. You will see a table like this (most likely, already changed a bit, because this was the author's very first attempt) :

Note:
You will find the "sourcecode" of these event-definitions in the "conditional_actions" folder, saved as EVE-RxTxControl.txt. You don't have to load them manually, because they are are also part of the SL configuration for the EVE experiment, at least in the file EVE-SDR-IQ-5kHz.USR . A general description of the conditional actions is here (in the SL manual) .

The proper function and the "progress" of the experiment can be checked on the programmable buttons on the left side of the main window. It shows (from top to bottom) the current date (YYYY-MM-DD), the current time (hh:mm:ss), the current time within a 5-minute RX or TX cycle ("Tp=2 min" means the current cycle is running for 2 minutes now; values above 5 minutes indicate a problem..).
Next, there is a programmable readout for the "effective voltage" within a certain frequency range (right-click on the button to edit it), and an indicator for the current RX / TX - state:

Programmable buttons in the EVE-configuration

The programmable buttons can be modified by clicking on them with the RIGHT mouse button. The LEFT mouse button invokes the programmed function (where applicable; some buttons are used for "display" only).

(*) To check if everything works "as it should", keep an eye on the "Veff"- and the "sigma" indicator during the RX-periods. Veff should remain constant (within a few percents of the initial value). If not, check if the AGC in the VHF receiver is turned off, and look outside .. there may be rain scatter on 10 GHz. The "sigma" value (standard deviation) should drop permanently, as more FFTs are added in the long-term average.
Sigma can be calculated as:
 sigma(dB) = 4.34 / sqrt(n) ,  where n is the number of FFTs added in the long-term average.

For example, with n=340000 (as in the "three-hour-test"), sigma should be 4.34 dB / sqrt(340000) = 0.0075 dB .

Minimizing the adverse effect of rain scatter (and similar "short disturbances")

If you observe rain scatter (or, something else causes the effective voltage to rises for a few minutes), write down the current time of day. The increase in broadband noise will most likely not be caused by Venus ! Later, when analysing all recorded 5-minute-chunks, you can skip that part of the recording. When analysing files, the files can be selected "individually" by holding the CONTROL key pressed in the windows file selector box. If you know the time of the rain-scatter event, you know the filename of the "unwanted recording", because the name of the wave file contains of the START-date and -time in the ISO8601 format. For example, EVE-070812_1700.wav is a file recorded in August 12th, 2007, on 17:00 "PC"-time.

In contrast of rain scatter (where an "unusually strong signal" would add unwanted energy into the average spectrum), an "overhead" could pass without scattered 10-GHz signals won't cause much harm, because the energy in the momentary FFTs would be weaker than stronger.

< To Be Continued >