Friday, August 31, 2018

RF Noise Antenna System

RF Noise Antenna System.

Block Diagram of RF Noise Antenna System


I Developed a rf noise monitoring system and there was a need to standardize on an active antenna for the system.

My thinking was sins most Remote Stations is Raspberry Pi with RTL-SDR dongles I felt developing a front end power injector with Galvanic isolation from active antenna with the necessary RF filters.
I also had a requirement to resolve the rtl-sdr dongle front end filter problem.
There is an Rf image on center frequency 14.2 Mhz.
I then Developed a Low and High band pass filters on the same PI HAT that can be switched via IO pins on Pi.

Here is the circuit diagram of my design with out the Active Antenna.


Raspberry Pi HAT with Filters,Isolation and Power injection

Filter design details. 

High Pass Filter

Low Pass Filter

I will work on an active Antenna board.

3d view of Board




Monday, June 11, 2018

How to fix Kicad PCB (pcbnew) cross cursor problem on Ubuntu 18.04

How to fix Kicad PCB (pcbnew) cross cursor problem on Ubuntu 18.04

When trying to create PCB you will get this cross cursor where ever the mouse moves.
Here is the steps that fixed my problem on Ubuntu 18.04 vanilla kicad install.

sudo add-apt-repository ppa:js-reynaud/kicad-4
sudo  apt-get update
sudo apt-get upgrade

Wala..

It is now working :-)


ref: http://kicad-pcb.org/download/ubuntu/

Wednesday, May 30, 2018

How to program and configure the Zastone D900 DMR radio.

How to program and configure the Zastone D900 DMR radio. (mototrbo)

Zastone D900 DMR (mototrbo)
Please not the programing cable is different to generic programing cables but microphone connections is the same as generic Microphones.

First Download the following application from TYT web
http://www.tyt888.com/?mod=download
Download MD-380
Go into unrared directory "MD-380 software\Firmware 2.30 and new programming siftware"
This file is also available here but could be old if  http://www.giga.co.za/Kit_Drivers/TYT_MD-380_20160908101439.rar

Then untarrar  MD-380(Tytera).rar
The directory then should look like this
UNRAR "MD-380(Tytera).rar"
After the unrar then run the MD-380(Tytera).exe


Continio installation using default settings.
After installation is complete there should be a Tytera MD-380

Run Application Tytera MD-380 in start menu
You should then see the following default screen.
Default Configuration screen
Now download the codeplug for South Africa Repeaters from Here http://www.giga.co.za/Kit_Drivers/Zastone_D900_Code_plug_v1_30_may_2018.rdt
Now load the code plug "Zastone_D900_Code_plug_v1_30_may_2018.rdt"

Load "Zastone_D900_Code_plug_v1_30_may_2018.rdt"
Now edit the DMR radio configuration as required.
Pres green button to download from Radio Red to load to Radio and Blou to save your configeration

Monday, May 21, 2018

How to Create a Linear Transponder using a RTL Dongle and HackRF with Gnu Radio.

How to Create a Linear Transponder using a RTL Dongle and HackRF with Gnu Radio.

I wanted to test the concept of a linear Transponder and how difficult would it be to build it with conventional components.
Hardware Configuration
Hardware required to build the Transponder.
  1. SDR IQ demodulator with rtl dongle (RF In)
  2. SDR IQ DSP processing system. Gnu Radio (DSP processing)
  3. SDR IQ Modulator. HackRF (RF Out)
  4. Background management system. (OBC) for additional telemetry.
Software required for Transponder.
  1. SDR IQ demodulator. rtl dongle (RF In)
  2. SDR IQ DSP processing system. Gnu Radio (DSP processing)
  3. SDR IQ Modulator. HackRF (RF Out)
  4. Background management system. (OBC)

Video of working Transponder.



Gnu Radio.


Transponder Block Diagram

RTL SDR Block.

RTL-SDR Source

Low Pass Filter Block.

 
Low Pass Filter Block

AGC Block.

AGC Block

Power Squelch.

Power Squelch

Osmocom Sink. (HackRF)

Osmocom Sink (HackRF)

FFT. (Spectrum View of input)

FFT Spectrum View

REF: Power point Slides at AMSAT SA

Ref:Source code is available on Github Version 4

Thursday, April 19, 2018

Setting up my Raspberry Pi as an BACAR Balloon telemetry system using GPSD,CW,AFSK,RPITX and Direwolf

Setting up my Raspberry Pi as an Balloon telemetry system using GPSD, CW, rpitx, AFSK and Direwolf.

 
Bacar Balloon

I wanted to take part in a local BACAR Balloon experiment and wanted to send my GPS position via the APRS network and also send a CW beacon every 7 minutes. I got an opportunity to add my payload to a school project.



Raspberry TX Hat for 2m Band

So I decided to use the the 2m transmitting board form Giga technology plug it on the raspberry pi. this should give me about 32.7mW (15.1dBm)
Power is a bit low but its a good for now.

Balloon Telemetry System


BACAR launch details is available here.
http://secundaweather.co.za/blog/?p=2012

Learnings
  1. Power to Low (Working on model 2 with a power output of 100 to 900mw output.
  2. Need to record everything that hapens in Local log so you can recover more information that you would send with Telemetry.
 Hardware.
  1. Raspberry Pi 3
  2. Giga Technology TX PI HAT for 2M
  3. RTL dongle for RX
  4. Giga Technology USB GPS unit.
  5. Giga Technology Battery and Powersupply Boards.
  6. Temperature sensor
  7. Optional USB Camera for CCTV.
High Level Hardware configuration

Software.
  1. CW (CW tone generation for Beacon transmission)
  2. Direwolf. (AX 25 telemetry generation for Temperature and on-board sensors  board 1200 AFSK)
  3. rtl_sdr (SDR receiver software for incoming commands controlling transponder)
  4. rpitx (RF transmitter details)
  5. ALSA loopback (this is needed to send audio between direwolf cw and rpitx)
  6. gpsd GPS positioning server read by direwolf APRS transmission
  7. Kal (rtl dongle frequensy calibaration details)
  8. csdr DSP libraries  that can be used for SDR.
  9. Some shell scripts taing this all together. Download from github https://github.com/antonjan/Raspberry_Telemetry

Software Block Diagram



Lets install the software for the system.

sudo apt-get install CW
sudo apt-get install direwolf
sudo apt-get install rtl-sdr
sudo apt-get install gpsd
sudo apt-get install alsa-utils
sudo modprobe snd-aloop
sudo apt-get instal python
Install the csdir sdr libraries doing the following.

git clone https://github.com/simonyiszk/csdr
cd csdir
make 
sudo make install
Get the direwolf example configurations
git clone https://github.com/wb2osz/direwolf
Lets install aprs utility.
got home directory
cd
git clone https://github.com/casebeer/afsk.git
cd afsk

sudo pip install afsk $ 
sudo pip install --allow-external PyAudio --allow-unverified PyAudio PyAudio
Lets test aprs util
aprs
You should see the following
usage: aprs [-h] -c CALLSIGN [--destination DESTINATION] [-d DIGIPEATERS]
            [-o OUTPUT] [-v]
            INFO
aprs: error: too few arguments


Lets check if alsa loop installed

sudo arecord -l
you should see this

card 2: Loopback [Loopback], device 0: Loopback PCM [Loopback PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 2: Loopback [Loopback], device 1: Loopback PCM [Loopback PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7



You cal also check it in the sound volume control

Right click on the Sound icon and select setting and you should see the following

Input Tab should have the loopback sound card


Output Sound Card should have loopback


Lets test CW

cw -h
You should see the help

Lets test gpsd

gpsd -h
You should see the gpsd help
Install the gps on USB port.

We now need to find out what is the usb port for your GPS device so we can run the following command.  sudo gpsd -D 5 -N -n /dev/ttyACM0
We assume here that your usb device is  /dev/ttyACM0
Run this command and then plug in and out your gps on usb port.
tail -f /var/log/syslog | grep ttyAC


You will the see something like this.

/dev/ttyACM0
Then run the command sudo gpsd -D 5 -N -n /dev/ttyACM0 replacing it with your GPS usb port.

gpsd:PROG: Changed mask: {ONLINE|TIME|LATLON|ALTITUDE|STATUS|MODE|PACKET|PPSTIME} with reliable cycle detection
gpsd:PROG: GPGSA sets mode 3
gpsd:CLIENT: => client(0): $GPGSA,A,3,08,27,16,09,,,,,,,,,4.12,1.97,3.61*0D\x0d\x0a
gpsd:PROG: Changed mask: {ONLINE|MODE|DOP|PACKET|USED} with reliable cycle detection
gpsd:PROG: Partial satellite data (1 of 4).
gpsd:CLIENT: => client(0): $GPGSV,4,1,14,01,64,012,08,03,05,014,,04,06,121,,07,69,274,18*72\x0d\x0a
gpsd:PROG: Partial satellite data (2 of 4).
gpsd:CLIENT: => client(0): $GPGSV,4,2,14,08,43,151,28,09,11,321,22,11,88,165,17,16,15,081,21*7E\x0d\x0a
gpsd:PROG: Partial satellite data (3 of 4).
gpsd:CLIENT: => client(0): $GPGSV,4,3,14,17,06,290,22,22,14,033,16,23,04,348,,27,14,129,08*7A\x0d\x0a
gpsd:CLIENT: => client(0): $GPGSV,4,4,14,28,16,230,11,30,40,230,21*75\x0d\x0a
gpsd:PROG: Changed mask: {ONLINE|DOP|SATELLITE|PACKET} with reliable cycle detection

There is also another utility that could give you the status of the gpsd service.

Ok lets make our gpsd configuration permanent.
Edit the following configuration file using your favorite text editor.
and change the usb interface to what was detected above

sudo Vi /etc/default/gpsd

# /etc/default/gpsd
START_DAEMON="true" 
GPSD_OPTIONS="-n" 
DEVICES="/dev/ttyAMA0
USBAUTO="false" 
GPSD_SOCKET="/var/run/gpsd.sock"

Lets restart the gpsd service

sudo systemctl stop gpsd.socket

sudo systemctl start gpsd.socket
You can now test if it works fine with the following command
sudo gpsmon
You should see the following if all is ok with gps connected.

This is the GPS details as read by GPSD














Lets test direwolf



direwolf -h
You should see the direwolf help screen in white inverse.


Ok lets check rtl_sdr. (the SDR receiving software)
Plug-in your rtl dongle and then run the following command.

sudo rtl_test

You should see some test results and gain parameters supported by your device.
Ok


Connect your ds18b20 temperature sensor.
One of the coolest things about OneWire and DS18B20 temperature sensors is that each sensor has its own embedded address so you can have many of them on 1 data wire. 




Configure the raspberry pi to read the temperature.
Edit the following config file
sudo vi /boot/config.txt


Add the following at the bottom of the file
dtoverlay=w1-gpio
Then reboot the pi
sudo reboot
We now need to load the device drivers for the temperature sensor.
sudo modprobe w1-gpio 
sudo modprobe w1-therm

Now check if the device was loaded

ls /sys/bus/w1/devices


you should see a directory like this below
28-000007602ffa
Go into this directory replacing the part in yellow with your directory
cd /sys/bus/w1/devices/28-000007602ffa
Now run the following command to get the temperature.
w1_slaveYou should see something lie this.
bd 01 4b 46 7f ff 03 10 ff : crc=ff YES
bd 01 4b 46 7f ff 03 10 ff t=27812

Ok its working.
We can now use a python script to read the readings
get the python code from github using the following command
git clone https://github.com/pimylifeup/temperature_sensor.git
goto the new directory that was created. temperature_sensor
cd temperature_sensor

Give the application executable writes by using the following command 
chmod 776  temperature_sensor_code.py

Then run the command
./temperature_sensor_code.py
You should see the temperature readings scrawling on the screen.
Ok we can stop it with CTRL C.
Ok now all seems to be working for the temperature readings



We now need to test the Radio transmitter.
Please note you need to have a Amateur Radio license to do this as you will be transmitting on the Amateur allocated frequencies.

Install the Raspberry Pi hat on the Raspberry Pi and then test the transmitter with the following commands.
Clone the following examples in your home directory from github
git clone https://github.com/antonjan/Raspberry_Telemetry.git

cd to the Raspberry_Telemetry directory.
Connect a 2m external antenna to the raspberry Pi Hat SMA connector.
run the following command
sudo ./Start_Carier_check.sh
Now check for a rf carrier on the following frequency
U can use a SDR dongle or HT (Baofeng) or spectrum analyzer if you have one.
The frequency can be changed by editing this file and changing the yellow value
vi ./Start_Carier_check.sh
sudo rpitx -m VFO -f 145300 -c1
Now that we have the transmitter tested we can start testing the telemetry.




Lets start doing some real time applications.

Now lets configure the APRS and CW telemetry to send sensor data and call sign. I suggest using 70cm up-link and 2M down-link not to overdrive the receiver front end. ( There is a diplex available from Giga technology)  
2m and 70cm Diplexer

  1. Sending APRS telemetry every minute with GPS and temperature readings.
  2. Sending a CW (morse code) with calsign and temperature reading every two minutes.
  3. Lets test a command to control the system. Replying with APRS got command.
  4. I wanted to send telemetry every minute but decided to make it a prime number just in case there is two telemetry transmissions disturbing each other so this would be helping to resolve this problem. (67 seconds)

Sending APRS telemetry every minute with GPS and temperature readings.


Download the following shell scripts
1) wget https://raw.githubusercontent.com/antonjan/Raspberry_Telemetry/master/sh/cron_aprs_gps_position.sh

2) wget https://raw.githubusercontent.com/antonjan/Raspberry_Telemetry/master/sh/crontab_bacar_cw.sh
3) wget https://raw.githubusercontent.com/antonjan/Raspberry_Telemetry/master/sh/crontab_bacar_aprs.sh
4) wget https://raw.githubusercontent.com/antonjan/Raspberry_Telemetry/master/telem-balloon_2.conf

Lets edit the file for your path.
vi cron_aprs_gps_position.sh
Fix the yellow with path to your direwolf file.
#load looback and start gpsd service for gps data.
sudo /usr/sbin/gpsd -D 5 -N -n /dev/ttyACM0 &
sleep 5
sudo /sbin/modprobe snd-aloop
sleep 2

#We now will write out the audio from loopback from direwolf to a wav file
sudo timeout 10s arecord -c1 -t wav -r 48000  --vumeter=mono -D hw:Loopback,1,0 -fS16_LE /home/pi/sh/direwalf.wav &
#We now will create a APRS audio withS position and send it to loopback device
sudo timeout 9s direwolf -c /home/pi/Downloads/direwolf/telemetry-toolkit/telem-balloon_2.conf

# we now wait for short wile
sleep 1

#we now convert the wav file in a format to transmit
sudo sox -v 0.3 -S /home/pi/sh/direwalf.wav /home/pi/sh/direwalf_sox.wav  rate -L -s 48000

# we now wait for short wile  
sleep 1
#we now convert the APRS audio file to a fm file for transmission.
sudo /home/pi/Downloads/rpitx_new/rpitx/pifm  /home/pi/sh/direwalf_sox.wav /home/pi/sh/direwalf_sox.wav.ft

# we now wait for short wile
sleep 1
#We now transmit the the APRS gps postion on the frequensy 144,801Mhz
sudo /home/pi/Downloads/rpitx_new/rpitx/rpitx -i /home/pi/sh/direwalf_sox.wav.ft -m RF -f 144801 -c1
Save the file

Lets send our first APRS GPS position.
Change the callsign in the  telem-baloon_2.conf to your callsign
Check that the paths in the telem-baloon_2.conf  and cron_aprs_gps_position.sh is correct in relation to the directory you check the code out.
run the script sudo ./cron_aprs_gps_position.sh and you should now send a gps position aprs message. If you change the frequency to your local packet  (APRS in SA its 144.8Mhz)frequency you should see the Balloon Icon on the APRS server.  https://aprs.fi/#!lat=60.169998&lng=24.94
You mite want to calibrate the TX frequency by changing the frequency in the script.
You can now add this script in a cron to send every 1minute ...

Here is an exsample just to send a message
sudo aprs --callsign NOCALL --output - "30mw Belloon Beacon de NOCALL" | csdr convert_i16_f | csdr gain_ff 7000 | csdr convert_f_samplerf 20833 | sudo rpitx -m RF -i - -f 144325 -c1

Ok lets now send some CW as a beacon.

Lets create a text file with the cw message we want to send.
vi cw_text 
enter the following example

                               32mw balloon experiment de NOCALSIGN
save the file (including the space in front)

we now will be using a different script crontab_bacar_cw.sh
edit this script.
vi crontab_bacar_cw.sh
#This script will send a cw at 10 words per minute 
#Start GPS and alsa loopback
sudo /usr/sbin/gpsd -D 5 -N -n /dev/ttyACM0 &
sleep 5
sudo /sbin/modprobe snd-aloop

#now setup loopback to write to file


sudo /usr/bin/timeout 11s /usr/bin/arecord -c1 -t wav -r 48000  --vumeter=mono -D hw:Loopback,1,0 -fS16_LE /home/pi/sh/bacar_cw.wav &
# now send cw to loopback with cw_text.txt
sudo /usr/bin/timeout 10s /usr/bin/cw -s a -d hw:Loopback,1,0 -t 1000 -v 50 -f /home/pi/sh/cw_text.txt
sleep 1
#Now convert the file to format that can be converted to fm signal.
sudo /usr/bin/sox -v 0.9 -S /home/pi/sh/bacar_cw.wav /home/pi/sh/bacar_cw_sox.wav  rate -L -s 48000
sleep 1
#now create transmission file for rpitx
sudo /home/pi/Downloads/rpitx_new/rpitx/pifm  /home/pi/sh/bacar_cw_sox.wav /home/pi/sh/bacar_cw_sox.wav.ft
sleep 1
#now send your CW (morse code on 144.328 Mhz) Change frequency as requerd
sudo /home/pi/Downloads/rpitx_new/rpitx/rpitx -i /home/pi/sh/bacar_cw_sox.wav.ft -m RF -f 144328 -c1

Save the file and run and test.
sudo ./crontab_bacar_cw.sh

Now listen on the tx frequency and run the script and you should here the message "32mw balloon experiment de YOURCALLSIGN"

Friday, April 13, 2018

ISS SSTV Recording 13/04/2018 08:04 CAT

This is a video of SSTV recording of SSTV from International Space Station.


I had some Local Repeater QRM during the reception.

This was a nice pass.
87 deg Elevation. a very nice pass

Thursday, February 22, 2018

Setting up my hackrf on ubuntu 17.10

Setting up my hackrf on ubuntu 17.10



sudo apt-get install gcc-arm-none-eabi

sudo apt install cmake
sudo apt install hackrf

if the hackrf command above did not work then follow the following steps


git clone https://github.com/mossmann/libopencm3.git
cd libopencm3/
ls
cd ../hackrf/
ls
git submodule init
git submodule update
cd firmware/libopencm3/
make
sudo synaptic
xhost +
sudo synaptic
make
sudo apt-get install gcc-arm-none-eabi
cd hackrf/firmware/hackrf_usb/build
mkdir build
cd build
cmake ..

hackrf_spiflash -w hackrf_usb.bin

Update firmware
hackrf_spiflash -w hackrf_one_usb.bin
hackrf_info

hackrf_cpldjtag -x hackrf_cpld_default.xsvf
hackrf_info

Thursday, November 16, 2017

HF Noise monitoring system running from Live Ubuntu Memory stick with fully configured environment.

HF Noise monitoring system running from Live Ubuntu USB flash drive.

Ubuntu Live Noise Monitoring system with Persistence. 16Gb
I wanted a Preconfigured RF Noise monitoring system that I can run on my Laptop when ever I am not using it for something else.
I made this Ubuntu 16.04 Live memory stick .

This dongle is available from my Shop at http://www.giga.co.za/ocart/index.php?route=product/product&product_id=344

So how dose it work?
Configure your BIOS to boot from USB flash drive and you should then see the following screens.

Just pres enter to continue
If you see the error above just pres Enter  You should then get a screen where you can select how you want to boot your environment Persistent or not.
Select  the Persistent live boot.
Select the Persistent live option.
You could then see a screen for a few seconds looking like this.
Black screen before startup
This picture is of Ubunto 17.10 The Noise system is on Ubuntu 16.04
Here is some of the sunscreens of some of the functionality

HF Heat map
 Noise Graphs  (link to a live system)

Here is a link to a brochure on the RF Noise Monitoring system

Master Data correlation server http://rfnoise.amsatsa.org.za

Master Correlation Server.


Thursday, November 2, 2017

How to make a persisnt ubuntu LIve Memeory stick (16GB)

How to make a persistent live Ubuntu USB Memory stick (16GB)


16Gb Flash disk
I wanted to create a boo-table Ubuntu live flash stick that will also keep my last installed applications and not default back to standard when I reboot.

Ubuntu 17.10 Live with all my apps

You need to create two partition fat32 for the Ubuntu instillation from ISO file and a ext4 casper-rw partition for the persistent part. The compleet proses is done with a application called DUS.
Here is the process to install the DUS software
sudo add-apt-repository universe
sudo add-apt-repository ppa:mkusb/ppa
sudo apt-get update
sudo apt-get install mkusb usb-pack-efi
sudo apt-get install mkusb-nox usb-pack-efi
dus




Select the following on the screens.

1) Double-Click on the i option Install(make a boot device):
2) Double-Click on the p option Persistent Live and select the .iso file:
3) Click on the USB drive to make persistent. I want to use my 32 GB one so I mark the radio button:
4) Click Use Defaults to let mkusb choose default:
5) Enter a percentage of the drive to reserve for persistence. Mine is a large (32GB) so I allot only 20% ~ 6GB. The remaining space will be turned into an empty portable drive:
6) Double check very carefully and confirm. You don't want to mess up another drive.

Waite until 'Done' is written go and get some coffie :-)

 Here was my configuration

Cruzer Blade     sdb                                                    14.6G
                 |-sdb4 iso9660 Ubuntu 17.10 amd64         1.4G
                 |-sdb2                                                         1M
                 |-sdb5 ext4    casper-rw                             12.9G
                 |-sdb3 vfat    usbboot                                 244M
                 `-sdb1                                                        1.1M


ref : https://askubuntu.com/questions/397481/how-to-make-a-persistent-live-ubuntu-usb-with-more-than-4gb
and https://help.ubuntu.com/community/LiveCD/Persistence

Wednesday, August 30, 2017

RF Noise Monitoring system for HF

RF Noise Monitoring system on a Raspberry Pi

(This system will record RF signals and upload the results to the Server where trending graphs are created)

The New Master Correlation server (Beta) is up and running and ready to be used.
Master RF noise server. (Data correlation server)

This system is now also available on Ubuntu live memory stick.

This system software is available pre installed on a 16GB SD card from Giga Technology
Raspberry Pi OS with HF monitoring software pre installed
What do I get on the software?
I system that will record 1Mhz band with samples every 2 minutes generating Graphs and Heat-map of the hole 1 to 30Mhz band.
What hardware do I need?
The  100khz to 1.7Ghz HF dongle as in picture below Screenshot of Graph monitoring.
url of link on system http://localhost/hf_noise/graph/1Mhz_Power.php
Link to live system is here http://zr6aic.giga.co.za/hf_noise/graph/1Mhz_Power.php 
Screen shot of the HF Monitoring system

How does this system work?

Block Diagram of the HF Noise Monitoring system

How dose the HF Noise Monitoring system work

  • The HF noise monitoring system takes a 1Mhz bandwidth samples every 2 minutes using the rtl_power utility and save the measurements in a CVS fie.
  • The 2 minute scheduling is done with a Crontab calling a script in the hf_noise directory.
  • The RF Samples is taken at a 1Mhz bandwidth from 1M to 30Mhz therefore 29 of  the CVS files is created and gets appended as the measurements is made.
  • A Perl script utility is the executed from a cron scedular  to read the 29 CSV files and import there data into a RRD database.
  • The RRD database then gets interrogated by another Perl script scheduled by a cronjob to generate the graph Images and and html files and get saved in the Apache web server graph directory witch is the exposed on http://localhost/hf_noise/graph/1Mhz_Power.php
  • Link to Live system http://zr6aic.giga.co.za/hf_noise/graph/1Mhz_Power.php 
  • The RRD database will then create the Hourly, Daily,Weekly,Monthly and Yearly averages and   then generates the graphs every 2 minutes according to the parameters passed to Database.
  • The generated graphs is then displayed in a php page via a Apache web server with an php plugin on the following url http://localhost/hf_noise/graph/1Mhz_Power.php.
  • Trends can then be seen on a graphs over time.
  • The heat map is generated by a python script from different CSV files and is saved in the /var/www/hf_noise/images directory and can be accessed on the local web server on the following url http://localhost/hf_noise/images
  • Link to live system http://zr6aic.giga.co.za/vhf_heatmap/images/
  • The systems has a static IP by default on Ethernet 192.168.10.200 and 192.168.10.205 on wifi if you have wifi dongle installed. You can change it to your ip

Here is a list of RRD database management tips

  • How to backup my RRD db data?
  • use rrdtool dump hf_noise.rrd > filename.xml to export the data to XML format.
  • How to Restore my RRD db data?
  • Transfer the XML dump to the target system. Run rrdtool restore [--range-check|-r] [--force-overwrite|-f] filename.xml filename.rrd to create a new RRD from the XML dump. See rrdrestore for details.
  • How to change some of the the RRD db data?
  • Use rrdtool dump to export RRD files to XML.
    Open the XML file, find and edit the bad data.
    Restore the RRD file using rrdtool restore.

 How to generate a brand new RRD db?


There is an script in the hf_noise directory called create_rrd_db.sh
You can just run this script in the hf_directory sudo ./create_rrd_db.sh

Here is the details of the scrip.
    rrdtool create  hf_noise.rrd --start now-2d --step 120 DS:1Mhz_Power:GAUGE:120:-50:10 DS:2Mhz_Power:GAUGE:120:-40:10 DS:3Mhz_Power:GAUGE:120:-50:10 DS:4Mhz_Power:GAUGE:120:-40:10 DS:5Mhz_Power:GAUGE:120:-50:10 DS:6Mhz_Power:GAUGE:120:-40:10 DS:7Mhz_Power:GAUGE:120:-50:10 DS:8Mhz_Power:GAUGE:120:-40:10 DS:9Mhz_Power:GAUGE:120:-50:10 DS:10Mhz_Power:GAUGE:120:-40:10 DS:11Mhz_Power:GAUGE:120:-50:10 DS:12Mhz_Power:GAUGE:120:-40:10 DS:13Mhz_Power:GAUGE:120:-50:10 DS:14Mhz_Power:GAUGE:120:-40:10 DS:15Mhz_Power:GAUGE:120:-50:10 DS:16Mhz_Power:GAUGE:120:-40:10 DS:17Mhz_Power:GAUGE:120:-50:10 DS:18Mhz_Power:GAUGE:120:-40:10 DS:19Mhz_Power:GAUGE:120:-50:10 DS:20Mhz_Power:GAUGE:120:-40:10 DS:21Mhz_Power:GAUGE:120:-50:10 DS:22Mhz_Power:GAUGE:120:-40:10 DS:23Mhz_Power:GAUGE:120:-50:10 DS:24Mhz_Power:GAUGE:120:-40:10 DS:25Mhz_Power:GAUGE:120:-50:10 DS:26Mhz_Power:GAUGE:120:-40:10 DS:27Mhz_Power:GAUGE:120:-50:10 DS:28Mhz_Power:GAUGE:120:-40:10 DS:29Mhz_Power:GAUGE:120:-50:10 DS:30Mhz_Power:GAUGE:120:-40:10 RRA:AVERAGE:0.5:1:864000 RRA:AVERAGE:0.5:60:129600 RRA:AVERAGE:0.5:3600:13392 RRA:AVERAGE:0.5:86400:3660

I have still got my csv files and want to reload it into my rrd database


HF Heat map monitoring System. 


Link on local network http://localhost/hf_noise/images

link to live system http://zr6aic.giga.co.za/hf_noise/images/
Here is  a screenshot of the monitoring page and the green button
 can be used to select the frequency band of interest


Single view

Here is a view of the HF spectrum from 1Mhz to 30Mhz (activity can now be seen for the day)
Here is a view of the heat map that was selected
If you click on the image the the image will zoomed to maximum

Enlarged View

Here is a screenshot of the enlarged view.

So here is an example of the utility that generate the CSV file

Noise Recording

rtl_power -f 24M:1.7G:1M -g 50 -i 20m -1 noise-unmodded.csv

Here is the instillation instruction

hf_noise

Source code avalibe from https://github.com/antonjan/hf_noise

Alt text

Instillation Instructions


#Create user hfnoise
sudo adduser hfnoise
sudo usermod -aG sudo hfnoise

#change to hfnoise user with the following command. (dont try to install with other users)
su - hfnoise
#enter your new password
sudo apt-get update
#If you get this error below do the following
#** (appstreamcli:2761): CRITICAL **: Error while moving old database out of the way. AppStream cache update failed.
#then run this command and try again sudo chmod -R a+rX,u+w /var/cache/app-info/xapian/default
#now lets upgrade and get some coffie :-)

sudo upgrade
#We now need to install git to download the noise monitoring application
sudo apt-get install git
sudo apt install cmake
sudo apt-get install libusb-1.0-0-dev
sudo apt install python-setuptools
sudo easy_install pip
sudo pip install image
cd
#for ubuntu 17.10 do the following.
wget http://archive.ubuntu.com/ubuntu/pool/universe/i/imageinfo/imageinfo_0.04-0ubuntu11_amd64.deb
sudo dpkg -i ./imageinfo_0.04-0ubuntu11_amd64.deb
# for Ubuntu 16.04 do the following
 wget http://archive.ubuntu.com/ubuntu/pool/universe/i/imageinfo/imageinfo_0.04-0ubuntu10_amd64.deb
sudo dpkg -i ./imageinfo_0.04-0ubuntu10_amd64.deb
cd
#git clone git://git.osmocom.org/rtl-sdr.git
#Please make sure you use this repository as the osmond one dont support direct conversion mode
git clone https://github.com/keenerd/rtl-sdr.git
cd rtl-sdr/
mkdir build
cd build
cmake ../
make
sudo make install
sudo ldconfig
ls
sudo vi /etc/modprobe.d/no-rtl.conf
#add the following blacklist dvb_usb_rtl28xxu
blacklist rtl2832
blacklist rtl2830
#reboot 
#we now need to install an application that will allow us to reset the USB port where the rtl dongle is installed if it locks up for some reason (this has append when there is lightning in the aria.So what i do is motor if the csv file get updated if not I reset the usb port with this utility.
cd
git clone https://github.com/jkulesza/usbreset.git
cd usbreset
#lets compile the application
cc usbreset.c -o usbreset
#we now need to establish witch usb port is your rtl dongle installed.
sudo lsusb
#we now need to edit the shell script that monitor the csv file update with your rtl usb port details.
#now look for the device with this name "Realtek Semiconductor Corp. RTL2838 DVB-T" in my case it was "Bus 003 Device 004" to be like E.g sudo /home/hfnoise/usbreset/usbreset /dev/bus/usb/003/004
cd
git clone  https://github.com/antonjan/hf_noise.git
cd hf_noise
cd sh
vi or use your editor and check_if_file_was_updated.sh
change the line that looks like this "vi /home/anton/Downloads/usbreset/usbreset /dev/bus/usb/"
002/005" with your detail
#pull the noise monitoring system from github
git clone https://github.com/antonjan/hf_noise.git
#install Apache2 server
sudo apt-get install apache2
sudo systemctl start apache2.service
sudo systemctl enable apache2.service
sudo apt install php7.1 libapache2-mod-php7.1
sudo a2enmod php7.1
sudo systemctl restart apache2
cd /var/www/html
sudo mkdir hf_noise
sudo mkdir hf_noise/graph
sudo mkdir hf_noise/images
cd hf_noise/graph
sudo cp -r /home/hfnoise/hf_noise/*.php ./
sudo cp -r /home/hfnoise/hf_noise/*.html ./
sudo cp -r /home/hfnoise/hf_noise/*.png ./
cd /var/www/html/hf_noise/images
sudo cp -r /home/hfnoise/hf_noise/image/*.php ./

sudo cp -r /home/hfnoise/hf_noise/image/*.html ./
sudo cp -r /home/hfnoise/hf_noise/image/*.jpg ./
sudo cp -r  /home/hfnoise/hf_noise/image/thumb ./
#now test if hfnoise webserver work.
firefox http://localhost/hf_noise
#you should see the index page with no pictures.
#We now need to install the heatmap application.
#lets go to home directory and install heatmap
cd
#got hf_noise directory and install heatmap by git from github
cd hf_noise
#Get the heatmap python progrm from github
git clone https://github.com/keenerd/rtl-sdr-misc.git 
cd /home/hfnoise/hf_noise/rtl-sdr-misc/heatmap
#give the application execution writes
chmod 776 *.py
#Now we need to install the perl librareries
#CPAN, the Comprehensive Perl Archive Network, is the primary source for #publishing and fetching the latest modules
cd
sudo apt-get install perl build-essential curl
sudo apt-get install rrdtool
sudo apt-get install librrds-perl
cd /usr/local/bin
sudo wget https://raw.github.com/miyagawa/cpanminus/master/cpanm
sudo chmod +x cpanm
cd
sudo cpanm --self-upgrade --sudo
#sudo apt-get install cpanminus
sudo cpanm Text::Trim
sudo cpanm Text::CSV
sudo cpanm Date::Manip
sudo cpanm Text::Trim
# mite be only trim
sudo cpanm RRD::Simple
cd
cd hf_noise
cd sh
#run then script to generate rrd db
chmod 776 create_rrd_db.sh
./create_rrd_db.sh
#Ok now connect your HF dongel to your laptop usb port and connect to an antenna.
#Ok now everything should be ready.
#we need to enable the scripts in the crontab (scheduler)
#run the script to setup the scheduler
sudo python ./create_crontab.py

#If you want to see how the crontab was setup you can check with this command
sudo crontab -e
#Below is the what you should see
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/hfnoise/hf_noise/sh/create_heatmap.sh 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58 * * * * /home/hfnoise/hf_noise/sh/run_hf_noise_monitor_and_graph.sh 1,6,11,16,21,26,31,36,41,46,51,56 * * * * /home/hfnoise/hf_noise

#save file and now and we are now done.
#The images and history file should now be automatically being created every 5 minutes.
#Make sure you have along wire antenna connected to your HF rtl dongle to hf noise recordings.
#********************end******************
#How can I check that data is collected.
#run the command
ls -lrt /home/hfnoise/hf_noise
#you should see somthing lyk this
-rw-r--r-- 1 root    root          0 Nov  8 19:06 hf_monitoring_08-11-2017.cvs
-rw-r--r-- 1 root    root          0 Nov  8 19:06 hf_monitoring_graph_08-11-2017.cvs
You should also see graph images in you browser.
firefox http://localhost/hf_noise/graph/1Mhz_Power.php

File upload to Master Analytics site will be available when it goes alive.