KZ2X Packet Node Description

# Jan 15, 2022

This is a description of the KZ2X packet node and its setup.

Overview

The packet node current hosts a single RF interface on 2m VHF, and listeners in the Boston metro area may hear KZ2X-1 on 145.090 MHz.

We also support two AXUDP interfaces (AX.25 encapsulated over IP/UDP): one for general connectivity with other Internet connected packet stations, and one that acts as a “loopback” for testing. Contact KZ2X if you would like to peer over AXUDP.

The packet station is a single computer running Linux, using the in-kernel AX.25 and NET/ROM implementations. Our NET/ROM address is KZPAD:KZ2X-3.

Hardware and Systems Software

A single machine for handles AX.25:

Host vhf1.kz2x.ampr.org (44.44.107.11). This is on my AMPRNet allocation, connected to the Internet for an IP-IP tunnel to the AMPRNet gateway at UCSD.
Udoo Bolt v8 SBC (AMD Ryzen x86_64, 16GiB RAM, 1TB NVMe SSD)
Arch Linux latest with the 5.16.0-rc7-1 mainline kernel compiled from source. The YO2LOJ NET/ROM socket cleanup patch is applied
uname -a: Linux vhf1.kz2x.ampr.org 5.16.0-rc7-1-mainline #1 SMP PREEMPT Mon, 03 Jan 2022 12:57:48 +0000 x86_64 GNU/Linux
Hamlib built from source
Running Direwolf built from dev branch source
ax25-tools “backported” from Debian and compiled from source (ChangLog v0.0.10)
ax25-apps similarly “backported” from Debian and compiled from source (ChangeLog v0.0.8)
Tigertronics SignaLink USB “sound card”.
Dedicated Icom IC-2100H dedicated to the packet station
75' of LMR-400 and an RF choke at the end
Homebrew j-pole soldered from copper plumbing pipe

Software built from source is installed into the /opt/local directory tree. Since this includes shared libraries, we ensure those libraries will be found by the runtime linker by adding creating /etc/ld.so.conf.d/opt containing the line /opt/local/lib.

We load any needed kernel modules by creating configuration files under /etc/modules-load.d. Presently, we only need to add netrom.conf containing the line netrom to load the NET/ROM module at boot.

AX.25 Port Configuration

As mentioned above, we define three AX.25 ports: one for the RF interface on 2m, and two AXUDP interfaces. We do this by adding entries in the axports file:

#
# The format of this file is:
#
# name callsign speed paclen window description
#
vhf0    KZ2X-0          9600    255     2       145.090 MHz (1200 bps)
udp0    KZ2X-2          115200  255     2       AXUDP0
udp1    KZ2X-15         115200  255     2       AXUDP1

A few notes about this file: the assigned SSIDs given to each port must be globally unique. The line rate is for the port itself, not the transmission media. We use the maximum MTU and a window size of two. The last field is just descriptive text.

See the Linux man page axports(5) for more details about the format of the file.

Configuring and Starting Direwolf

Here’s the direwolf.conf file, with comments and blank lines stripped out:

: vhf1; grep -v '^#' direwolf.conf | noblanks
ADEVICE plughw:CODEC,0
ACHANNELS 1
CHANNEL 0
MYCALL KZ2X
MODEM 1200
AGWPORT 8000
KISSPORT 8001
FIX_BITS 2
IGTXLIMIT 6 10
TTPOINT  B01  37^55.37N  81^7.86W
TTPOINT  B7495088  42.605237  -71.34456
TTPOINT  B934  42.605237  -71.34456
TTPOINT B901  42.661279  -71.364452
TTPOINT B902  42.660411  -71.364419
TTPOINT B903  42.659046  -71.364452
TTPOINT B904  42.657578  -71.364602
TTVECTOR  B5bbbddd  37^55.37N  81^7.86W  0.01  mi
TTGRID   Byyyxxx    37^50.00N  81^00.00W  37^59.99N  81^09.99W
TTUTM  B6xxxyyy  19T  10  300000  4720000
TTCORRAL   37^55.50N  81^7.00W  0^0.02N
TTMACRO  xx1yy  B9xx*AB166*AA2B4C5B3B0A1yy
TTMACRO  xx2yy  B9xx*AB170*AA3C4C7C3B0A2yy
TTMACRO  xxyyy  B9xx*AB180*AA3A6C4A0Ayyy
TTMACRO  z  Cz
: vhf1;

[Note: It occurs to me that I’ve never really looked at any of the TT* options, but I believe that is only for used for APRS which is not currently in use.]

Direwolf and the AX.25 software components are started by the systemd service manager, but before we start direwolf, we set the audio levels on the audio device using the ALSA mixer. The service files for both are as follows:

[Unit]
Description=mixer levels for the TNC audio device
Wants=dev-bus-usb-001-001.device network-online.target cleanup-old-state.service
After=dev-bus-usb-001-001.device network-online.target cleanup-old-state.service

[Service]
Type=oneshot
ExecStart=/opt/local/etc/rc.d/tnc-audio

[Install]
WantedBy=multi-user.target

/opt/local/etc/rc.d/tnc-audio is a small shell script:

#!/bin/sh
amixer -c CODEC set PCM 95%

We use service dependencies to ensure that this happens before starting direwolf. Note that we also use dependencies to make sure that the hub the SignaLink USB device is connected to is online as well as the network. We notify systemd once USB is online by adding /etc/udev/rules.d/20-usb-bus.rules:

KERNEL=="usb[1-2]", TAG+="systemd"

Note we also invoke a service to clean old state:

[Unit]
Description=cleanup old system state

[Service]
Type=oneshot
ExecStart=/opt/local/etc/rc.d/cleanup-old-state

[Install]
WantedBy=multi-user.target

The script this invokes just removes old files and is omitted.

The direwolf service file:

[Unit]
Description=Direwolf software TNC
After=tnc-audio.service
Wants=tnc-audio.service

[Service]
Type=exec
ExecStartPre=/opt/local/etc/rc.d/pre-direwolf
ExecStart=/opt/local/etc/rc.d/direwolf

[Install]
WantedBy=multi-user.target

Direwolf allocates a POSIX PTY pair that it uses to synthesize a KISS TNC. Since PTYs are dynamically allocated by the system, and thus don’t have predictable file files, direwolf helpfully creates a symbolic link pointing to the PTY file client software should connect to: /tmp/kisstnc. The /opt/local/etc/rc.d/pre-direwolf shell script simply deletes that link if it exists before direwolf starts:

#!/bin/sh
rm -f /tmp/kisstnc

And /opt/local/etc/rc.d/direwolf invokes direwolf itself:

#!/bin/sh
/opt/local/bin/direwolf -X 1 -q d -t 0 -p -c /opt/local/etc/direwolf.conf

Some notes on the invocation:

The -X 1 option enables the FX.25 extension to AX.25, which embeds forward-error correction codes into transmitted frames.
The -q d option suppresses printing APRS debugging information, as we’re not using APRS in this configuration.
The -t 0 option disables text colors. As we start direwolf under systemd, its output is put into the systemd log. The terminal escape sequences that would be insert colors into its output make no sense in that environment.
The -p option enables the KISS interface pseudo-TTY.
The -c /opt/local/etc/direwolf.conf option gives the path to the direwolf configuration file.

Finally, we use kissattach to attach the KISS TNC pseudo-device provided by direwolf. Again, we use a systemd service for this:

Description=KISS AX.25 interface for Direwolf software TNC
Requires=direwolf.service
After=direwolf.service

[Service]
Type=forking
ExecStartPre=/opt/local/bin/wait-symlink /tmp/kisstnc
ExecStart=/opt/local/etc/rc.d/kiss-vhf0
ExecStartPost=/opt/local/etc/rc.d/post-kiss-vhf0

[Install]
WantedBy=multi-user.target

The /opt/local/bin/wait-symlink /tmp/kisstnc is a short script that waits for a symbolic link to be created. Invoking this as a “pre” command avoids a race condition where kissattach tries to attach to the symbolic link before direwolf creates it. Since the kissattach service won’t be run until after the direwolf service starts executing, and the “pre” job for direwolf removes any stale symlinks before it invokes direwolf itself, this prevents kissattach from running until the /tmp/kisstnc symlink exists and points to the currently running instance of direwolf.

Here is /opt/local/bin/wait-symlink:

#!/bin/sh
while ! test -L "$@"
do
        sleep 1
done

We will see this script again when we set up the AXUDP links. In the meantime, here is the kissattach startup script:

#!/bin/sh

PATH=/opt/local/sbin:$PATH
export PATH

kissattach -l /tmp/kisstnc vhf0

The “post” script /opt/local/etc/rc.d/post-kiss-vhf0 sets parameters on the bound AX.25 interface:

#!/bin/sh

PATH=/opt/local/sbin:$PATH
export PATH

kissparms -c 1 -p vhf0

The -c 1 sets the port CRC type to “none” while -p vhf0 specifies the RF port.

AXUDP Interfaces

The two AXUDP interfaces for internet connectivity and the testing loopback are started after similarly, except that instead of direwolf, we run the ax25ipd program implement AX.25 over TCP/IP. Additionally, we employ a trick due to Marius Petrescu, YO2LOJ, and use the socat program to create symbolic links giving the PTY pairs used by ax25ipd and kissattach stable names; the wait-symlink script seen above makes another appearance here.

The services and scripts to create pty pairs for the udp0 and udp1 ports are as follows:

/etc/systemd/service/socat-udp0.service:

Description=persistently named PTYs for AX.25 interface udp0

[Service]
Type=exec
ExecStartPre=/opt/local/etc/rc.d/pre-socat-udp0
ExecStart=/opt/local/etc/rc.d/socat-udp0

[Install]
WantedBy=multi-user.target

/opt/local/etc/rc.d/pre-socat-udp0:

#!/bin/sh
optvar=/opt/local/var
rm -f $optvar/ax25/ptyAXUDP0 $optvar/ax25/kissAXUDP0

/opt/local/etc/rc.d/socat-udp0:

#!/bin/sh
optvar=/opt/local/var
socat pty,link=$optvar/ax25/ptyAXUDP0,raw,echo=0 pty,link=$optvar/ax25/kissAXUDP0,raw,echo=0

The corresponding service and scripts for udp1 are nearly identical and omitted for brevity.

Now, we start ax25ipd. The configuration file for udp0 sets up routes and configuration options.

With comments and blank lines stripped:

socket udp 10093
mode tnc
device /dev/ptmx
speed 115200
loglevel 3
broadcast QST-0 NODES-0
route kz2x-15 localhost udp 10094 b
route kz2x-14 pi1.kz2x.ampr.org udp 10093 b
route hb8nod hb1bbs.net udp 93 b
route hb8nos hb1bbs.net udp 93 b
route ka5d bbs.ka5d.com udp 10093 b
route n3hym n3hym.ddns.net udp 10093 b
route w1jt 76.24.175.218 udp 10093 b

Most of this is self-explanatory; if unsure, consult the Linux manpage for ax25ipd.conf(5).

Note the use of the device /dev/ptmx option. Since we use socat to create the PTY pair and specify this as a command line parameter when we invoke ax25ipd, this is superfluous and could probably be removed.

The service file and script:

[Unit]
Description=KISS AX.25 interface for AXUDP port udp0
Wants=kiss-vhf0.service socat-udp0.service
After=kiss-vhf0.service socat-udp0.service

[Service]
Type=forking
ExecStartPre=/opt/local/bin/wait-symlink /opt/local/var/ax25/ptyAXUDP0
ExecStart=/opt/local/etc/rc.d/ax25ipd-udp0

[Install]
WantedBy=multi-user.target

Note the use of wait-symlink to wait for the PTY device created by socat.

/opt/local/etc/rc.d/ax25ipd-udp0:

#!/bin/sh

PATH=/opt/local/sbin:$PATH
export PATH

optvar=/opt/local/var

ax25ipd -d $optvar/ax25/ptyAXUDP0 -c /opt/local/etc/ax25/ax25ipd.conf

Finally, we attach the KISS TNC synthesized by ax25ipd to an AX.25 interface by another service:

Description=KISS AX.25 interface for AXUDP port udp0
Wants=ax25ipd-udp0.service
After=ax25ipd-udp0.service

[Service]
Type=forking
ExecStartPre=/opt/local/bin/wait-symlink /opt/local/var/ax25/kissAXUDP0
ExecStart=/opt/local/etc/rc.d/kiss-udp0

[Install]
WantedBy=multi-user.target

/opt/local/etc/rc.d/kiss-udp0:

#!/bin/sh

PATH=/opt/local/sbin:$PATH
export PATH

optvar=/opt/local/var

kissattach -l $optvar/ax25/kissAXUDP0 udp0

The loopback AXUDP interface is set up nearly identically, except for the configuration:

/opt/local/etc/ax25/ax25ipd-loopback.conf:

socket udp 10094
mode tnc
device /dev/ptmx
speed 115200
loglevel 2
route kz2x localhost udp 10093 b

Note two things: first, we use UDP port 10094, and we configure a single route to localhost port 10093, which is the AXUDP port associated with the udp0 interface. Second, in the ax25ipd.conf file for udp0, we configure a route to KZ2X-15 (the SSID associated with port udp1) to localhost port 10094, with this instance of ax25ipd is bound on. This, we can “call” ourselves over AXUDP by calling one of our SSIDs via port udp1; e.g., axcall udp1 kz2x-1.

The service definition files and startup scripts for udp1 are nearly identicatl to those for udp0, and differ only in the port name and path to the ax25ipd configuration file.

A Note on AXUDP versus AXIP

AX.25 traffic is most commonly tunneled over IP using two protocols: AXIP, which defines an IP protocol type for tunneled AX.25 traffic, and AXUDP, which embeds AX.25 frames in UDP packets.

Of the two, there is little reason to prefer AXIP. While AXUDP does add a small amount of overhead in the form of a UDP header, this is negligible (8 octets) and AXUDP has the significant advantage of port addressibility compatibility with network software that works with UDP and not AXIP. Furthermore, the two have the same characteristics with respect to reliability and delivery guarantees.

AXIP has sigificant issues with respect to tranversing firewalls and the like and is just as unreliable as AXUDP (AXIP frames, after all, are just IP datagrams).

AXIP should probably be retired and phased out in favor of AXUDP.

NET/ROM Configuration and Startup

Once the AX.25 interfaces are up, we start NET/ROM. Linux NET/ROM is configured using two files, nrports and nrbroadcast.

/opt/local/etc/ax25/nrports:

#
# The format of this file is:
#
# name callsign alias paclen description
#
netrom  KZ2X-3          KZPAD   235     KZ2X Amateur Radio Computing Resource Complex

This is similar to the axports file we saw earlier. The first field is the port name. We only have one netrom port, so we just use the name netrom. The second is the SSID associated with the port: this must be unique with respect to all SSIDs in use on the system; one cannot reuse an SSID from AX.25, for example. The third field is an alias for the system: KZPAD alludes to an X.25 Packet Assembly/Disassembly device, but I refer to it as a Public Access Dialthrough; this using is not accurate. Note that the MTU is somewhat smaller than that for AX.25: the NET/ROM protocol introduces some overhead that we must account for, hence the smaller packet size. For more details on the file’s format, see nrports(5).

Our nrbroadcat file is:

#
# The format of this file is:
#
# ax25_name min_obs def_qual worst_qual verbose
#
vhf0    1       200     100     0
udp0    1       200     100     1

See the nrbroadcast(5) man page for details; note only that we clear the “verbose” flag on the RF port, so we do not broadcast downstream NET/ROM routes other than our own on that interface.

We invoke a service to attach the netrom port to a NET/ROM network interface:

[Unit]
Description=NET/ROM interface attach

[Service]
Type=exec
ExecStart=/opt/local/etc/rc.d/nrattach

[Install]
WantedBy=multi-user.target

/opt/local/etc/rc.d/nrattach:

#!/bin/sh

PATH=/opt/local/sbin:$PATH
export PATH

nrattach netrom

Finally, we invoke the netromd daemon:

[Unit]
Description=NET/ROM service daemon
Wants=kiss-vhf0.service kiss-udp0.service kiss-udp1.service nrattach.service
After=kiss-vhf0.service kiss-udp0.service kiss-udp1.service nrattach.service

[Service]
Type=forking
ExecStart=/opt/local/etc/rc.d/netromd

[Install]
WantedBy=multi-user.target

Note that the service requires all of the AX.25 interfaces and the NET/ROM interface to be up before starting.

/opt/local/etc/rc.d/netromd is simple:

#!/bin/sh

PATH=/opt/local/sbin:$PATH
export PATH

netromd -i -l

The -i option causes it to broadcast a route immediately on startup, and -l enables error logging.

Services

Once the interfaces are online, we run a few local services.

We run the mheardd daemon to keep track of what stations we hear on all interfaces. It’s service definition:

[Unit]
Description=mheard daemon
Wants=kiss-vhf0.service kiss-udp0.service kiss-udp1.service netromd.service
After=kiss-vhf0.service kiss-udp0.service kiss-udp1.service netromd.service

[Service]
Type=forking
ExecStart=/opt/local/etc/rc.d/mheardd

[Install]
WantedBy=multi-user.target

/opt/local/etc/rc.d/mheardd is trivial:

#!/bin/sh

PATH=/opt/local/sbin:$PATH
export PATH

mheardd

We also run the beacon program to send out a periodic announcement of our system on the RF interface:

Description=periodic beacon daemon
Wants=kiss-vhf0.service kiss-udp0.service kiss-udp1.service netromd.service
After=kiss-vhf0.service kiss-udp0.service kiss-udp1.service netromd.service

[Service]
Type=forking
ExecStart=/opt/local/etc/rc.d/beacon

[Install]
WantedBy=multi-user.target

/opt/local/etc/rc.d/beacon:

#!/bin/sh

PATH=/opt/local/sbin:$PATH
export PATH

beacon vhf0 -H -d 'BEACON BROCK' 'KZ2X-1 Unix and public access dial-through for Amateur Radio Timesharing. Cambridge MA USA'

Finally, we run the ax25d daemon to direct incoming connections to useful services. Here is the configuration file:

#
# ax25d Configuration File.
#
# AX.25 Ports begin with a '['.
#
[KZ2X-1 via vhf0]
NOCALL   * * * * * *  L
default  * * * * * *  - root    /opt/local/sbin/axtip axtip
#
[KZ2X-4 via vhf0]
NOCALL   * * * * * *  L
default  * * * * * *  - root    /opt/local/sbin/ttylinkd ttylinkd -f /opt/local/etc/ax25/ttylinkd.conf
#
[KZ2X-1 via udp0]
NOCALL   * * * * * *  L
default  * * * * * *  - root    /opt/local/sbin/axtip axtip
#
[KZ2X-4 via udp0]
NOCALL   * * * * * *  L
default  * * * * * *  - root    /opt/local/sbin/ttylinkd ttylinkd -f /opt/local/etc/ax25/ttylinkd.conf
#
# NET/ROM Ports begin with a '<'.
#
<netrom>
NOCALL  * * * * * *  L
default * * * * * *  -  root  /opt/local/sbin/axtip     axtip

The service definition:

[Unit]
Description=ax25 daemon
Wants=kiss-vhf0.service kiss-udp0.service kiss-udp1.service netromd.service
After=kiss-vhf0.service kiss-udp0.service kiss-udp1.service netromd.service

[Service]
Type=forking
ExecStart=/opt/local/etc/rc.d/ax25d

[Install]
WantedBy=multi-user.target

/opt/local/etc/rc.d/ax25d:

#!/bin/sh

PATH=/opt/local/sbin:$PATH
export PATH

ax25d -c /opt/local/etc/ax25/ax25d.conf

Note that we configure service on AX.25 SSID KZ2X-1 on all ports pointing to the axtip program. This is a custom program that’s derived from Joerg Reuter (DL1BKE’s) axwrapper from the ax25-tools package. It uses the sockets interface to grab the calling user’s callsign and protocol (AX.25, NET/ROM, whatever) and invokes my “public access dial-through” program attached to a new pseudo-tty.

axpad implements a user interface that lets users connect to the machines I have running locally on my network (mostly via telnet) while handling things like buffering the output from the distant end before writing to the AX.25 or NET/ROM interface.

The ttylinkd program available at KZ2X-4 uses the BSD ntalk protocol to send a chat request to me on kz2x.ampr.org, the primary Unix machine on my AMPR subnetwork. However, ttylinkd is very buggy and this only works sometimes.

Interactive Use and Monitoring

With the configuration presented thus far, I can use the call program to connect make outbound AX.25 connections over both RF and AXUDP. For example, axcall vhf0 BROCK will connect me to the W1MV-7 digipeater in Brockton, MA via RF. Similarly, axcall udp0 ka5d-7 will connect me to the BPQ node at the University of Texas in Austin via AXUDP and axcall netrom GAMES will connect me to a European text game server via NET/ROM over AXUDP.

When logged in interactive, I often run listen to monitor AX.25 traffic and NET/ROM on VHF and the general use AXUDP interfaces. For example, to watch RF traffic in a tmux window:

: vhf1; sudo /opt/local/bin/listen -ar -tttt -p vhf0

The -a option shows outbound frames in addition to incoming data, -tttt presents a timestamp on each line, -r makes the output “readable” and -p vhf0 specifies the RF port.

: vhf1; is my shell prompt.

Problems, Observations and Debugging Tips

Putting this together was the product of a lot of Internet searching and reading the Linux AX.25 HOWTO document, along with lots of experimentation.

Something that helped greatly, particularly while debugging AXUDP, was being able to monitor traffic on my border router as it egressed my network and headed for the Internet. tcpdump is a critical tool in one’s debugging arsenal.

Sadly, the “new” style of Linux networking discarded the traditional complement of ifconfig, route, and netstat. The functionality of the former two was subsumed by the ip tool, while the latter is replaced by ss. While the new tools provide enhanced functionality relative to their venerable ancestors, ss in particular does not understand either AX.25 or NET/ROM, so to get status information about the network I had to install netstat. Caveat emptor.

AX.25 seems to work pretty reliably. I have no problems connecting to stations near my QTH.

Occasionally I have observed destructive behavior with NET/ROM connections: systems will infinitely loop sending data to my system, but that data is never acknowledged. Usually I have to restart the system to reset these connections, which is obviously not ideal. I have no idea what’s going on, but have some packet capture data if anyone would like it for debugging.

I have had to manually apply the YO2LOJ’s NET/ROM socket cleanup patch to the kernel. Prior to doing so, NET/ROM connections would not be properly cleaned up and the machine would panic on reboot.

The use of a software TNC in lieu of a hardware TNC has been useful. Software modems can employ advanced signal processing techniques to recover data from noisy RF environments, while hardware modems usually cannot. For example, Direwolf runs multiple AFSK 1200 demodulators in parallel, returning frames decode successfully by any of them. It also applies heuristics based on knowledge of the structure of AX.25 to fix one and two bit decoding errors. Also, soft modems can evolve and advance on the same hardware, while hardware is constrained to how it was built. FX.25 introduces forward error correction in a backwards compatible way into AX.25 frames: TNCs built before the invention of FX.25 cannot take advantage of this, while software can be upgraded.

Conclusion

And this concludes our description of the KZ2X packet station. Putting this together has been an interesting and educational challenge; I do wish there was more use of packet generally to make the exercise truly useful.