New startup and echo_env scripts

Redesigned echo_env and startreflect to cope with situations where a channel may be misconfigured due to a typo in echo_config, or where the integrated conference configuration may be invalid (missing echo_env, or missing tbd.conf/tlb.conf, or non numeric port value). In these instances, the system will revert to using sfreflect for the channel. The checks aren't perfect, but should catch a number of common issues.

New configure script and bug fixes.

Wrote a new script to prepare an integrated conference channel for use. This script creates all necessary directories and symbolic links. All that needs to be done after running make_integrated is to configure tbd or tlb and set echo_config .

Fixed a bug in the ACL management scripts which created a bogus ACL entry in tbd or tlb under certain conditions. The bug fix will also remove the bogus entry if it is found, so there is no need to manually clean up tbd/tlb ACLs after installing this bug fix.

Bug found by VE7LTD squashed

Dave reported a bug to me this morning, which caused sfreflect to start on the wrong ports under certain conditions. The fix proved to be simple. The cause of the problem was a conflict between one of the variables used in the startreflect script, and one in the local environment of the tbd/tlb channels. Renaming the variable in startreflect fixed the problem.

More bug fixes!

Installed the new reflector code on reflector 9500 (with the blessing of the administrator). I had intended to wait until after Dave had a chance to review the code, but the upgrade was brought forward due to issues at the site. Anyway, this gave me a chance to test the reflector startup properly, and I found a few bugs which have now been fixed.

The bugs found were:

Fixed a bug in the reflector startup that prevented integrated conferences from being started. This was a stupid typo (one missing character!). Ths bug prevented the next bug from being detected earlier.

Check that channels configured to run tbd or tlb are not already running, before attempting to start them.

Modified one of the maintenance scripts to correctly count the new integrated channels, and subtract them from the total number of reflector channels, to determine whether the correct number of sfreflect copies are running.

Minor bugfixes and the new Back Bar!

Just sorted out a couple of minor bugs on the integrated conference. As it turns out, these weren't bugs in the code, but instead were due to a subtle configuration error in tbd.conf, which caused some issues with the security on some connections. Reconfiguring tbd for the offending channel resolved the issue, after quite a bit of head scratching.

The Virtual Pub now has a new "back bar" - reflector 9550 is now permanently linked to reflector 9500, so the Virtual Pub can be found at either location. For efficiency reasons, some internal links were reconfigured to suit the new layout.

Currently, there are now 4 tlb or tbd channels on the new reflector, and two of these are integrated channels. The current configuration of the reflector is:

ref9550 - Virtual Pub "back bar", running tlb, but using the packet reflector (not transcoding) and ADPCM codec. No Echolink support (but linked to an external transcoder outside of virtual pub hours).

ref9551 - Running sfreflect

ref9552 - Running sfreflect

ref9553 - Running sfreflect

ref9554 - Running sfreflect

ref9555 - Running tlb as a transcoding reflector. No Echolink support yet (need another IP address).

ref9556 - Running sfreflect

ref9557 - Running sfreflect

ref9558 - Running tbd as an integrated conference (GSM only). Echolink *VK3JED*

ref9559 - Running tbd as an integrated conference (GSM only). Echolink *AUSSIE*

ref9559 is handling fairly heavy traffic with around a dozen connections most of the time.

Minor tweak

Connected to reflector 9559 today, and after a while, thought things were a bit quiet. Discovered that the audio path had become disconnected at the reflector end, and the security system had blocked the node's attempts to reestablish audio.

Worked around this by calling the "kicknode" script to disconnect the control side of the IRLP node, if the audio connection ends with an "rtcp_timeout" (which means either a network problem, or the reflector initiated a disconnection). This also means that if a sysop or admin accidentally uses .disconnect or .kick instead .ikick, the node will eventually be disconnected completely.

However, I'm not 100% satisfied with this solution, because it means that connections with transient minor packet loss may disconnect, rather than recover gracefully. I will monitor and see how important this issue is. At least the node will be completely kicked if the audio path dies, which is better than before.

Integrated Reflector status Sep 16

Completed the EchoLink commands for controlling connected IRLP nodes and tested them. All the commands work on both EchoLink and IRLP nodes, with the scripts responding appropriately for each node type.

Commands supported are:

.ikick - disconnects the specified node

.imute - mutes the specified node and adds that node to the reflector channel's mute list.

.iunmute - unmutes the specified node and removes that node from the reflector channel's mute list.

.iban - allows nodes to be banned (blocked) and unbanned (unblocked).

.ikick, .imute and .iunmute are sysop level commands, .iban is an admin level command. This is in line with their inbuilt equivalents.

Upgraded tlb on the system to the latest beta (0.44). Waiting on the new beta of tbd to upgrade this software as well.

I also observed that the conference killer seems to be working perfectly on the new system. :D

Reflector update Sep 15

Tested and debugged the transcoding conference support. This is now working on ref9555.

Implemented the first of the administrative commands, a sysop level command called "ikick", which will kick the IRLP or Echolink node specified. Getting the code right was a pain!

More features and bugfixes!

Today, I implemented full duplex/transcoding reflector channels. The port from exp0018 was pretty straightforward. Just have to test it at some stage.

Support for persistent mute lists and unmute lists is now in place. During the development of this feature, I discovered a couple of serious bugs in the listen only channel support code, which have now been fixed. Mute support has been tested and is working.

Progress with new IRLP/EchoLink Integration.

Spent yesterday working on the new reflector, adding EchoLink integration to the system. These modifications will bring integrated conferences in line with the current IRLP reflector design. The changes that are being implemented are:

New security model. The new reflector handles security differently, so the integration scripts need to work with the new security subsystem. This work is complete and in beta testing. The integrated conferences may be even more secure than regular IRLP reflector channels. They are more particular about the correct node ID being used on a link.

Multiple integrated conferences per reflector host. The new system uses a set of common scripts to implement multiple integrated conferences. The scripts detect whether a channel uses tlb or tbd, instead of the normal sfreflect, then by loading a local environment, they tailor their behaviour for each integrated conference. The only constraint is that there must be an IP address available for each integrated conference. This support is working and is in final beta testing, with two tbd channels running, and plans for a tlb transcoding channel for future tests.

Integrated multiconference control scripts. The "conference killer" scripts, which have featured on some reflectors and EchoLink conferences have been modified to work under the common script model, and manage multiple integrated conferences. This feature can be enabled or disabled on a per channel basis. The conference killer is running, and is under beta test.

Management of IRLP nodes from EchoLink - EchoLink users with sysop and admin privileges will be able to kick, ban and unban IRLP nodes from their EchoLink client, using commands similar to the ones they already know. The scripts to manage IRLP nodes have been written, but the EchoLink event handling has to be added, so that the new commands will be recognised and the scripts triggered.

Support for listen only channels. The new reflector code from VE7LTD supports listen only channels. The new integration code supports this feature, so that integrated channels will become listen only, if the listen only flag is set. Both IRLP and EchoLink nodes will be muted on connection, when listen only is activated for an integrated conference. This feature has been implemented, but needs to be tested. I would also like to add mute list and unmute list features, so specific nodes can be listed as muted on a regular channel, or unmuted on a muted channel. These extra features will be specific to integrated conferences.

Transcoding and full duplex conferences. The new integrated conferences can be configured to support both transcoding between codecs and full duplex. This feature requires tlb to be used in a specific configuration, and support is activated by a symbolic link to the port linking scripts in the conference's channel directory. Support is yet to be implemented. The port linking scripts will be ported from the exp0018/VK3JED-R experimental node. Transcoding conferences are inherently full duplex (this has been proven through testing on exp0018).

New IRLP Reflector

On Friday, Dave Cameron VE7LTD installed IRLP Reflector 9550 on the Adelaide server. Still got some technical hitches, which I've managed to work around for development purposes.

The new reflector is reflector 9550 - 9559, and it will be integrated into the national network.

New infrastructure for VK

With WICEN Victoria showing some interest in IRLP and EchoLink, I've started a programme of creating some redundant infrastructure within Australia for the VoIP based modes. I had been running a number of EchoLink conferences for some time, and decided it was time to setup a second complete set of reflectors for IRLP and D-STAR as well. This provides redundancy, in case one of the Australian reflectors fails for some reason.

The new D-STAR reflector is REF023. Information about the new system is at http://dstar.vkradio.com .

The new IRLP reflector is coming soon. Details will be provided as soon as they are known. The IRLP reflector will also be used for developing a new set of IRLP-Echolink integration scripts, to streamline installation of such systems, and to make them compatible with the latest IRLP reflector code. The existing EchoLink conferences will be integrated into the new reflector, which will streamline the Australian/Irish "EchoCloud" network.

Let's not forget DV Data

My previous post on D-STAR <--> legacy interoperability focused on the voice side of things. However, D-STAR is more than just voice. There is also a data channel to consider, which could be useful. Now, what could we do with the data channel to interact with other systems?

1. Link the data channel to an EchoLink conference. If software such as D-RATS is mediating the link, text messages could be filtered out from other data such as file transfers and GPS co-ordinates and then passed onto the EchoLink side. And EchoLink messages could be converted to D-RATS text messages and sent to the D-STAR radio. Other options for bridging D-RATS messaging include Internet based services, such as a private Jabber server, open only to hams, APRS messages or packet radio converse bridges.

2. DPRS GPS data is already being gated to the APRS network, so there is already some interoperability with the data channel today.

3. File transfers and other non text data require a bit more thought. Some of this could be passed to a Jabber based system (since Jabber supports file transfers).

In any case, this integration of D-STAR and non D-STAR messaging and data transfer will require similar policies and access controls to that discussed for voice bridges.

D-STAR <--> IRLP/Echolink Interoperability

Recently, there has been a lot of talk about the place (if any) analogue and legacy linking systems such as IRLP and EchoLink have in the D-STAR world. This was precipitated by some unauthorised testing conducted on D-STAR reflectors and gateways by the author of one of the EchoLink compatible packages. As one would expect, the D-STAR administrators who were affected were very upset at this development, and there is now a lot of tension surrounding this issue. Some D-STAR people want nothing to do with analogue, and make various arguments, from the annoyance of improperly configured repeater links, to incompatibilities with the way D-STAR works. For the most part, these arguments have a lot of merit.

However, I believe that keeping both systems totally separate is short sighted, but on the other hand, uncontrolled cross-system linking is just going to annoy a lot of people and render D-STAR, and possible the legacy systems unusable. However, in the middle ground, there is room for some limited and controlled interconnection between EchoLink/IRLP and D-STAR, as well as standalone FM systems. For me, this is not a new issue. Many in the IRLP and EchoLink worls will remember that I was one of the primary developers of EchoIRLP, and I am also responsible for the integrated IRLP/EchoLink conferences, such as IRLP reflector 9219/*WX_TALK*, which is depended upon during the US hurricane season. Like now, initially, there was a lot of hostility towards linking IRLP to EchoLink, but these issues were resolved, by adopting a set of guidelines that respected the rights of system owners to control what traffic they see, and allow flexibility for interconnection. The original discussion and guidelines from 2002 are still on the web at http://vkradio.com/irlp-echolink.html .

In the same spirit, I would like to offer a similar set of guidelines that those interested in integrating D-STAR with other networks can follow. While D-STAR is quite different, and the feature set has less overlap with IRLP, EchoLink or standalone FM systems, I believe there is still some room to interconnect the different systems under specific circumstances. Again, the emphasis will be on giving the respective admins crontrol over their systems, while offering the users some options.

To save double handling, I will now post the guidelines, which I've copied and pasted from an email I sent to the dstarsoftware Yahoo group on November 19, 2008.

After an email discussion with a German amateur friend from IRLP, I managed to collect a few thoughts on the whole D-STAR - IRLP/Echolink issue. By borrowing a few ideas from EchoIRLP and the whole IRLP/Echolink interoperability issue, I have formulated a similar set of proposals for D-STAR. I would like to throw the ideas here for consideration:

First, the proposal attempts to meet the following guidelines:

1. No non-D-STAR system can connect to a D-STAR reflector or gateway without the owner's approval. The owners must be the ultimate authority of what connects to their systems.

2. There are two levels of interoperability - one requires gateway modification, but offers more flexibility. The other involves no gateway modifications, but is limited to designated places where the different systems can "mingle".

3. No accidental crosslinks are possible. For example, I don't want it to be possible to connect via Echolink to my local D-STAR gateway and then be broadcast on the national D-STAR reflector. Lockouts would need to be built in to prevent this. Similarly, no deliberate crosslinks should be possible, especially ones where the remote gateway could be commanded to make a bridge. While bridging is sometimes desirable for unusual circumstances, this should always be a manual operation directly controlled by the gateway or reflector owner.

For a more thorough explanation of these rules (until I get around to throwing up a blog with updated information), the following page should give an idea of where I'm coming from. (Note added Nov 25 - this is that page :D ).

http://vkradio.com/irlp-echolink.html

While this was written 6 years ago in the context of IRLP and Echolink, I believe the underlying principles are still valid here.

Anyway, here goes, first with a recap on what happened between IRLP and Echolink.

There ended up being 3 scenarios for IRLP <--> Echolink interoperability.

1. EchoIRLP - allows individual IRLP nodes to also become native Echolink nodes.

2. Linked conferences - IRLP reflector channels linked to Echolink conferences. One of the best known examples of this configuration is the VK National Network. The Western Reflector (925x) also has a few linked channels.

3. Integrated/shared conferences - These allow both IRLP and Echolink stations to connect directly to the same physical server. This gives more transparency and control over operations. Example IRLP 9219/*WX_TALK*, which is used for US hurricane nets.


Admins should have the right to decide what passes through their system. That was one of the cornerstones of EchoIRLP and the whole IRLP-Echolink interoperability ideals.


The issues back then were posted at http://vkradio.com/irlp-echolink.html

, which helped put a lot of node owners' minds at ease. Once people realised what EchoIRLP was trying to do, opinion rapidly changed from hostile to "this is a great idea". Not sure how many EchoIRLP nodes there are today, but seems to be a lot around. Scott's work has added to that count also, as a number of EchoIRLP like nodes now use rtpDir. :) There are 999 subscribers to the EchoIRLP Yahoo group as of November 2008.

Anyway, at this stage, we can say there are two workable (possibly more will be discovered) interoperability scenarios for D-STAR and IRLP/Echolink.

1. Linked reflectors/conferences. Setup a dedicated reflector or two on the D-STAR side, and also on the IRLP/Echolink side and link them together using rtpDir (or other suitable software). I'm more than happy to provide resources and run a D-STAR reflector. I could possibly add a new Echolink conference as well for the project. My transcoding hardware and software would be available for formal trials (I don't want to leave the big system on 24x7, and also want to leave my resources free for emergency operations). With some co-operation from Robin AA4RC to install and configure the new reflector, this scenario is feasible today.

As no such reflector exists on the D-STAR side, I would like to offer my services to host a reflector dedicated to cross system linking on a high bandwidth site. I plan on using the 3 channels as follows:

A - EMCOMM and formal exercises only - cross links would be ones I setup or permit.
B - General ragchew. A single crosslink between this channel and an appropriate Echolink conference. Other than providing resources, I don't anticipate having much input into the workings here.
C - Open testing. This channel would have no limits on who can setup links. The idea being to have a place to test setups. Normal amateur courtesy would apply.

2. Multi system node. This would require rtpDir (or alternative), a DV Dongle and possibly a new Gateway addon (similar to dplus) to be installed at the gateway. The addon would manage the interaction between dplus and rtpDir to prevent accidental cross links to the general dplus network (similar to how the EchoIRLP control scripts work). The Dongle is required, because the gateways have no audio processing hardware on board, and we need raw (PCM) audio to be able to transcode to the formats used by IRLP and Echolink. rtpDir, of course, would manage the connections to Echolink and IRLP (hmm, running IRLP on a gateway, that would be an interesting technical project).

Challenges:

1. Echolink callsigns are often invalid in the D-STAR world. Might have to use node numbers instead. For example, to link to *VKEMCOMM*, you might need to have users setup their radios like:

MY VK3JED
UR 270177 L
R1 VK3RWN B
R2 VK3RWN G

which translates to
"Connect to Echolink node number 270177".

For an IRLP connection, you would have something like the above, but with

UR STN6390L

to connect to my IRLP node, or

UR REF9508L


Or alternatively, to avoid clashes with the existing dplus codes, an IRLP connection could be specified like:

UR 9508 I

or

UR REF9508I

where I stands for "connect to IRLP", and a similar suffix could be used for Echolink to avoid clashing with dplus.

UR 270177 K (for echolinK, don't want to use E, since that already has special meaning in dplus).
to connect to IRLP reflector 9508.

Not sure if the dplus ^^^^^^UL command would be appropriate to drop the connection or if there would be conflicts. Robin would be the person to consult there.

2. Getting IRLP and the gateway software to play nicely on the same box for IRLP connections. Parts of IRLP are needed to be able to connect to the IRLP network.

Anyway, just throwing a few thoughts into the ring. I'd like to see this issue resolved for the benefit of all. For Gateway operators like Mike who doesn't want any analogue traffic passing through their systems, by not installing the add ons and locking out the interconnected reflector(s), you can keep analogue traffic off your systems. Other operators can decide how much they want to participate.