Which Way Do I Go? (Effects on Routing When the Network Changes)

As a general rule, all WIN-T nodes route traffic in pretty much the same way using pretty much the same configuration.  We all know that at a CPN, both SIPR and NIPR traffic goes out the nodes TDMA link because, well, it has no other way to go.  For a JNN, if the traffic is going to one of our subordinate units it goes out our TDMA, and if it is going somewhere else, it likely goes out our FDMA.  We understand what it does, but most people have stopped thinking about why it does that.  This is fine and dandy until we start making changes to the network.

WIN-T increment 1 (A and B) operate exclusively with Open Shortest Path First (OSPF) as our routing protocol.  OSPF picks the path that it sends packets based on the “cost” of each link between two points (the lower the cost, the better).  While OSPF is perfectly capable of determining the cost of links itself, WIN-T statically configures those costs.  It is these costs on these particular links that make the traffic flow in our network the way it does.

Note:  The architecture of the RHN is different from that of the JNN/CPN.  For ease of understanding, I am using the same names that we do for the JNNs/CPNs and leaving out a few intermediate devices in the signal flow, however, the overall OSPF costs are accurate.

The above diagram shows a portion of our NIPR network with only TDMA links up.  So why does data flow the way it does?  Well, the short answer is because it’s the only way for it to go.  In this example, and the following, we will examine data as it moves from NIPR tier 2 router to NIPR tier 2 router.  In this first example, we’ll look at data going from pretty much any JNN/CPN going to another node or the RHN.  For example NIPR traffic going from the JNN to the CPN originates at the JNN NT2R and moves down to the AES router with an OPSF cost of 460.  Once at the AES router, it goes out the TDMA tunnel with a cost of 1050 and arrives at the CPN’s AES router where it then flows up to the NT2R with another cost of 460.   Total cost for the trip is 1970.  Likewise for data moving to the RHN, the trip is exactly the same.

Now let’s add in our FDMA link between the JNN and RHN and examine the data flow again.  Again moving from the JNN to the CPN routes exactly the same as before with a total cost of 1970, but when we send data to the RHN, now we have two paths; TDMA as before or FDMA.  Following TDMA the cost exactly the same as before, 1970.  However, now since we have a direct link between N2TR to NT2R with a cost of only 1050, it is cheaper (by 920) to take the FDMA link instead of TDMA.

So why are we talking about this, so far I would hazard to guess that I haven’t said anything that you didn’t already know?

As many of you probably know, I am a huge fan of HCLOS and almost always advocate its use.  However, when we add HCLOS to the network, we are adding additional paths for data to flow which can have both intended and unintended consequences.  For this example, we will cost the HCLOS link to the WIN-T baseline value of 100.

So again, when we send data from the JNN to the RHN we take our FDMA link with a cost of 1050 instead of the combined TDMA cost of 1970.  Now when we send data from the JNN to the CPN we have two paths; TDMA of 1970 or the direct router to router HCLOS link with a cost of 100.  Guess which way it goes.  Again, so far nothing earth shattering.

But now lets look at it from a slightly different angle and this time send data from the CPN to the RHN like when our users want to check Facebook or go to AKO (I’ll let you guess which website produces more traffic during a typical rotation).  As usual, the CPN has a combined TDMA path cost to the RHN of 1970, and that’s it…..Right?  Actually now that we have introduced HCLOS into the picture we have given (maybe unknowingly) the data a second path to the RHN through the JNN.  In this case, the cost is 100 across the HCLOS and then 1050 out the FDMA interface for a total cost of 1150 vs. the 1970 for TDMA.  Which way will it go?

So is this a big deal?  And if so, why did WIN-T do it this way?  Well, the answer to the first one is “maybe” and the answer to the second one is “who the hell knows”.  Having your CPN route to the RHN through the JNN’s FDMA can be good or bad depending on the situation.  For example, let’s say that you have an under-utilized TDMA link that averages less than 50% utilization on average.  In this case, sending the CPN’s traffic may well be a good choice because we are freeing up bandwidth on TDMA for other units to use for what they need.  On the other hand, if the FMDA link is already frequently saturated, adding additional network traffic to it is probably not the best idea in the world.

The bottom line is that we as net techs have to recognize what will happen when we add a link into the network, realize what the impacts of that change are, and then decide if it will adversely impact the network and if so, how to mitigate it.  So with that in mind, let’s say that routing traffic across FDMA is not a good option.  How do we fix it while still allowing the traffic to route to the JNN using the HCLOS link?

Well, the way I see this happen more often than not is that we just start pulling random numbers out of the air until we find one that does what we want it to.  While that is “a way”, it may well adversely impact the network while we attempt to discover that magic number and is definitely not the most efficient way to do it.  So instead, let’s try using some math to figure out the magic number on paper.  Well, our cost has two conditions it has to meet.  First, that the cost of that link plus the cost of the FDMA link has to be greater than 1970 (x+1050 > 1970) so that our TDMA link is cheaper than the HCLOS/FDMA link.  Second, that the cost of the link itself has to be less than 1970 (x <1970) otherwise TDMA between the JNN and CPN is cheaper than the HCLOS link.  So that means that the cost of that link must be between 921 and 1969 which gives us a fair bit of room.  I would recommend giving yourself a little bit of room in either direction in case you have to make adjustments later on in life (I personally usually start with 950 just so I have wiggle room).