Reply
Contributor
catejust
Posts: 6
Registered: ‎07-11-2012

Inefficient BGP Routing in Michigan

Hello!

 

I work from my home in Ann Arbor, MI several days a week to keep my gas costs down. This is much nicer than having to drive to Southfield, MI (about 40 minutes away) every day. 

 

I have to use real-time communications to connect to my company's server hosted at 123.net's colo in Southfield. 

 

Traceroutes reveal that to reach Southfield, my data are routed from:

Ann Arbor, MI -> Taylor, MI (all Comcast)

Taylor, MI -> Ashburn, VA (all Comcast)

Ashburn, VA -> New York, NY (all Comcast)

NY, NY -> Chicago, IL (all Comcast)

Chicago, IL -> Southfield, MI (Comcast to 123.net)

 

I have a hard time believing that Comcast doesn't have any interconnects between itself and either 123.net, Level 3, or Cogent networks in Michigan. It's 1,500 miles as the crow flies (3,000 miles roundtrip!) to connect to a server 28 miles away...

 

Is there a process a customer can initiate to request a BGP route update... or at least a review of existing interconnects? My round-trip-time is anywhere from 50 - 90ms and with jitter this high, it makes voice and video communications extremely challenging... 

 

An AT&T DSL connection from anywhere in Southeastern Michigan typically has RTTs between 10 - 20ms, with jitter of only 1 or 2ms.. I understand the Cable network is nothing like a DSL connection and wouldn't expect it to be, but none the less...

 

Any help or more information would be significantly and sincerely appreciated!

Bronze Star Contributor
bnaughty
Posts: 225
Registered: ‎06-16-2012

Re: Inefficient BGP Routing in Michigan

Try configuring your router with OpenDNS or Google DNS servers...  therefore, not using the comcast DNS servers...

Contributor
catejust
Posts: 6
Registered: ‎07-11-2012

Re: Inefficient BGP Routing in Michigan

Thanks for the reply bnaughty!

 

I suppose I could have included in my post a little bit more information..

 

When I traceroute just to the server's IP address (thereby bypassing the DNS system completely), I have the same results.

 

I do, though, use one of google's IPv6 DNS as my primary resolver and one of openDNS' IPv4 as my secondary. 

 

 

That route I mentioned that my data take is the exact same with IPv4 and IPv6. I'm running dual stack on my modem and so have a native IPv6 address through Comcast as well.

 

 

When I traceroute from the server back to the public IP on my router, I end up going from 123.net through Chicago and then Chicago back to Ann Arbor. It is a slightly shorter path but still completely out of the way to go across Lake Michigan...

Bronze Problem Solver
andyross
Posts: 2,651
Registered: ‎10-17-2003

Re: Inefficient BGP Routing in Michigan


bnaughty wrote:

Try configuring your router with OpenDNS or Google DNS servers...  therefore, not using the comcast DNS servers...


A different DNS server wouldn't make a difference unless it returns a different IP address. The path TO/FROM an address has nothing to with the DNS server.

Contributor
catejust
Posts: 6
Registered: ‎07-11-2012

Re: Inefficient BGP Routing in Michigan


A different DNS server wouldn't make a difference unless it returns a different IP address. The path TO/FROM an address has nothing to with the DNS server.



Heh.. I was trying to be a bit nicer but well said. I'm on my company's network engineering team, which is why the route bugged me enough to post about it. 

Connection Expert
EG
Posts: 34,237
Registered: ‎12-24-2003

Re: Inefficient BGP Routing in Michigan

[ Edited ]

And the BGP announcement for the return path (not visible in a forward trace) is determined by the other end.

 

Routing is rarely symmetrical.

Bronze Star Contributor
bnaughty
Posts: 225
Registered: ‎06-16-2012

Re: Inefficient BGP Routing in Michigan

sorry for misinfo

Contributor
catejust
Posts: 6
Registered: ‎07-11-2012

Re: Inefficient BGP Routing in Michigan

Any word on this? Comcast?

Contributor
catejust
Posts: 6
Registered: ‎07-11-2012

Re: Inefficient BGP Routing in Michigan

Does Comcast not check these forums at all? 

 

This is really disappointing to not have had any real responses yet....

Contributor
bhamrick
Posts: 8
Registered: ‎11-19-2008

Re: Inefficient BGP Routing in Michigan

Posting an actual traceroute would be more helpful. Without any IPs its hard to start digging.

Regular Visitor
Regular Visitor
Posts: 2
Registered: ‎08-21-2006

Re: Inefficient BGP Routing in Michigan

[ Edited ]

I also noticed a month or maybe more ago that all my traffic to the Internet would exiti Michigan with the next hop always being Ashburn.  I'm in Lansing, MI.

 

I used to be able to hit chicago.voip.ms (20 ms times) and it would route to Chicago directly after the Comcast Taylor hop.  Now it routes to Ashburn to NY then to Chicago (50ms times).   Regardless of destination, my traffic always goes through Ashburn. 

 

here's an example

 

 1  192.168.10.1 (192.168.10.1)  0.952 ms  1.359 ms  1.728 ms
 2  24.11.200.1 (24.11.200.1)  19.652 ms  20.210 ms  39.927 ms
 3  te-7-6-ur01.holt.mi.michigan.comcast.net (68.85.48.89)  19.419 ms  19.734 ms  19.858 ms
 4  te-0-1-0-1-ar01.taylor.mi.michigan.comcast.net (69.139.254.125)  35.929 ms  37.332 ms  37.485 ms
 5  pos-3-5-0-0-cr01.ashburn.va.ibone.comcast.net (68.86.90.217)  53.405 ms  54.323 ms  54.472 ms
 6  pos-0-5-0-0-cr01.newyork.ny.ibone.comcast.net (68.86.89.246)  54.608 ms  43.194 ms  46.766 ms
 7  pos-0-2-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.88.138)  53.136 ms  53.269 ms  53.392 ms
 8  pos-1-4-0-0-pe01.350ecermak.il.ibone.comcast.net (68.86.86.162)  52.559 ms  52.733 ms  52.867 ms
 9  66.208.229.50 (66.208.229.50)  66.040 ms  72.500 ms  72.669 ms
10  edge1.chi2.ubiquityservers.com (72.37.148.138)  57.560 ms  57.732 ms  57.975 ms
11  64.120.22.242.ubiquityservers.com (64.120.22.242)  56.872 ms  57.034 ms  57.168 ms

 

 

 

Regular Visitor
Regular Visitor
Posts: 2
Registered: ‎08-21-2006

Re: Inefficient BGP Routing in Michigan

Here's what my comcast tracrt to www.123.net shows (which looks pretty much like the one the OP summarized)

 

 

traceroute to www.123.net (216.234.97.5), 30 hops max, 40 byte packets
 1  192.168.10.1 (192.168.10.1)  1.157 ms  1.632 ms  2.040 ms
 2  24.11.200.1 (24.11.200.1)  35.751 ms  35.943 ms  36.106 ms
 3  te-7-6-ur01.holt.mi.michigan.comcast.net (68.85.48.89)  15.742 ms  16.711 ms  16.882 ms
 4  te-0-1-0-0-ar01.taylor.mi.michigan.comcast.net (68.87.190.14)  23.641 ms  23.830 ms  27.981 ms
 5  pos-3-3-0-0-cr01.ashburn.va.ibone.comcast.net (68.86.90.105)  39.938 ms  43.071 ms  43.247 ms
 6  pos-0-15-0-0-cr01.newyork.ny.ibone.comcast.net (68.86.87.198)  43.520 ms  44.933 ms  45.110 ms
 7  pos-0-5-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.88.150)  49.858 ms  50.053 ms  50.212 ms
 8  pos-1-0-0-0-pe01.350ecermak.il.ibone.comcast.net (68.86.86.34)  48.203 ms  48.384 ms  48.541 ms
 9  cr-1.sfld-mi.123.net (66.208.233.62)  54.153 ms  61.909 ms  65.503 ms
10  ws1.123.net (216.234.97.5)  46.864 ms  52.202 ms  54.966 ms

Contributor
catejust
Posts: 6
Registered: ‎07-11-2012

Re: Inefficient BGP Routing in Michigan

[ Edited ]

Interesting... if I trace to 123.net, I get a result from Ann Arbor to Pontiac.. 

 

_________________________________________________________Packets_______________Pings
_Host__________________________________________________Loss%___Snt___Last___Avg__Best__Wrst_StDev
_1._2601:4:1dx0:4x:xxx0:xxxf:xexc:xxxe__________________0.0%____47____0.6___0.6___0.5___0.8___0.1
_2._2001:558:4020:46::1_________________________________0.0%____46___24.2__25.3__18.9__30.6___3.2
_3._te-8-2-ur03.sannarbor.mi.michigan.comcast.net_______0.0%____46___10.8__10.6___8.8__13.9___1.0
_4._te-0-9-0-6-ar01.pontiac.mi.michigan.comcast.net_____0.0%____46___16.6__16.6__13.6__20.5___1.5
_5._he-4-5-0-0-cr01.350ecermak.il.ibone.comcast.net_____0.0%____46___28.1__27.2__18.9__44.7___4.6
_6._pos-1-3-0-0-pe01.350ecermak.il.ibone.comcast.net____0.0%____46___20.3__21.8__20.3__23.8___0.8
_7._cr-1.sfld-mi.123.net________________________________0.0%____46___29.3__43.2__27.5_205.2__38.6
_8._ws1.123.net_________________________________________0.0%____46___28.2__28.9__27.3__43.1___2.3

 

 

But if I trace to cr-1.sfld-mi.123.net, the routing goes through Virginia and New York again..

_________________________________________________________Packets_______________Pings
_Host__________________________________________________Loss%___Snt___Last___Avg__Best__Wrst_StDev
_1._192.168.15.2________________________________________0.0%____23____0.6___0.6___0.4___0.7___0.1
_2._98.224.216.1________________________________________0.0%____23___24.3__24.2__10.6__38.9___5.8
_3._te-8-2-ur02.sannarbor.mi.michigan.comcast.net_______0.0%____23___10.7__10.1___8.5__11.6___0.9
_4._te-0-10-0-4-ar01.taylor.mi.michigan.comcast.net_____0.0%____23___20.8__16.8__11.2__23.7___3.9
_5._pos-3-5-0-0-cr01.ashburn.va.ibone.comcast.net_______0.0%____22___29.0__27.1__23.3__32.8___2.1
_6._pos-0-7-0-0-cr01.newyork.ny.ibone.comcast.net_______0.0%____22___32.6__32.6__30.5__34.9___1.4
_7._pos-0-1-0-0-cr01.350ecermak.il.ibone.comcast.net____0.0%____22___37.1__37.3__35.6__39.8___1.1
_8._pos-1-1-0-0-pe01.350ecermak.il.ibone.comcast.net____0.0%____22___40.1__35.4__33.9__40.1___1.5
_9._cr-1.sfld-mi.123.net________________________________0.0%____22___46.0__73.1__41.2_241.7__52.0_

 

 

Latency goes from 28ms to 46ms by adding the extra hops.