Home Networking / Router / & WiFi Gateway Help

[ryangoldstein]

I've been playing around with IPv6 settings in Tomato (specifically, Toastman 1.28.7498.1 STD) trying to get a /64 block assignment, and I haven't been successful so far.

I've tried both Native IPv6 from ISP and DHCPv6 with Prefix Delegation, with the settings shown in the respective screenshots and all reasonable variants, without success (the DHCPv6 one automatically acquires the correct IPv6 DNS servers, 2001:558:feed::1 and 2001:558:feed::2, but a WAN-facing IPv6 address is not assigned). Any ideas?

Thanks,

-Ryan

EDIT: I got 6to4 Anycast Relay working with the settings in that screenshot, but I'd much prefer native IPv6.

[wirbly]

I'm in the same boat, with the same setup.  Reading around it looks like Comcast hasn't deployed native dual-stack in all geographical areas, so I guess it's just a waiting game.

[ryangoldstein]

They've actually deployed it here (I'm in Center City Philadelphia, literally across the street from the Comcast Center, actually - I'm looking at it right now out my window!). When I plug my Windows computer into the modem directly, I'm assigned an IPv4 and IPv6 address, so it's definitely deployed here. It's just not working on my Tomato router, despite having tried every reasonable configuration.

[ComcastTuska]

---

wirbly wrote:

I'm in the same boat, with the same setup.  Reading around it looks like Comcast hasn't deployed native dual-stack in all geographical areas, so I guess it's just a waiting game.

---

I checked on the CMTS your connected to, it is ready but not enabled yet.

[ryangoldstein]

What settings should I use to try to get the /64 block assigned to my router? LAN-facing IPv6 is working fine; it's just not acquiring an IPv6 address/block from Comcast (only the IPv6 DNS servers). Reading through other forums, many Comcast/Tomato users appear to be having the same issue.

Thanks,

-Ryan

[virtuallynathan]

Are you sure tomato *actually* supports IPv6?

[ryangoldstein]

Yes, I'm quite sure that it does. As mentioned above, I'm able to get IPv6 via 6to4 Anycast Relay (but I want native, not a tunnel), and the configuration I tried for DHCPv6 with Prefix Delegation successfully acquires the IPv6 DNS servers, just not an IP address or block.

It seems like it's almost working; I'm hoping the issue is one of local configuration. For example, Tomato permits Dnsmasq custom configuration, which may be required to get this working, but I have no idea what to try (it's just a big text box representing a Dnsmasq .conf file).

Thanks,

-Ryan

[LanceMoreland]

Yes, Tomato fully supports IPv6.  I am having the same issue.  I can get an IPv6 address if I plug the computer directly in the modem but can't pull one with the router.  I am not sure but I suspect the /64 addresses have not been deployed yet.

[virtuallynathan]

It sounds like it doesn't support DHCPv6 properly.

[ComcastTuska]

---

LanceMoreland wrote:

Yes, Tomato fully supports IPv6.  I am having the same issue.  I can get an IPv6 address if I plug the computer directly in the modem but can't pull one with the router.  I am not sure but I suspect the /64 addresses have not been deployed yet.

---

Lance,

The CMTS your on it is fully deployed with v6, with that said if you pull a v6 address with your desktop directory connected to the Cable Modem your router should do the same for both address and prefix.  Just remember it needs to support DHCPv6.

[LanceMoreland]

Well Changing the setting from "Native IPv6 from ISP" to "DHCP with prefix delegation" does pull an address but running IPv6 test come up negative.  So I suppose that as some have suggested Tomato is not working properly with that setting.  I will play with it some more this weekend and maybe look into switching to ddwrt. 

[ryangoldstein]

I switched from DD-WRT because it doesn't support IPv6 at all. Tomato at least claims to (though seemingly not too well). Many have switched from DD-WRT to Tomato or back to stock firmware because of DD-WRT's lack of support for IPv6.

[Murphy]

I took a different approach.

I retired my Linksys E3000 running DD-WRT and replaced it with a Linksys EA3500.

It worked fine right out of the box.

DD-WRT does support IPv6 tunneling.  I ran it with a Hurricane Electric tunnel for a long time.

It does not support DHCPv6 and unless one of the prime authors ends up on a system that requires DHCPv6 it probably never will.

[virtuallynathan]

If you have a WNDR3700v2 or WNDR3800 I would look into Cerowrt (or use stock firmware). If you are more tech-savvy, OpenWRT can be configured to properly support DHCPv6.

[wirbly]

> I checked on the CMTS your connected to, it is ready but not enabled yet.

I appreciate the heads up.  Any idea on when they'll throw the switch?

[morrildl]

The reason that tomato doesn't get a v6 IP for the router is because it's not sending the NA flag along with the PD flag when using DHCPv6 when it hasn't received a router advertisement yet. This is non-spec behavior.

If you edit /etc/dhcp6c.conf and add this line:

 send ia-na 0;

...immediately after this line:

 send ia-pd 0;

...so that the whole file looks like this:

interface vlan2 {
send ia-pd 0;
send ia-na 0;
send rapid-commit;
request domain-name-servers;
script "/sbin/dhcp6c-state";
};
id-assoc pd 0 {
prefix-interface br0 {
sla-id 0;
sla-len 0;
};
};
id-assoc na 0 { };

...and then restart dhcp6c (ps | grep dhcp6c; kill -TERM <pid>; dhcp6c -T LL <interface>) then it picks up a v6 IP for the router in addition to the prefix delegation it gets.

Unfortunately it's still not actually forwarding any v6 packets, at least on my box. Forwarding looks like it's enabled as far as I can see, and the routes all seem okay. At this point I am not 100% sure that my CMTS is enabled, I guess, although it seems weird that I'd get an IP but not have it routable.

It would be interesting to see if this helps anyone else on a different CMTS I guess.

[morrildl]

Bah. Obviously 15 minutes after I post, I figure out what's up.

Looks like even after fixing the dhcp6c.conf file, there is still a spurious v6 default route you have to delete:

route -A inet6 del default gw :: metric 1024 vlan2 # on my box, the WAN is vlan2

I'm not 100% clear on where this comes from or why I had to delete it... I guess v6 wants default routes to go out the link-local address.

Anyway with the dhcp6c.conf change, and deleting the spurious default route, I can get to ipv6.google.com and ipv6-test.com works.

Unfortunately these aren't permanent changes -- they'll be blown away by a reboot, etc. Looks like the dhcpc6.conf file is generated by a binary(!) so it means a patch to Tomato and new build before it'll work out of the box. And I don't even yet have any idea what's generating the spurious route....

But, if you're feeling adventurous, that works for me.

[ryangoldstein]

Which Toastman version are you using? Version 1.28.7499 was just released a few days ago: http://www.4shared.com/dir/v1BuINP3/Toastman_Builds.html#dir=118755049

[koitsu]

I've brought both the dhcp6c (DHCPv6 client) and busted spurious default route to the attention of Toastman (maintainer of the TomatoUSB Toastman builds).

This matter has been discussed heavily on both the official tomatousb.org and linksysinfo.org forums recently, as it affects more than just Toastman builds (also affects Shibby):

http://tomatousb.org/forum/t-484538/comcast-ipv6-not-working-with-tomato

http://www.linksysinfo.org/index.php?threads/shibby-mod-ipv6-and-comcast.38006/

I would also like to see a response here from Comcast network engineers explaining the requirement of DHCPv6 IA-NA bit in the DHCP option header.  For non-technical readers, the IA-NA flag specifically means request a, quote, "non-temporary address".  See RFC3315 for details (search for IA_NA):

http://tools.ietf.org/html/rfc3315

I imagine this probably impacts more than just TomatoUSB routers too, and might explain quite a bit.  Given that Comcast engineers (the ones I've talked to out of Englewood) have been insisting that "many routers are broken, our stuff works just fine with Windows 7", this non-standard behaviour puts the ball back into Comcast's court.  TL;DR -- be very careful blaming routers when your own configuration (and documentation -- or lack thereof) requires something that's a bit unorthodox.  You should, bare minimum, document the IA-NA requirement somewhere.  It would save people a lot of headaches.

[koitsu]

---

ryangoldstein wrote:

Which Toastman version are you using? Version 1.28.7499 was just released a few days ago: http://www.4shared.com/dir/v1BuINP3/Toastman_Builds.html#dir=118755049

---

Build 7499 has nothing to do with this problem nor does it address anything pertaining to this issue.  Please read the ChangeLog that's available at the link you posted before recommending people upgrade their firmware; again, build 7499 does not address either the dhcp6c config issue or the spurious/incorrect route issue.

[ryangoldstein]

My apologies; I'm quite new to Tomato. I also didn't intend to imply that 7499 fixes the issue, just that it's something to consider, as it was just released. I hadn't had time to try it myself, and an update to dnsmasq 2.61 was indicated in the changelogs (dnsmasq 2.61 changelogs), which could certainly affect IPv6 functionality (but it's now clear that it doesn't fix this issue).

Reading about this on other forums, it seems to be a result of Comcast implementing IPv6/DHCPv6 in a non-standard way (yet somewhat functional with select manufacturers). I agree with the point you made elsewhere that Comcast should provide more complete documentation and/or support DHCPv6-PD in a more standard-compliant manner.

Thanks for contacting Toastman and otherwise documenting your research here and elsewhere.

[koitsu]

1. dnsmasq has nothing to do with the dhcp6c client.  DHCP client requests (between the router and the ISP) are handled 100% within dhcp6c, not within dnsmasq.

2. dnsmasq has nothing to do with default routes.

dnsmasq is a lightweight DNS proxy server for your LAN; clients on the LAN can use 192.168.1.1 (your router IP address) as a DNS server, and dnsmasq will siphon requests to the DNS servers delegated to the router by Comcast.  In other words, a machine on your home network (say 192.168.1.200) uses 192.168.1.1 as its DNS server.  192.168.1.200 does a DNS lookup for www.yahoo.com, and asks 192.168.1.1.  192.168.1.1's dnsmasq software sees the request and then asks the Comcast DNS servers handed to it during DHCP phase (when the router came online), which usually are 75.75.75.75 and 75.75.76.76.  Those servers respond with the result, which goes back through dnsmasq and back to the LAN client.

Make sense?

The part that confuses most users is this: dnsmasq also includes a built-in DHCP server, which is what LAN clients talk to when asking the router for an LAN IP address.  This is completely separate from the DHCP client that runs on the same router talking to the ISP for a WAN IP.  For DHCPv4 requests (from the router to the ISP) the udhcpc client is used.  For DHCPv6 requests (from the router to the ISP) the dhcp6c client is used.

So to recap and make it more clear:

* udhcpc -- daemon which asks ISP for a public (WAN) IPv4 address

* dhcp6c -- daemon which asks ISP for a public (WAN) IPv6 address

* dnsmasq -- daemon which provides DNS proxy services for LAN clients (both IPv4 and IPv6), as well as provides a DHCP server for LAN clients (definitely IPv4 and probably IPv6)

All these daemons are running simultaneously.

[ryangoldstein]

Yup, got it. Thanks for the explanation. If I had more free time these days, I'd have done all that research myself.

Here's to hoping that Comcast and the open router firmware devs take note of this issue and work toward a good long-term solution!

[koitsu · Accepted solution]

I've written a WAN Up script as a workaround for this problem:

http://www.dslreports.com/forum/r27235017-

Hope this helps folks.

[Morac2]

Is there a way to tell if DHCPv6 / IPv6 is enabled on your CMTS or not?  I'd like to try getting this working, but I can't even pull a IPV6 address.

[koitsu]

The only way is to ask a Comcast CSR or network engineer and provide them with your geographic location (city, state, and zip code), and ask if IPv6 has been rolled out to your area.

I ranted about Comcast's lack of rollout transparency quite recently.  A CC engineer (who shall remain nameless) contacted me privately and was in agreement that disclosure of locations which have IPv6 really should be made more apparent.

In the interim, a customer came up with this interesting thing using Google Maps.  Here's the thread where that was disclosed.

[ComcastTuska]

---

Morac2 wrote:

Is there a way to tell if DHCPv6 / IPv6 is enabled on your CMTS or not?  I'd like to try getting this working, but I can't even pull a IPV6 address.

---

I checked your CMTS, it is ready but not enabled yet.  So it should be soon hopefully.

[LanceMoreland]

Amarkey came up with a fix: http://www.linksysinfo.org/index.php?threads/ipv6-and-comcast.38006/

It is a one line script for WAN Up. Post number 22.

[koitsu]

And to expand on what Lance just said -- the thread over on DSLReports/BroadbandReports where I've been documenting this ordeal has also been updated:

http://www.dslreports.com/forum/r27234575-TomatoUSB-and-Comcast-IPv6-bugs-found

The most important part (for readers here anyway), aside from the workarounds, is the fact that the previously-mentioned-here DHCPv6 IA-NA modification is not required to get working IPv6.  The root cause at this point usually consists of either the spurious route bug (which morrildl found in this thread), or certain checkboxes not being set inside of TomatoUSB's IPv6 page.

Please be sure to read both the workaround pages mentioned in the above link to understand what all is required to get IPv6 working with TomatoUSB on Comcast.

[ryangoldstein]

Thanks so much for all your hard work on this (koitsu and everyone else)! This will hopefully be rolled into TomatoUSB (or at least Toastman's build) shortly, such that only DHCPv6 needs to be selected to get it working.

So awesome. Thank you!

[adamgundy]

can somebody in Comcast confirm whether or not my CMTS is enabled for IPv6 yet? I see IPv6 'neighbor solicitation' and 'router advertisement' packets (and tomato seems to pick up those routes etc), but my DHCPv6 requests are ignored.

I'm in Aurora, CO 80017. the CMTS MAC is 00:01:5c:24:88:01

[adamgundy]

---

adamgundy wrote:

can somebody in Comcast confirm whether or not my CMTS is enabled for IPv6 yet? I see IPv6 'neighbor solicitation' and 'router advertisement' packets (and tomato seems to pick up those routes etc), but my DHCPv6 requests are ignored.

I'm in Aurora, CO 80017. the CMTS MAC is 00:01:5c:24:88:01

---

don't know if somebody poked the Aurora CMTS for me.. or I got very lucky with the timing. some time in the last hour or so, my router just grabbed an IPv6 address. awesome!

[adamgundy]

---

adamgundy wrote:

---

adamgundy wrote:

can somebody in Comcast confirm whether or not my CMTS is enabled for IPv6 yet? I see IPv6 'neighbor solicitation' and 'router advertisement' packets (and tomato seems to pick up those routes etc), but my DHCPv6 requests are ignored.

I'm in Aurora, CO 80017. the CMTS MAC is 00:01:5c:24:88:01

---

don't know if somebody poked the Aurora CMTS for me.. or I got very lucky with the timing. some time in the last hour or so, my router just grabbed an IPv6 address. awesome!

---

OK, this is not working as well as I hoped. IPv6 will *sometimes* grab a range of addresses, then it will lose connectivity somewhere between 2 and 5 minutes after it brings it up.

I've been poking around a bit, and it seems to be related to neighbor discovery - either the tomato router is not sending neighbor discovery packets to the CMTS, or the CMTS is not replying to them (not sure yet). the neighbor table shown by 'ip neigh' shows the CMTS either as 'FAILED' or 'INCOMPLETE'.

I poked the neighbor table in tomato like this:

 ip neigh change fe80::201:5cff:fe24:8801 lladdr 00:01:5c:24:88:01 nud permanent dev vlan2

(which is a horrible hack.. you have to know the IPv6 address AND the MAC address of the CMTS to do this), and wa-la - working IPv6 that doesn't drop out after a few minutes...

of course the minute comcast change the CMTS address or MAC it will all fail badly.

[MrKiasu]

---

adamgundy wrote:

---

OK, this is not working as well as I hoped. IPv6 will *sometimes* grab a range of addresses, then it will lose connectivity somewhere between 2 and 5 minutes after it brings it up.

I've been poking around a bit, and it seems to be related to neighbor discovery - either the tomato router is not sending neighbor discovery packets to the CMTS, or the CMTS is not replying to them (not sure yet). the neighbor table shown by 'ip neigh' shows the CMTS either as 'FAILED' or 'INCOMPLETE'.

I poked the neighbor table in tomato like this:

 ip neigh change fe80::201:5cff:fe24:8801 lladdr 00:01:5c:24:88:01 nud permanent dev vlan2

(which is a horrible hack.. you have to know the IPv6 address AND the MAC address of the CMTS to do this), and wa-la - working IPv6 that doesn't drop out after a few minutes...

of course the minute comcast change the CMTS address or MAC it will all fail badly.

---

Hi,

This is exactly the behaviour I'm seeing as well. It'll go live briefly and then die for many minutes at a time, then come back for a few, and on and on. I did a brief test with a single PC connected to the modem and the connection came straight up and seemed stable, so I'm not sure its an upstream issue, although I thought it was. Perhaps some incompatibility between tomatoUSB and the CMTS.

I tried watching the neighbour table until I saw:

fe80::201:5cff:fe32:3181 dev vlan2 lladdr 00:01:5c:32:31:81 router STALE

and then added a permanent mapping as you did, but it hasn't restored my connectivity.

[morrildl]

Oh, nice, I had been trying to figure out how to report this somewhere that Toastman or somebody else would see it, but the Tomato forum situation in general is pretty hard for a n00b to figure out. Thanks!!

[adamgundy]

---

MrKiasu wrote:

---

adamgundy wrote:

---

OK, this is not working as well as I hoped. IPv6 will *sometimes* grab a range of addresses, then it will lose connectivity somewhere between 2 and 5 minutes after it brings it up.

I've been poking around a bit, and it seems to be related to neighbor discovery - either the tomato router is not sending neighbor discovery packets to the CMTS, or the CMTS is not replying to them (not sure yet). the neighbor table shown by 'ip neigh' shows the CMTS either as 'FAILED' or 'INCOMPLETE'.

I poked the neighbor table in tomato like this:

 ip neigh change fe80::201:5cff:fe24:8801 lladdr 00:01:5c:24:88:01 nud permanent dev vlan2

(which is a horrible hack.. you have to know the IPv6 address AND the MAC address of the CMTS to do this), and wa-la - working IPv6 that doesn't drop out after a few minutes...

of course the minute comcast change the CMTS address or MAC it will all fail badly.

---

Hi,

This is exactly the behaviour I'm seeing as well. It'll go live briefly and then die for many minutes at a time, then come back for a few, and on and on. I did a brief test with a single PC connected to the modem and the connection came straight up and seemed stable, so I'm not sure its an upstream issue, although I thought it was. Perhaps some incompatibility between tomatoUSB and the CMTS.

I tried watching the neighbour table until I saw:

fe80::201:5cff:fe32:3181 dev vlan2 lladdr 00:01:5c:32:31:81 router STALE

and then added a permanent mapping as you did, but it hasn't restored my connectivity.

---

phew... not just me then!

check that the entry is now 'permanent' in the table. you should be able to ping6 the router's address now, like this:

ping6 fe80::201:5cff:fe32:3181%vlan2

assuming that works, try restarting the dhcp6 server. it can take some time to realize that connectivity is back, so restarting it should make things work in a few seconds...

mine's been stable for an hour and a half now, which beats the previous record of five minutes..

as far as I can tell, tomato is NOT sending 'neighbor solicitation' packets on vlan2 interface. doesn't matter what (link local) address you ping, tcpdump sees nothing. if you ping LAN machines on the br0 interface, it sends them and works fine! I can't find a reason for the difference, though. sigh.

[MrKiasu]

---

adamgundy wrote:

---

phew... not just me then!

check that the entry is now 'permanent' in the table. you should be able to ping6 the router's address now, like this:

ping6 fe80::201:5cff:fe32:3181%vlan2

assuming that works, try restarting the dhcp6 server. it can take some time to realize that connectivity is back, so restarting it should make things work in a few seconds...

mine's been stable for an hour and a half now, which beats the previous record of five minutes..

as far as I can tell, tomato is NOT sending 'neighbor solicitation' packets on vlan2 interface. doesn't matter what (link local) address you ping, tcpdump sees nothing. if you ping LAN machines on the br0 interface, it sends them and works fine! I can't find a reason for the difference, though. sigh.

---

Bingo, that's done it. At least that narrows down the problem a lot. I'm contemplating trying to compile

NDPMon for tomato, but I don't relish that thought..

[adamgundy]

---

MrKiasu wrote:

---

adamgundy wrote:

---

phew... not just me then!

check that the entry is now 'permanent' in the table. you should be able to ping6 the router's address now, like this:

ping6 fe80::201:5cff:fe32:3181%vlan2

assuming that works, try restarting the dhcp6 server. it can take some time to realize that connectivity is back, so restarting it should make things work in a few seconds...

mine's been stable for an hour and a half now, which beats the previous record of five minutes..

as far as I can tell, tomato is NOT sending 'neighbor solicitation' packets on vlan2 interface. doesn't matter what (link local) address you ping, tcpdump sees nothing. if you ping LAN machines on the br0 interface, it sends them and works fine! I can't find a reason for the difference, though. sigh.

---

Bingo, that's done it. At least that narrows down the problem a lot. I'm contemplating trying to compile

NDPMon for tomato, but I don't relish that thought..

---

aha! I've found what's happening to the neighbor solicitations. they are ending up on eth0, instead of vlan2. it's a routing issue! if you look at the v6 routing table (route -A inet6), there is a massive long list of routes for 'broadcast' addresses that have been received (or autogenerated, not sure) that all look like this:

ff02::1:ff1f:26df/128 ff02::1:ff1f:26df UC 0 3 0 vlan2
ff02::1:ff1f:46a2/128 ff02::1:ff1f:46a2 UC 0 1 0 vlan2
ff02::1:ff1f:77a8/128 ff02::1:ff1f:77a8 UC 0 15 0 vlan2
ff02::1:ff20:1c6c/128 ff02::1:ff20:1c6c UC 0 1 0 vlan2
ff02::1:ff23:c4f1/128 ff02::1:ff23:c4f1 UC 0 6 0 vlan2
ff02::1:ff24:bda/128 ff02::1:ff24:bda UC 0 1 0 vlan2
ff02::1:ff24:8801/128 ff02::1:ff24:8801 UC 0 2 0 vlan2
ff02::1:ff24:8802/128 ff02::1:ff24:8802 UC 0 10 0 eth0
ff02::1:ff24:8802/128 ff02::1:ff24:8802 UC 0 4 0 vlan2
ff02::1:ff25:d28c/128 ff02::1:ff25:d28c UC 0 12 0 vlan2
ff02::1:ff26:ac23/128 ff02::1:ff26:ac23 UC 0 12 0 vlan2
ff02::1:ff28:a3d8/128 ff02::1:ff28:a3d8 UC 0 15 0 vlan2
ff02::1:ff29:3ac9/128 ff02::1:ff29:3ac9 UC 0 1 0 vlan2
ff02::1:ff2a:527f/128 ff02::1:ff2a:527f UC 0 12 0 vlan2
ff02::1:ff2a:9a46/128 ff02::1:ff2a:9a46 UC 0 3 0 vlan2

look carefully at the target interfaces.. there's ONE which is directed at eth0. and that happens to be the critical one (NOTE: it will probably be a different one to mine, depending on your routers address). now we just need to figure out why that one is wrong...

[adamgundy]

---

adamgundy wrote:

aha! I've found what's happening to the neighbor solicitations. they are ending up on eth0, instead of vlan2. it's a routing issue! if you look at the v6 routing table (route -A inet6), there is a massive long list of routes for 'broadcast' addresses that have been received (or autogenerated, not sure) that all look like this:

ff02::1:ff24:8802/128 ff02::1:ff24:8802 UC 0 10 0 eth0 

look carefully at the target interfaces.. there's ONE which is directed at eth0. and that happens to be the critical one (NOTE: it will probably be a different one to mine, depending on your routers address). now we just need to figure out why that one is wrong...

---

sigh. IPv6 is just too complex for its own good. those routes seem to be generated on demand by something (the kernel? radvd? who knows). the odd thing is, that eth0 rule will *only* show up if you first attempt to contact it (eg ping - I pinged the CMTS address, with the last digit changed to test). so something, somewhere is generating those routes immediately before they are needed, and its putting in the wrong interface...

[adamgundy]

OK, killing dhcp6 and restarting it has busted things again for me. even with the 'fixed' MAC address for the CMTS, it still wants do neighbor discovery on eth0 instead of vlan2, so DHCP6 fails all the time.

must have got lucky one time.

[MrKiasu]

---

adamgundy wrote:

OK, killing dhcp6 and restarting it has busted things again for me. even with the 'fixed' MAC address for the CMTS, it still wants do neighbor discovery on eth0 instead of vlan2, so DHCP6 fails all the time.

must have got lucky one time.

---

It's worked reliably for me, but that's with me going all the way back to the beginning of the process each time.

# killall dhcp6c

# ip neigh del fe80::201:5cff:fe32:3181 lladdr 00:01:5c:32:31:81 nud permanent dev vlan2

# ip neigh change fe80::201:5cff:fe32:3181 lladdr 00:01:5c:32:31:81 nud permanent dev vlan2

# ping6 fe80::201:5cff:fe32:3181%vlan2

# dhcp6c -T LL vlan2

Interestingly, it *still* fails even if I delete eth0's ipv6 link-local address - so that's impossible to route anything through eth0. The neighbour discovery still fails and the needed broadcast route is never added. I'm sure that explains something about where its going wrong, but I don't know what. ipv6 really is a strange beast.

[adamgundy]

---

MrKiasu wrote:

---

adamgundy wrote:

OK, killing dhcp6 and restarting it has busted things again for me. even with the 'fixed' MAC address for the CMTS, it still wants do neighbor discovery on eth0 instead of vlan2, so DHCP6 fails all the time.

must have got lucky one time.

---

It's worked reliably for me, but that's with me going all the way back to the beginning of the process each time.

# killall dhcp6c

# ip neigh del fe80::201:5cff:fe32:3181 lladdr 00:01:5c:32:31:81 nud permanent dev vlan2

# ip neigh change fe80::201:5cff:fe32:3181 lladdr 00:01:5c:32:31:81 nud permanent dev vlan2

# ping6 fe80::201:5cff:fe32:3181%vlan2

# dhcp6c -T LL vlan2

Interestingly, it *still* fails even if I delete eth0's ipv6 link-local address - so that's impossible to route anything through eth0. The neighbour discovery still fails and the needed broadcast route is never added. I'm sure that explains something about where its going wrong, but I don't know what. ipv6 really is a strange beast.

---

there's something very odd going on. I had to drop the 'permanent' address to recover the connection.

if you 'watch ip neigh' then generate some IPv6 traffic (eg ping6 google.com) you can see the cache stepping through various states about every 10 seconds - DELAY, PROBE, REACHABLE (or FAILED). but the most interesting thing is that you can see it testing BOTH vlan2 and eth0 - eth0 always fails (of course), but it's a race between the two. as long as they are both being tested, there are route entries (which you can see with route -A inet6), and that seems to be somewhat OK, but if one of the routes stays active (due to traffic) all future tests go that way, and it can get stuck.

I made it somewhat more stable by increasing /proc/sys/net/ipv6/neigh/vlan2/mcast_solicit to 10 (from 3), which makes it try harder to get an address from vlan2. it will still fail eventually though.

the real issue is that it is trying both eth0 and vlan2 to find the address, but I have no idea why. if you force a ping out of vlan2, it doesn't do that. so something about the way the routed traffic is being generated is causing it to try both NICs.

[adamgundy]

---

adamgundy wrote:

 

the real issue is that it is trying both eth0 and vlan2 to find the address, but I have no idea why. if you force a ping out of vlan2, it doesn't do that. so something about the way the routed traffic is being generated is causing it to try both NICs.

---

OK, so here's where I am currently. I have an IPv6 connection that seems to be rock solid, running for about 40 minutes so far (and 20 minutes prior to that before I messed it up ;-))

the issue seems to be that for some reason, when neighbor solicitation needs to happen on vlan2, in some cases, it uses the FIRST INTERFACE IN THE ROUTE LIST. which is not vlan2. it doesn't seem to have this problem on other interfaces, so it may well be an issue with dhcp6c binding to the wrong interface to make its queries (or not binding to ANY interface).

my 'fix' is this:

1) kill dhcp6c so that there are only link local addresses on each interface

2) now remove, then re-add the link local address on br0, eth0, and vlan1. they all have the same address, so you can do it easily once you've found the address using ifconfig:

ifconfig (to show all the addresses, find the link local one from any of br0, eth0, vlan1)

ip route del fe80::3246:9aff:fea5:bdb0/64 dev br0

ip route add fe80::3246:9aff:fea5:bdb0/64 dev br0

ip route del fe80::3246:9aff:fea5:bdb0/64 dev eth0

ip route add fe80::3246:9aff:fea5:bdb0/64 dev eth0

ip route del fe80::3246:9aff:fea5:bdb0/64 dev vlan1

ip route add fe80::3246:9aff:fea5:bdb0/64 dev vlan1

(swap in your own link address, obviously.. they're all the same)

now check:

ip -6 route

at the end of the list you should see something like this:

ff00::/8 dev vlan2 metric 256 mtu 1500 advmss 1440 metric 10 4294967295
ff00::/8 dev eth1 metric 256 mtu 1500 advmss 1440 metric 10 4294967295
ff00::/8 dev eth0 metric 256 mtu 1500 advmss 1440 metric 10 4294967295
ff00::/8 dev vlan1 metric 256 mtu 1500 advmss 1440 metric 10 4294967295
ff00::/8 dev br0 metric 256 mtu 1500 advmss 1440 metric 10 4294967295

note that vlan2 is now at the top of the list.

now, restart dhcp6c

dhcp6c -T LL vlan2

wait a couple of seconds... IPv6 should be up and running.

obviously this is not the 'correct' fix. but it seems to work!

[adamgundy]

still good at 4+ hours. looks like it really is 'fixed'.

I'm wondering.. are the other people seeing this (MrKiasu, others?) using wireless-N routers? I believe the interface layout is somewhat different on a wireless-G router, so maybe the ordering in the route table is different, too..

[MrKiasu]

---

adamgundy wrote:

still good at 4+ hours. looks like it really is 'fixed'.

I'm wondering.. are the other people seeing this (MrKiasu, others?) using wireless-N routers? I believe the interface layout is somewhat different on a wireless-G router, so maybe the ordering in the route table is different, too..

---

Mine's a Linksys E2000. I suspect the specific physical and vlan configuration affects this. I tried your delete and add trick with the routes, but I ended up with the route table in the same order at the end, so the neighbour discovery failed in the usual way. Only the permanent trick works for me.

[adamgundy]

this fix is obsolete.. see the next post

can you paste the 'ip -6 route' output here please. basically, you should have something like this:

ff00::/8 dev eth0 metric 256 mtu 1500 advmss 1440 metric 10 4294967295
ff00::/8 dev vlan1 metric 256 mtu 1500 advmss 1440 metric 10 4294967295
ff00::/8 dev br0 metric 256 mtu 1500 advmss 1440 metric 10 4294967295

ff00::/8 dev vlan2 metric 256 mtu 1500 advmss 1440 metric 10 4294967295
ff00::/8 dev eth1 metric 256 mtu 1500 advmss 1440 metric 10 4294967295

so dropping then re-adding eth0, vlan1, br0 puts them to the bottom of the list so you end up with this:

ff00::/8 dev vlan2 metric 256 mtu 1500 advmss 1440 metric 10 4294967295
ff00::/8 dev eth1 metric 256 mtu 1500 advmss 1440 metric 10 4294967295
ff00::/8 dev eth0 metric 256 mtu 1500 advmss 1440 metric 10 4294967295
ff00::/8 dev vlan1 metric 256 mtu 1500 advmss 1440 metric 10 4294967295
ff00::/8 dev br0 metric 256 mtu 1500 advmss 1440 metric 10 4294967295

NOTE: they won't move places if there are other IPv6 addresses on the interface - that's why you have to kill dhcp6c first, to remove any global addresses.

[adamgundy]

OK. the issue is a kernel bug that causes 'neighbor solicitation' packets to get misrouted if they get marked up for QOS - in other words, if you turn on QOS, and use IPv6, it will be flaky.

the fix (workaround) is to copy this into Administration->Scripts->Firewall:

# prevent neighbor solicitations and other low level ICMP messages from being 'marked', since
# it seems to trigger a kernel bug that prevents them being sent on the correct interface
ip6tables -t mangle -I OUTPUT 1 -o `nvram get wan_iface` -p icmpv6 \! --icmpv6-type echo-request -j RETURN

[MrKiasu]

---

adamgundy wrote:

OK. the issue is a kernel bug that causes 'neighbor solicitation' packets to get misrouted if they get marked up for QOS - in other words, if you turn on QOS, and use IPv6, it will be flaky.

the fix (workaround) is to copy this into Administration->Scripts->Firewall:

# prevent neighbor solicitations and other low level ICMP messages from being 'marked', since
# it seems to trigger a kernel bug that prevents them being sent on the correct interface
ip6tables -t mangle -I OUTPUT 1 -o `nvram get wan_iface` -p icmpv6 \! --icmpv6-type echo-request -j RETURN

---

Nice one. That fix has worked for me. I'm guessing this has been fixed upstream? Do you know which kernel release?

[Ingenium]

So I followed all these steps on Toastman's build. I did the firewall rule and the WAN up script, and my router gets an IPv6 address (a /64 on br0 and a /128 on vlan2). I can successfully ping6 ipv6.google.com from the router via ssh. My computer though gets 2 /64 IPv6 addresses per interface (wired and wireless) for some reason in addition to the usual fe:80 address. However, does not actually have any internet IPv6 connectivity (pinging ipv6.google.com or 2001:4860:4001:800::1012). I can successfully ping the router via it's IPv6 address. When I do a traceroute6 to 2001:4860:4001:800::1012 (ipv6.google.com), it shows it going to the router and then stops. 

So it seems like either the router isn't correctly forwarding the IPv6 packets, or Comcast isn't accepting them for some reason. Any thoughts?

EDIT: Nevermind, rebooting my router seemed to fix it. 

[WRuser]

I am running v1.28.782 mipsr1-toastman-nd-k26-miniipv6 on a Linksys WRT54Gv2 (or at least trying).

The changes to /etc/dhcp6c.conf get an error: deamon error dhcp6c[926]:add_options:/etc/dhcp6c.conf 3

ia_na 0 is not defined

Doing a search for the dhcp6c.conf file and dhcp6c comes up with quite a few things to try, none of the

ones I tried would work - thay all produced errors.

My experence with Comcast in the Denver burbs is: you get the wan interface address and a prefix using

dhcp. (The tow have different prefixes!) The default IPv6 route comes from the source address from RAs.

I have a firewall running Linus (xubuntu) on a Neoware thin client using the above method/proceedure.

It appears that dhcp6c can not get an address for the wan  and a prefix for the lan. At least all that I have tried will not work.

Where is the version of dhcp6c that tomato/toastman uses, documented??? Is it??

I got the info on how comcast works from John Brzozowski a Comcast IPv6 person (engineer?). He has authored several presentations on Comcasts IPv6 rollout.