IPV6 Routing Issue

[ietf@…]

I have the following situation to report:

1.  I go to the web site "http://coap.me"
2.  I give that web site the IPV6 address of my laptop and request that it
run coap against it.
3.  I get the coap request from the server and send out a response.  I have
verified this with wireshark
4.  The web site does not ever get the response and does a re-try operation
against my server.  (which it receives and processes).

Jim

[cdoyle@…]

bi-directional IPv6 testing all looks good, but that doesn't mean there isn't a problem.

Would you mind providing a bit more info on your issue, please? What IPv6 address are you sourcing your traffic from? This info will help me to test from your specific network. Also, if you can provide the packet capture (filter out the coap packets if you would), that would help me test bidi communication by identifying the destination. Second, would you mind being more specific about your testing process? I've not used the coap.me service before and I want to make sure I'm replicating your process.

BTW, I live in PDX and I've been to August Cellars. I wouldn't presume your affiliation based on your email address, but for what it's worth, it's a beautiful property and has some really lovely wines.

Cheers

Colin in the IETF NOC

[ietf@…]

Here is the wireshark from a run of the tests this morning.

Coap.me is running at the university of Bremen.  It is designed to visit the address using the protocol to get the list of resources and then visit each resource.  That is what it is doing by trying to visit the well-known resource.

At Coap.me, I place the IP address of my laptop in the "Crawl CoAP Server on IP Address" text box and then press the "Query" button on that line.  This will then visit my server with a set of UDP packets to which I respond.

Looking at the wireshark - you see the request online 1, response on line 2 - this is reported as a timeout on the website an resends the request several more times but only one additional response from my system.   I can configure to do more responses but have not at this time.

Yes, I am the winemaker and owner of August Cellars winery, it is always nice to get complements.

Jim



> -----Original Message-----
> From: IETF Tickets/NOC [mailto:tickets@meeting.ietf.org]
> Sent: Tuesday, March 28, 2017 8:23 AM
> To: undisclosed-recipients:
> Subject: Re: [IETF Tickets/NOC] #1092: IPV6 Routing Issue
> 
> #1092: IPV6 Routing Issue
> --------------------------+--------------------------------
>       Reporter:  ietf@…   |                Owner:  cdoyle@…
>           Type:  defect   |               Status:  accepted
>       Priority:  minor    |            Milestone:  ietf-98
>      Component:  network  |           Resolution:
>       Keywords:           |  My Current Location:
> My MAC  Address:          |                My OS:
> --------------------------+--------------------------------
> Changes (by cdoyle@…):
> 
>  * owner:  panda@… => cdoyle@…
>  * priority:  tbd => minor
>  * status:  assigned => accepted
>  * component:  wireless => network
> 
> 
> Comment:
> 
>  bi-directional IPv6 testing all looks good, but that doesn't mean there  isn't a
> problem.
> 
>  Would you mind providing a bit more info on your issue, please? What IPv6
> address are you sourcing your traffic from? This info will help me to test  from
> your specific network. Also, if you can provide the packet capture  (filter out the
> coap packets if you would), that would help me test bidi  communication by
> identifying the destination. Second, would you mind being  more specific about
> your testing process? I've not used the coap.me  service before and I want to
> make sure I'm replicating your process.
> 
>  BTW, I live in PDX and I've been to August Cellars. I wouldn't presume  your
> affiliation based on your email address, but for what it's worth,  it's a beautiful
> property and has some really lovely wines.
> 
>  Cheers
> 
>  Colin in the IETF NOC
> 
> --
> Ticket URL: <https://tickets.meeting.ietf.org/ticket/1092#comment:2>
> IETF Tickets/NOC <https://tickets.meeting.ietf.org>
> IETF Meeting Tickets - NOC pages

Y.pcapng​

[ietf@…]

Added by email2trac

[cdoyle@…]

Well, your destination is reachable from the core routers:

colin@RtrA> ping 2001:638:708:30da:223:24ff:fe93:e128
PING6(56=40+8+8 bytes) 2001:1890:c00:e666::111b:9435 --> 2001:638:708:30da:223:24ff:fe93:e128
16 bytes from 2001:638:708:30da:223:24ff:fe93:e128, icmp_seq=0 hlim=53 time=126.090 ms
16 bytes from 2001:638:708:30da:223:24ff:fe93:e128, icmp_seq=1 hlim=53 time=124.707 ms
16 bytes from 2001:638:708:30da:223:24ff:fe93:e128, icmp_seq=2 hlim=53 time=118.950 ms

And a trace looks complete and healthy

RtrA (::)(tos=0x0 psize=64 bitpattern=0x00) Tue Mar 28 14:26:10 2017
Keys: Help Display mode Restart statistics Order of fields quit

Packets Pings

Host Loss% Snt Last Avg Best Wrst StDev?

1. 2001:1890:c00:e666::ee1b:9435 0.0% 4 5.1 2.3 1.3 5.1 1.8
2. cgcil22crs.ipv6.att.net 0.0% 4 1.7 3.2 1.7 4.9 1.4
3. cgcil403igs.ipv6.att.net 25.0% 4 5.4 6.0 3.7 8.9 2.7
4. be3039.ccr41.ord03.atlas.cogentco.com 0.0% 4 2.5 9.8 1.9 32.4 15.1
5. be2765.ccr41.ord01.atlas.cogentco.com 0.0% 4 4.7 2.6 1.8 4.7 1.4
6. ???
7. be2993.ccr21.yyz02.atlas.cogentco.com 0.0% 4 16.6 17.1 16.3 18.5 1.0
8. be2090.ccr21.ymq01.atlas.cogentco.com 0.0% 4 25.3 25.7 25.3 26.6 0.7
9. be3042.ccr21.lpl01.atlas.cogentco.com 33.3% 4 95.7 96.3 95.7 96.8 0.8

10. be2182.ccr41.ams03.atlas.cogentco.com 0.0% 4 108.5 111.9 102.0 134.6 15.4
11. be2815.ccr41.ham01.atlas.cogentco.com 50.0% 4 110.7 110.7 110.7 110.8 0.0
12. be2198.rcr11.b015814-1.ham01.atlas.cogentco.com 0.0% 4 108.7 108.8 108.7 108.9 0.1
13. 2001:978:2:8::c:2 33.3% 4 120.3 129.1 120.3 137.9 12.4
14. cr-han2-be7.x-win.dfn.de 33.3% 4 120.6 119.1 117.7 120.6 2.1
15. kr-bre66.x-win.dfn.de 0.0% 3 143.7 127.3 118.9 143.7 14.2
16. 2001:638:708:ff::e 0.0% 2 167.7 143.4 119.0 167.7 34.4
17. 2001:638:708:ff::26 0.0% 2 128.4 126.5 124.6 128.4 2.7
18. zoo.informatik.uni-bremen.de 0.0% 2 119.1 122.7 119.1 126.3 5.1

Your packet capture looks good, with src/dst IP and MAC look good.

Port 5683 looks good from our network and we know your source is ok since you are sending acks...

dhcp-0056:~ colin$ sudo nmap -6 -sU -p 5683 2001:67c:370:128:1cc7:423f:a774:4b59 -Pn

Starting Nmap 6.47 ( ​http://nmap.org ) at 2017-03-28 09:39 CDT
Nmap scan report for t2001067c037001281cc7423fa7744b59.v6.meeting.ietf.org (2001:67c:370:128:1cc7:423f:a774:4b59)
Host is up (2.0s latency).
PORT STATE SERVICE
5683/udp filtered unknown

Nmap done: 1 IP address (1 host up) scanned in 11.08 seconds
dhcp-0056:~ colin$ sudo nmap -6 -sU -p 5683 2001:638:708:30da:223:24ff:fe93:e128

Starting Nmap 6.47 ( ​http://nmap.org ) at 2017-03-28 09:40 CDT
Nmap scan report for zoo.informatik.uni-bremen.de (2001:638:708:30da:223:24ff:fe93:e128)
Host is up (0.12s latency).
PORT STATE SERVICE
5683/udp open|filtered unknown

I have to assume your traffic is reaching the Internet since we aren't filtering anything at the network perimeter. Unfortunately, we do not have a switch or passive sniffer between our core router and the uplinks, so I cannot sniff the packets on the wire. UDP v6 traceroutes on port 5683 take an enormous amount of time to pass through Cogent's AS, which might account for your timeouts, but i don't like that as an explanation since I see similar performance using common ports and the website certainly loads properly.

I have to confess I'm a bit stumped. There shouldn't be anything keeping your packet from going out the front door, and I don't see much once it hits the Internet that would keep it from getting back to coap server. I can tell you that they are prepending the heck out of their peering via Cogent. Peering with HE, DTAG, and a few others look normal.

If you would, let's eliminate one last variable. Can you go to the terminal room and plug into the wire so we can make sure our controller isn't doing something goofy?

[ietf@…]

Same effect when plugged into the network.

Jim


> -----Original Message-----
> From: IETF Tickets/NOC [mailto:tickets@meeting.ietf.org]
> Sent: Tuesday, March 28, 2017 10:18 AM
> To: undisclosed-recipients:
> Subject: Re: [IETF Tickets/NOC] #1092: IPV6 Routing Issue
> 
> #1092: IPV6 Routing Issue
> --------------------------+--------------------------------
>       Reporter:  ietf@…   |                Owner:  cdoyle@…
>           Type:  defect   |               Status:  accepted
>       Priority:  minor    |            Milestone:  ietf-98
>      Component:  network  |           Resolution:
>       Keywords:           |  My Current Location:
> My MAC  Address:          |                My OS:
> --------------------------+--------------------------------
> 
> Comment (by cdoyle@…):
> 
>  Well, your destination is reachable from the core routers:
> 
>  colin@RtrA> ping 2001:638:708:30da:223:24ff:fe93:e128
>  PING6(56=40+8+8 bytes) 2001:1890:c00:e666::111b:9435 -->
>  2001:638:708:30da:223:24ff:fe93:e128
>  16 bytes from 2001:638:708:30da:223:24ff:fe93:e128, icmp_seq=0 hlim=53
>  time=126.090 ms
>  16 bytes from 2001:638:708:30da:223:24ff:fe93:e128, icmp_seq=1 hlim=53
>  time=124.707 ms
>  16 bytes from 2001:638:708:30da:223:24ff:fe93:e128, icmp_seq=2 hlim=53
>  time=118.950 ms
> 
> 
>  And a trace looks complete and healthy
> 
>  RtrA (::)(tos=0x0 psize=64 bitpattern=0x00)  Tue Mar 28 14:26:10 2017
>  Keys:  Help   Display mode   Restart statistics   Order of fields   quit
>  Packets               Pings
>   Host
>  Loss%   Snt   Last   Avg  Best  Wrst StDev
>   1. 2001:1890:c00:e666::ee1b:9435
>  0.0%     4    5.1   2.3   1.3   5.1   1.8
>   2. cgcil22crs.ipv6.att.net
>  0.0%     4    1.7   3.2   1.7   4.9   1.4
>   3. cgcil403igs.ipv6.att.net
>  25.0%     4    5.4   6.0   3.7   8.9   2.7
>   4. be3039.ccr41.ord03.atlas.cogentco.com
>  0.0%     4    2.5   9.8   1.9  32.4  15.1
>   5. be2765.ccr41.ord01.atlas.cogentco.com
>  0.0%     4    4.7   2.6   1.8   4.7   1.4
>   6. ???
>   7. be2993.ccr21.yyz02.atlas.cogentco.com
>  0.0%     4   16.6  17.1  16.3  18.5   1.0
>   8. be2090.ccr21.ymq01.atlas.cogentco.com
>  0.0%     4   25.3  25.7  25.3  26.6   0.7
>   9. be3042.ccr21.lpl01.atlas.cogentco.com
>  33.3%     4   95.7  96.3  95.7  96.8   0.8
>  10. be2182.ccr41.ams03.atlas.cogentco.com
>  0.0%     4  108.5 111.9 102.0 134.6  15.4
>  11. be2815.ccr41.ham01.atlas.cogentco.com
>  50.0%     4  110.7 110.7 110.7 110.8   0.0
>  12. be2198.rcr11.b015814-1.ham01.atlas.cogentco.com
>  0.0%     4  108.7 108.8 108.7 108.9   0.1
>  13. 2001:978:2:8::c:2
>  33.3%     4  120.3 129.1 120.3 137.9  12.4
>  14. cr-han2-be7.x-win.dfn.de
>  33.3%     4  120.6 119.1 117.7 120.6   2.1
>  15. kr-bre66.x-win.dfn.de
>  0.0%     3  143.7 127.3 118.9 143.7  14.2
>  16. 2001:638:708:ff::e
>  0.0%     2  167.7 143.4 119.0 167.7  34.4
>  17. 2001:638:708:ff::26
>  0.0%     2  128.4 126.5 124.6 128.4   2.7
>  18. zoo.informatik.uni-bremen.de
>  0.0%     2  119.1 122.7 119.1 126.3   5.1
> 
>  Your packet capture looks good, with src/dst IP and MAC look good.
> 
>  Port 5683 looks good from our network and we know your source is ok since
> you are sending acks...
> 
>  dhcp-0056:~ colin$ sudo nmap -6 -sU -p 5683
>  2001:67c:370:128:1cc7:423f:a774:4b59 -Pn
> 
>  Starting Nmap 6.47 ( http://nmap.org ) at 2017-03-28 09:39 CDT  Nmap scan
> report for t2001067c037001281cc7423fa7744b59.v6.meeting.ietf.org
>  (2001:67c:370:128:1cc7:423f:a774:4b59)
>  Host is up (2.0s latency).
>  PORT     STATE    SERVICE
>  5683/udp filtered unknown
> 
>  Nmap done: 1 IP address (1 host up) scanned in 11.08 seconds  dhcp-0056:~
> colin$ sudo nmap -6 -sU -p 5683
>  2001:638:708:30da:223:24ff:fe93:e128
> 
>  Starting Nmap 6.47 ( http://nmap.org ) at 2017-03-28 09:40 CDT  Nmap scan
> report for zoo.informatik.uni-bremen.de
>  (2001:638:708:30da:223:24ff:fe93:e128)
>  Host is up (0.12s latency).
>  PORT     STATE         SERVICE
>  5683/udp open|filtered unknown
> 
>  I have to assume your traffic is reaching the Internet since we aren't  filtering
> anything at the network perimeter. Unfortunately, we do not have  a switch or
> passive sniffer between our core router and the uplinks, so I  cannot sniff the
> packets on the wire. UDP v6 traceroutes on port 5683 take  an enormous
> amount of time to pass through Cogent's AS, which might  account for your
> timeouts, but i don't like that as an explanation since I  see similar performance
> using common ports and the website certainly loads  properly.
> 
>  I have to confess I'm a bit stumped. There shouldn't be anything keeping  your
> packet from going out the front door, and I don't see much once it  hits the
> Internet that would keep it from getting back to coap server. I  can tell you that
> they are prepending the heck out of their peering via  Cogent. Peering with HE,
> DTAG, and a few others look normal.
> 
>  If you would, let's eliminate one last variable. Can you go to the  terminal
> room and plug into the wire so we can make sure our controller  isn't doing
> something goofy?
> 
> --
> Ticket URL: <https://tickets.meeting.ietf.org/ticket/1092#comment:4>
> IETF Tickets/NOC <https://tickets.meeting.ietf.org>
> IETF Meeting Tickets - NOC pages

Z.pcapng​

[ietf@…]

Added by email2trac

[colin.doyle@…]

I'll talk to some of the other routing guys in the NOC, but for now I can say that this issue looks like it's something between here and there, not within our routing domain. Sorry I don't have a resolution for you.

If you would like to talk about this, you can stop by the NOC on the 40th floor and ask for Colin. I'll be here for another 50 minutes or so today.

[colin.doyle@…]

I'm changing the status of this ticket to "pending". If you would like to work on this a bit, please stop by the NOC on the 40th floor.

[Rick Alfvin]

Milestone renamed