Verizon PA ADSL High Ping Time Issue

emuman100

Hello,

I have an odd issue with ADSL which has becoming more frequent as of late. I experience high ping times (500-700ms or more) instead of the normal 30-40ms, and general sluggish internet, despite being up the street from the CO (PTTNPAPI). In those cases, ADSL signal parameters look great as well as the ATM layer. To fix it, Verizon DSL support would “refresh the IP address”, which causes the modem to retrain and I get a new IP address. Pings return to normal and the Internet is usable. In most cases, this would have to be done once in a blue moon, but now is frequent. It has to be done at least once a day, sometimes twice in a service call. The poor DSL tech support at Verizon is “not permitted” to give me any technical detail as to what they are doing when they “refresh my IP address”. Anyone have a clue as to what is going on?

Thanks.

pjsutton

I just put this on your other thread…

What you describe might be a byproduct of the lousy software in the combo modem/router. I had that problem, with the Actiontec. It gets to a point when there are too many devices connected that it can’t handle anymore, craps out, and restarts itself. It’s a memory leak in the firmware somewhere. I’ve long since put the modem in bridge mode and used my own router. Actually, the D-Link seems to work better in bridge mode IMO.

I think you can force the “refresh” just by restarting the modem.

I did just have a problem, though, where my upload speeds were very low, like 0.1-0.3 or sometimes even lower. Sending any kind of email was failing. That was about a week-10 days ago. It was intermittent. A tech came out and tested my line, said it was great, then went down to the remote terminal but not sure what he did. It’s been good for a few days, though.

emuman100

I really appreciate the reply! I agree, those “craptiontec” gateways are terrible, worse than the Westell gateways. I don’t use one of those, however. I use a Cisco NIM-VAB-A NIM module on a Cisco ISR4321 enterprise router and was using that since I ordered the service almost two years ago. With Cisco’s IOS, it allows me to see various information about the link, including the ATM, PPPoE, and PPP layers. In my debugging, I cannot find anything physically wrong with any of those layers. There is a software firmware minor bug (I believe undocumented) where the module stops passing traffic if the ATM link is not up for a long time (shutdown interface or unplugged), and PPPoE packets don’t get sent across the line until I issue a reset command to the NIM. Other than that, it’s been rock solid reliable as a modem.

When this issue occurs, it starts without warning and I get high ping times and sometimes no ping replies back, and appears to be every internet host. As a result, use of the internet, such as browsing, is sluggish. When I issue a reset command to my modem, it reboots (it contains an embedded Linux running on it) and retrains with the same IP address. In a few unrelated occasions, that would fix it, but hasn’t as of late. I even tried the previous generation Cisco ISRG2 with the ADSL/VDSL EHWIC module and it exhibited the same high ping time issue, but since it was a different mac address, the IP address changed. I had used this for years at another DSLAM without any issues, so I knew it was still good.

But in every case, the ADSL physical layer, ATM, PPPoE, and PPP layers are all good without any issue and this ran reliably for almost two years with next to no issues other than the high ping time issue. It’s a mystery to me, and I don’t know if there is something wrong related to the IP layer within Verizon’s network. Their limited support can’t figure it out and neither can I.

Thanks!

emuman100

Another really odd thing about this is that for the past three days at least, the high ping times would return regularly at 8PM. I know this because I have the router send one ping to three different hosts on the internet once every 5 seconds. It tracks whether these pings succeed or fail and if they fail, it produces a message notification. At 8 PM the past three days, I see the notification of failed pings:

Aug 12 20:01:40.588: %TRACK-6-STATE: 100 ip sla 100 reachability Up -> Down
Aug 12 20:01:40.588: %TRACK-6-STATE: 101 ip sla 101 reachability Up -> Down
Aug 12 20:01:40.588: %TRACK-6-STATE: 103 ip sla 103 reachability Up -> Down
Aug 12 20:01:40.753: %TRACK-6-STATE: 1 list boolean or Up -> Down

Now, sometime after 8PM, I’d have them call and do whatever they do that causes the modem to retrain and a new IP address issued, so I don’t know if it is something that expires at that time or if it changes depending on the time it’s done, but I find that interesting. Once they do what they do, the reachability status goes from down to up and everything works like it should again.

pjsutton

emuman100 well, that’s beyond my knowledge 🤣 but I’m glad someone else is using the DSL network like I am!

What are your typical speeds like? Above 9 Mbps?

emuman100

Above my knowledge too, but I’ve been using Cisco enterprise routers at home for years, just for the fun of it, so I learned a lot playing with them. My first Cisco router was a 1721 with ADSL WIC, then 1841 with ADSL HWIC, then the 1941 with ADSL EHWIC (adsl2+ annex a support) and the ADSL NIM with the 4321 currently, so I’ve had ADSL from Verizon since 2003. I’m on the 10/1 speed profile, probably like you are, and I get those speeds without issue, including upstream. The CO PITTNPAPI is right down the street. According to the signal measurements, the line is usually capable of over 20mbit down. Here is the modem controller information:

sh cont vdsl 0/1/0
Controller VDSL 0/1/0 is UP

Daemon Status: UP

                    XTU-R (DS)              XTU-C (US)

Chip Vendor ID: ‘BDCM’ ‘IFTN’
Chip Vendor Specific: 0×0000 0×71C5
Chip Vendor Country: 0xB500 0xB500
Modem Vendor ID: ‘CSCO’ ‘ ’
Modem Vendor Specific: 0×4602 0×0000
Modem Vendor Country: 0xB500 0×0000
Serial Number Near: FOC<snip> 4321/K9 16.9.3
Serial Number Far:
Modem Version Near: 16.9.3
Modem Version Far: 0×71c5

Modem Status: TC Sync (Showtime!)
DSL Config Mode: ADSL2+
Trained Mode: G.992.5 (ADSL2+) Annex A

TC Mode: ATM
Selftest Result: 0×00
DELT configuration: disabled
DELT state: not running

Failed full inits: 0
Short inits: 0
Failed short inits: 0

Modem FW Version: 4.14L.04
Modem PHY Version: A2pv6F039t.d26d
Modem PHY Source: bootflash:
Modem PHY FW Pkg: nim_vab_phy_fw_A39t_B39g1_Bond39t.pkg

Line 0:

                    XTU-R (DS)              XTU-C (US)

Trellis: ON ON
SRA: enabled enabled
SRA count: 0 0
Bit swap: enabled enabled
Bit swap count: 25 0
Line Attenuation: 27.0 dB 14.7 dB
Signal Attenuation: 20.5 dB 15.0 dB
Noise Margin: 12.9 dB 11.3 dB
Attainable Rate: 23948 kbits/s 1168 kbits/s
Actual Power: 0.0 dBm 11.0 dBm
Total FECC: 1896 0
Total ES: 0 0
Total SES: 0 0
Total LOSS: 0 0
Total UAS: 80 80
Total LPRS: 0 0
Total LOFS: 0 0
Total LOLS: 0 0

              DS Channel1     DS Channel0   US Channel1       US Channel0

Speed (kbps): NA 11768 NA 1157
SRA Previous Speed: NA 0 NA 0
Previous Speed: NA 11768 NA 1166
Total Cells: NA 186295696 NA 18269846
User Cells: NA 28525 NA 32064
Reed-Solomon EC: NA 1896 NA 0
CRC Errors: NA 0 NA 0
Header Errors: NA 0 NA 0
Interleave (ms): NA 8.29 NA 2.96
Actual INP: NA 4.14 NA 0.50

Training Log : Stopped
Training Log Filename : flash:vdsllog.bin

pjsutton

emuman100 I’m not supposed to be in 10/1, because I’m on a remote terminal which is usually locked at 3/768k, but I know someone and they turned me up. But, as a result, I don’t get that speed all the time. But it is often higher than 3. I think my remote only has 12 Mbps feeding into it.

emuman100

I found the solution to this! Embarrassingly, there was nothing wrong on Verizon’s end. I was speaking to one agent who actually used a useful diagnostic tool to observe live traffic numerically and saw there was quite a bit of traffic and it was saturating the upstream. They had always said this, but this time it was live. I disabled the modem and brought the interface down and the traffic stopped. I began to contemplate that it could be some rogue traffic that I did not know about. Since nothing was connected, I was stumped.

I did “debug ip packet” on the router and saw these hosts flooding the IOS DNS server. After disabling the IOS DNS server, the traffic ceased. It was then I knew it was a DDOS attack since the IOS DNS server was open to the outside. After that, everything was fine and there was not a single issue.

Now the next challenge for me is to write ACLs to block incoming DNS requests from the outside since I like using the IOS DNS server for DNS caching.