vCMTS unloaded latency and jitter

plat2on1 · 2025-06-08T00:29:09+00:00

Finally got the “next gen” upgrades here after 2 days of downtime, and it’s absolutely terrible. We get years of LLD “trials” but they just roll out this garbage like it’s nothing? Left side unloaded, Right side upload loaded with 30 Mbps with iperf.

Devious

In Seattle too but no issues at all with gaming on Comcast connection.
Your modem can affect bufferbloat or latency as well.
I average 8 to 9 ms ping on Comcast.

cypherstream

Heres same system in PA, legacy Scientific Atlanta sub-split plant:

Heres the vCMTS mid-split trace. All our vCMTS sites hop past our comcast buisness gateway are private or cg-nat IP’s.

(Blanked out some of the first hop IP past our router since we pay for statics).

Our biggest issue is we straddle two Comcast markets. The one above is generally 26-36ms pings to major sites, sometimes approaching 40ms. The other Comcast site is half that, 12-22ish pings to majors. Its a huge difference and with all the Comcast overbuilding you could take a short bicycle ride and be in “good ping” territory or “bad ping” territory. Bad ping routes 269 miles out of the way to the west towards Pittsburgh while “good ping” territory routes more direct through Philly burbs to Newark then 111 8th Ave NYC or Ashburn, depending where you are headed.

Here’s the good ping area:
Legacy SA Gainmaker

vCMTS mid-split

I brought up a bunch of traceroutes showing how different (and worse) the routing is from adjacent markets to Jason Livingood and he did take them up to engineering. He did claim its going to take a long time but theres a “network 2.0” project janus in the background that may be able to fix it with establishing new redundant and shorter path routes between markets thanks to all of the physical overbuilding thats been happening in the last few years.

Its going to take a very long time but the supposedly the traces were relayed up the chain.

plat2on1

cypherstream The CRAN problem is a whole other issue that also affects me. New York to Boston and then back to New York for everything, a roughly 400-mile detour. I have no hope that it will ever be rectified.

Still can’t believe how awful this is. Is Comcast still using Harmonic vCMTS exclusively? Interested to see if another vendor performs differently.

fmd

plat2on1 I’m not sure you have given us the full picture. If we take your in-game jitter measurement. How did it look before?

plat2on1

fmd Jitter on the old cBR-8 was comparable to this vCMTS with the synthetic load applied.

cypherstream

plat2on1 Comcast accounts for about half of Harmonic’s total revenue. They have up to 7.8 million shares of Harmonic stock at a fixed price due to a warrant agreement in 2016. They don’t own Harmonic but they are huge investors with a big stake in what they do. I don’t think Comcast will want to deviate from them.

I guess we could ask some people on Spectrum or Cox if they have vcmts and what their trace looked like before and after. Spectrum was using some Vecima gear.

For the cran issue, at least here I could phisically walk between both markets. All they need is some fiber to connect them up and some routing decisions made on both sides. Honestly a shortcut path while maintaining the western PA path adds redundancy, load balancing, resiliency.

There was a fire early in the year from electric wires that melted a fiber optic cable in the city. It took out over 135 nodes. An entire zip code just gone from Comcast’s network from around 7:30 PM to about 3 PM the next day. Time for a redundant path.

PalmDude

Hi all, this is my first post, so if I do anything wrong, I apologize 🙂

A few months ago, I was migrated to vCMTS, and I immediately noticed my latency increased to the CMTS from around 9ms to around 12ms. I also noticed that the CMTS location changed from the local hub in my town to a few towns away. I ran 50 pings from my VPS to my cable modem a year ago while on the legacy CMTS, and rerunning it on the current vCMTS setup shows that average ping increased by about 2.5ms (I realize this test isn’t a very accurate way to compare latency setups due to peering, routing differences, ICMP deprioritization, etc. however, I still found it interesting.) It also shows an increase in jitter from 1.4ms to around 2-4ms (I ran this test a few times, and while the results varied a lot, I can say that watching my ping numbers while playing games, and even in traceroutes and speedtests, it definitely seems to be a bit more variable than I remember the legacy CMTS setup being).

As previously mentioned, I would guess this is in part due to the CMTS being moved further away from the hub site to the primary headend (however please correct me if my assumption is wrong). There is an interesting video on it that I found here:

I’ve seen setups with 2nd hop latency ranging from around 9ms (when the vCMTS is in the same town as the legacy CMTS was), to as high as 16-17ms (where the vCMTS is quite a few towns away).

I wonder if PGS would potentially fix this (I know jlivingood expressed interest in potentially testing this out here: which would be awesome if implemented at some point, or maybe R-MACPHY (although this design probably won’t be implemented due to R-PHY being the preferred choice it seems), as it is interesting to have two modems a few towns away from each other where one could get 9ms to the CMTS, and another could get 16 due to luck of where the primary headend is. Most would never notice this difference though, so I do understand why Comcast chose this path. There is an interesting paper I found here: https://www.nctatechnicalpapers.com/Paper/2023/3588_Hajduczenia_5290_paper
Just found this stuff interesting, thought I would share it 🙂

hex0101

PalmDude Yeah, most customers in a given area are likely not connecting to each other’s modems, so local latency doesn’t seem to get a high priority. Should be a focus though, imagine two customers are served off of the same headend and in the same town and are playing an online video game where one of the PCs/Consoles is chosen as the host, you want the lowest latency possible between the two modems for the best experience. And you would want the other nearby headends to have the lowest latency between them as well to cover those who may be living further away.

fedupcc10

For those that remember me on DSLR, TOLD YA SO! Borderline cloud-enabled F!&$ S#!% RPHY BS has RUINED the connectivity experience for Comcast’s users. The legacy CMTS architecture was telco designed and telco grade. The people building this HARMONIC S#!% are probably hires straight from Bezos’ US EAST team. Any day now, they’re going to move the data and control planes to AWS, and you can say goodbye to your dreams of low latency cable. Your D2.0 connection you used on your venerable surfboard will run circles around your new CLOUD ROUTING, and you will be looking back at a fallen past, before it was ruined by containerization and virtualization of telco applications and set six feet under by moving to the CLOUD.

hex0101

fedupcc10 I do remember you and your post on DSLR from earlier this year!

Have you been upgraded to RPHY yet? Do you still have your 6 or however many additional IP’s on your account? Oh and that’s right, now that I remember, you said that you managed to work out a deal with Comcast and it was a fair compromise! I wonder what it was! Did you move your services over to domain names instead of IP’s so that an IP change on your home internet connection will not cause you to lose access to your stuff?

fmd

fedupcc10 I’m thinking that it is too soon to blame vCMTS as a concept.

cypherstream

fmd I wonder if MACPHY would have a better choice than RPHY?

fmd

cypherstream It can never hurt having the upstream scheduler next to the modem.

I don’t think Comcast has talked about new distances in their papers… How may the longer distance (how much longer?) impact the request-grant, if smaller CMTS sites are being decommissioned.

plat2on1, is your modem bonding the OFDMA channel?

cypherstream

fmd I think in the system that serves me, they transition from the E6000 cmts in the Wyomissing PA to vCMTS racks at the Lebanon PA master headend. I could be wrong but it is a little further away.

In the adjacent Comcast system to my east, they transition from a New Hanover PA hub to a larger facility in Norristown, PA.

So yes they have moved where it terminates. I don’t know if that means the hub site has switches and the vCMTS servers are just at the master site. That’s at least what I would presume. If they just jumped the fiber coming into the hub to feeds towards the master headend, I don’t know if the optics support the additional distance. I think 80km optics would be sufficient but you never know with all the slack along the way.

hex0101

cypherstream Jason probably knows!

PalmDude

cypherstream

I think that is correct. It looks like primary headends have the vCMTS, and the hub sites that once had the legacy CMTS have a DAA switch.

The reasons for these changes I think are explained here and is where I got the screenshot from:

plat2on1

fmd No, my modem hasn’t been blessed by Comcast yet.

CommunityMember

plat2on1 No, my modem hasn’t been blessed by Comcast yet.

Comcast does not do most of the “blessing”. That would be up to your modem manufacturer to submit (with payment and with commitment of their own staff (which means more money)) to the Kyrio testing regime (and if there is new firmware required, that also means they may get to pay for additional costs). Only a few COAM manufacturers have chosen to do so, but it is, ultimately, a business decision for them. If you want your modem to be supported for mid-split, contact your modem manufacturer and request they start the process and see if they agree.

plat2on1

CommunityMember So your saying it’s up to Comcast then.

MacLeech

plat2on1 So your saying it’s up to Comcast then.

No, it’s up to your modem maker to go through the process.
https://kyrio.com/comcast-coam-modem-testing-program/

Comcast set a modem standard.
Kyrio tests modems to that standard.
The modem maker has to meet that standard.

This is the result of the Zoom modem lawsuit against Comcast over a decade ago. Since then, the makers have known they have to go through this process when launching a retail modem.

What make/model do you have?

« Previous Page Next Page »