BOSTON (CBS) — At the quarter point of the season, the Patriots defense has looked pretty awful.

The D is allowing an NFL-worst 456.8 yards per game and rank 31st in allowing 32 points per contest. New cornerback Stephon Gilmore has become a bit of a whipping boy after his first month in a Patriots uniform, and he deserves his share of the blame for the defense’s struggles, but former Patriots safety Brandon Meriweather said he’s not the only one.

READ MORE: Vince Wilfork voted into Patriots Hall Of Fame

Meriweather, joining 98.5 The Sports Hub’s Toucher & Rich for his weekly appearance on Wednesday, said the defense as a whole is making too many mistakes.

“It ain’t all just one person. If it was one person it would be fixed by now,” said Meriweather. “It’s everyone making mistakes. It’s more than just Gilmore. It’s a lot more than that.”

Meriweather admits that the New England defensive system is a lot harder to pick up than other defenses around the league, but for many, it’s just a matter of spending extra time in the playbook.

“It’s more than a defense, it’s the terminology. A lot of people call things completely different things than what the Patriots call it. It depends on how much you’re in your playbook and how much you put into it. Some people have trouble with the different terminology,” he said. “The Patriots have different things you do, little movements, and a lot of people don’t have that.”

Meriweather doesn’t think the defense has been “dumbed down” to start the season and he wouldn’t chalk up the struggles to poor communication, either, because Devin McCourty and Patrick Chung are still the leaders in the secondary. He said he’d apply more pressure to the quarterback while the secondary figures things out.

“Honestly, I’d probably blitz more. If you’re going to beat us, we’re putting our best on your best and you’ve got three seconds. You’re not going to sit back there and let it go,” said Meriweather. “Bill is a great coach, Matt is a great coach and they’ll fix it.”

READ MORE: LOOK: Killer whale spotted by fisherman off Nantucket

Though he doesn’t think the Patriots should go out and sign free agent corner Darrelle Revis, Meriweather said they should be open to asking Rob Ninkovich to come out of retirement.

“Rob is one of those people who is a born leader, a guy who plays hard no matter what and he always ends up in the right spot. Bringing him back would at least boost the morale of some of the people around him,” he said.

Meriweather sparked some controversy in last week’s interview, saying that Larry Izzo was his “grossest” teammate because he used to bring food with him into the bathroom. Izzo, now the special teams coach in Houston, reached out to Meriweather and asked him to kill the story, so that’s what Meriweather did on Wednesday. Sort of.

“He told me I have to fix this because he’s getting killed down [in Houston],” said Meriweather. “He told me to kill it because he’s a special teams coach and he’s getting killed down there. I’m going to do a friend a favor and [squash] the story. I’ll be a good friend now.”

“I don’t know why he cared. It wouldn’t matter to me,” he said. “But everyone is different.”

Toucher & Rich also hit Meriweather with his weekly serving of 5 Questions. Listen to the full podcast above!

MORE NEWS: 2 Burlington driveway workers seriously hurt when dump truck contacts live wires