(CNN) — House Speaker Kevin McCarthy helped secure a debt limit deal – now he has to secure its passage in the House, with little room for error and a looming threat to his speakership.

McCarthy faces a test on Tuesday as House lawmakers return to Washington and a powerful committee takes up the bill to suspend the debt limit through January 1, 2025. Some of the bill’s loudest conservative critics sit on the panel and pressure is growing on a key Republican swing vote as leadership works to advance the deal to a final floor vote on Wednesday.

One of those critics – GOP Rep Chip Roy – made the strongest threat yet to McCarthy’s gavel during an interview with conservative commentator Glenn Beck, saying if the deal can’t be killed in the Rules Committee or on the floor: “Then we’re going to have to then regroup and figure out the whole leadership arrangement again.”

The comments were confirmed to CNN through his spokesperson.

McCarthy said later on Tuesday he is not concerned about losing his speakership over the debt ceiling deal.

“Everybody can have their opinion, but explain to me what in the bill are they opposed to … I am not sure what people are opposed to in the bill,” he said.

When pressed on if he would keep his speakership, McCarthy said, “You guys ask me all the time. I’m still standing.”

“You know what, keep underestimating me. You’ll go far that way,” he said.

The timeframe to get the bill passed through both chambers of Congress and signed into law is extremely tight. Lawmakers are racing the clock to avert a catastrophic default ahead of June 5, the day the Treasury Department has said it will no longer be able to pay all of the nation’s obligations in full and on time.

In a positive sign for the bill’s prospects in the House, a wide range of members on both sides of the aisle – many of them moderates – appear poised to coalesce behind the deal to avert default. Republicans believe they are pushing toward 150 Republican votes or more, two sources told CNN.

That’s more than a majority of the Republican conference, which McCarthy has been promising for days he could get on the bill.

McCarthy under pressure

Another key concession McCarthy made to win the speakership that looms over the effort to push the debt limit deal: Any member of the House can move to force a vote to topple the speaker known as a motion to vacate.

Republican Rep. Jim Jordan of Ohio said Tuesday that he’s not worried about McCarthy’s speakership. “I think he is doing a good job,” he said.

Rep. Patrick McHenry said Tuesday he was not worried about McCarthy losing his speakership and defended the deal he helped cut.

“With a narrow majority in the House, we have the most conservative outcome we possibly could,” McHenry said.

“I’m proud of the package. I wanted more. I absolutely wanted more, (but) what we have here is better than what was about to come.”

Asked if the speaker’s job is now in jeopardy, McHenry said, “no.”

The powerful House Rules Committee will meet Tuesday afternoon on the debt limit bill. The panel must adopt a rule to set parameters for debate – a hurdle that needs to be cleared before a final House vote can take place.

To win the speakership, McCarthy agreed to name three conservative hardliners – Roy as well as Reps. Ralph Norman of South Carolina and Thomas Massie of Kentucky – to the committee, a major concession since usually the panel is stacked with close allies of the leadership.

Roy accused McCarthy on Monday of cutting a deal during the January speaker’s race that could complicate efforts to pass the debt ceiling bill this week. But McCarthy’s allies quickly refuted the Texas Republican, underscoring the tension ahead of the committee meeting – and putting new pressure on Massie, who has yet to take a position on the plan.

Roy argued that McCarthy cut a hand-shake deal in January that all nine Republicans on the powerful panel must agree to move any legislation forward, otherwise bills could not be considered by the full House for majority approval.

“A reminder that during Speaker negotiations to build the coalition, that it was explicit both that nothing would pass Rules Committee without AT LEAST 7 GOP votes – AND that the Committee would not allow reporting out rules without unanimous Republican votes,” Roy tweeted.

Senior GOP sources acknowledged that there was an agreement for seven Republican members to agree to move forward in order to advance a bill to the floor, but they flatly dispute that there was a deal for all nine to sign off for legislation to advance.

“I have not heard that before. If those conversations took place, the rest of the conference was unaware of them,” said Rep. Dusty Johnson of South Dakota. “And frankly, I doubt them.”

The dispute is significant because Roy sits on the committee – which is divided between nine Republicans and four Democrats – as does Norman, and both have emerged as leading critics of the bipartisan debt limit bill.

A third conservative who sits on the panel – Massie – has so far been mum about how he plans to handle the rule vote in committee. If Massie were to join Roy and Norman and vote against the rule at Tuesday’s meeting, he could effectively stall the measure in committee.

On Tuesday ahead of the committee meeting, Massie declined to say how he would vote in Rules or on the debt deal. But he did tell CNN he is “not aware” of an alleged secret agreement that the Rules Committee must pass with support of all nine Republicans on the committee. “If there was, I might have used it already,” Massie quipped.

But in January, Massie told CNN he was reluctant to vote against rules to stop bills in their tracks.

“I would be reluctant to try to use the Rules Committee to achieve a legislative outcome, particularly if it doesn’t represent a large majority of our caucus,” Massie said at the time. “So I don’t ever intend to use my position on there to like, hold somebody hostage – or hold legislation hostage.”

Pressed on Tuesday, if he was confident he would have the Republican votes he needs in rules, McCarthy told CNN, “yes.”

Republicans close to McCarthy refuted the notion that bills could only advance with a majority support.

“I’m a rules guy,” Johnson said. “And when I checked, there wasn’t a rule that something has to come out of Rules Committee unanimously. Now, Chip is a rules guy too. So I think he’s going to understand that, that this is a majoritarian institution, and that ultimately, we’re going to serve Americans the best way that the majority of us know how – that’s going to be to pass this bill.”

Other McCarthy allies agreed.

“I don’t know what Speaker McCarthy agreed to, but that has not been something that any of us were familiar with,” Rep. Stephanie Bice of Oklahoma said. “I think that comment was that it had to be unanimous to come out of the Rules Committee to go to the floor is the tweet that I read. And I think that is inaccurate, at best, but I don’t know because I wasn’t in the room. I don’t know how you would have something like that functionally work.”

This story has been updated with additional developments.