Treasury yields rose alongside the dollar, while stocks fell as Jerome Powell slammed Wall Street's dovish bets.

Only eight days after his speech fueled speculation that the Federal Reserve was done raising interest rates, Powell said officials will not hesitate to tighten again if necessary. While this is essentially what several Fed speakers have said, it was the part that drew investors' attention on Thursday, especially given the recent rally in both stocks and bonds.

The S&P 500 halted what would have been its longest gain since 2004, while two-year interest rates surpassed 5%. A poor performance of 30-year notes weighed on sentiment, raising concerns about the market's ability to absorb new debt. Powell's remarks also increased traders' expectations of another Fed hike, while decreasing expectations of a rate cut before July.

Meanwhile, the amount of money that investors are parking at a major Fed facility dropped below $1 trillion for the first time in more than two years. Demand for the facility has been fading this year, especially as the Treasury ramped up fresh bill issuance, offering an alternative for short-term investors. That buying has accelerated as traders bet that the Fed is near the end of its interest-rate hiking cycle.