• The 10-year Treasury yield topped 1.68% in the previous session, following minutes from the Fed's April meeting.
• New claims for unemployment benefits are expected to total 452,000 for the week ended May 15, slightly lower than 473,000 in the week prior, according to economists polled by Dow Jones.

U.S. Treasury yields fell on Thursday as investors digested comments from the U.S. Federal Reserve, suggesting it might taper its asset purchases if the economy continues to recover rapidly.

The yield on the benchmark 10-year Treasury note dropped roughly five basis points to 1.634% in afternoon trading. The yield on the 30-year Treasury bond fell to 2.343%. Yields move inversely to prices. (A basis point is equal to 0.01 percentage points).

The 10-year Treasury yield topped 1.68% in the previous session, after minutes from the Fed's April meeting showed the central bank would reconsider its easy monetary policy if the economy continued to rapidly improve.

"A number of participants suggested that if the economy continued to make rapid progress toward the Committee's goals, it might be appropriate at some point in upcoming meetings to begin discussing a plan for adjusting the pace of asset purchases," the meeting summary stated.

On Thursday, Dallas Fed President Robert Kaplan that he would like to see the central bank start to trim its asset purchases sooner rather than later, Reuters reported.

Max Gokhman, the head of asset allocation at Pacific Life Fund Advisors, said the jump and then reversal in yields is a sign that the move was a false start by traders who are trying to make sure they aren't caught on the sidelines whenever the Fed does change its policy.

"I think it was almost not so much worrying about the taper as much as worrying about other people worrying about it," Gokhman said. He added that the minutes were from a meeting that occurred before the weaker-than-expected April jobs report, which could have made central bankers less inclined to change their dovish stance.

Richard Kelly, head of global strategy at TD Securities, told CNBC's "Street Signs Europe" on Thursday that the Fed's comments were not a sign that it was about to start tapering but was instead just a "shot across the bow for markets."

He didn't think tapering would "even start till next year so this is just going to be a long road of talk for now."

Given that fuller employment is one of the Fed's goals, investors will be keeping a close eye on weekly jobless claims data.

The latest data, released on Thursday, showed 444,000 initial claims for the week ended May 15. Economists surveyed by Dow Jones were expecting 452,000 claims.

Auctions were held Thursday for $40 billion of 4-week bills, $40 billion of 8-week bills and $13 billion of 9-year 8-month Treasury inflation-protected securities.

— CNBC's Yun Li contributed to this market report.