Spot gold fell 0.72 percent to $1,213.54 per ounce, after touching its lowest since Nov. 16 at $1,213.94.
U.S. gold futures dropped 0.77 percent to $1,213 an ounce.
"Most of those comments (from Fed officials) so far have been along the lines of continuing along the path of current rate hikes... certainly some of those comments have supported the dollar and pressured gold," said David Meger, director of metals trading at High Ridge Futures.
The dollar gained after Clarida said the central bank should continue to gradually raise interest rates, but it is "especially important" to closely monitor new economic data as monetary policy is getting close to a neutral stance.
"Clarida softened gold with body blows and the stronger dollar on the back of consumer confidence (data) just delivered a left hook," said Tai Wong, head of metals trading at BMO.
"I think gold will be dependent on the dollar for rest of day, and we could see a bounce if the dollar retraces."
Higher interest rates tend to boost the dollar, making the gold more expensive for holders of other currencies.
In a separate interview, St. Louis Fed President James Bullard said the central bank may face a year of difficult decisions, with rates still low by historic standards but growth ebbing.
Meanwhile, investors are also "jockeying for position" going into the G20 summit, said Michael Matousek, head trader at U.S. Global Investors.
Ahead of a meeting with Chinese President Xi Jinping at the summit in Argentina this week, U.S. President Donald Trump said that he was "highly unlikely" to accept China's request to hold off from raising tariffs due to take effect on Jan. 1.
The dollar has been the preferred safe haven this year as the U.S.-China trade war unfolded, thereby denting the appeal of bullion.
Investors also awaited the minutes from the U.S. Fed's meeting on Nov.7-8 for indications on the future path of rate hikes in 2019, while Fed Chairman Jerome Powell is due to speak on Wednesday.
"The FOMC minutes could be a catalyst and with the G20 coming up, you can see some volatility at the end of the week. If the minutes are dovish then you will see people running to gold," Matousek said.
Among other precious metals, spot silver fell 0.76 percent to $14.12 an ounce, having touched $14.13, its lowest since Nov. 15.
Platinum dropped 0.77 percent at $833.30, after touching its lowest in nearly 2 weeks.
Palladium rose 0.79 percent to $1,150.