Northern Ireland international Colgan should have easily collected a low cross by Town's Andy Holdsworth at his near post on 55 minutes.
But he paused fatally and Taylor-Fletcher punished the error by tucking a low shot from close range into the net off Colgan to register his 13th of the season.
A similar hesitation had nearly allowed Town to go in front, but Colgan tipped away Andy Booth's header on 17 after defender Antony Kay stood watching as Jonathan Worthington 's cross went past him.
Huddersfield sub David Graham, who had lost 10lbs in weight after suffering from flu all week, nearly sealed victory late on but Chris Shuker cleared off the line.
Those were however, among the few chances for the hosts, in a game ten-man Barnsley dominated and at times it looked like Town's shaky defence, missing Martin McIntosh, would succumb to the pressure.
Shuker hit the bar on 58 with a long-range rocket. Martin Devaney fluffed the rebound and a third attempt from Marc Richards was cleared off the line by Worthington.
The midfielder also cleared Shuker's effort off the line after Paul Hayes, who tormented Huddersfield's back four all afternoon, put him through.
Hayes deserved a goal for his efforts and was unlucky not to get onto the scoresheet on several occasions.
A glancing header from Stephen McPhail's long throw nearly caught Town keeper Phil Senior out. Then Hayes blasted over after Paul Reid's cross was flicked on by Brian Howard.
But Barnsley's fine display was soured by the crazy dismissal of sub Daniel Nardiello.
He was booked for kicking the ball away on 84 minutes and received a second yellow a minute later for swearing at the referee when the official gave a free-kick against him.