Defender Kenny Milne made a fairytale return to action after 17 months out with a career-threatening knee injury to help Scunthorpe through to round four.
The 30-year-old came on as a substitute at half-time for what was only his third appearance for United since joining them from Falkirk in the summer of 2008.
He helped his side keep a rare clean sheet against the club who were shock semi-finalists two years ago.
And it was former Barnsley striker Paul Hayes who sank his old side with the only goal of the game after 68 minutes when he raced in at the far post to apply the finishing touch to a cross from Jonathan Forte.
The Yorkshire side went into the tie unbeaten in nine games and had the edge in the early stages without creating much in front of goal.
It was not until just before half-time when Emil Hallfredsson had a well-struck shot saved by keeper Josh Lillis.
But Scunthorpe were creating the best chances and Garry Thompson should have done better when a mistake by goalkeeper Luke Steele left him a clear sight of goal.
Forte too was only inches away from getting to a cross from Thompson as it flashed across the face of the goal.
Defences were on top in the second half, with Scunthorpe finally making the breakthrough after a quick break from their own half.
Barnsley threw men forward in search of an equaliser and Daniel Bogdanovic squandered a great opening after forcing his way into the box.
Substitute John Macken also went close but it was Scunthorpe who hung on to reach the fourth round for the first time in six years.