The Bluebirds will take on Premier League opposition in the shape of Portsmouth after edging a tight encounter with their fellow Championship side courtesy of Cardiff-born Ledley's early effort.
The 21-year-old Wales international hooked a superb volley with his left foot over Luke Steele from 15 yards after Tony Capaldi's long throw was only partially cleared.
Barnsley bossed possession for long periods of the match but lacked the necessary quality in the final third to cause the Bluebirds too many problems on a comfortable afternoon for their solid-looking defence.
Tykes defender Steve Foster made a surprise recovery from a foot injury to start, so manager Simon Davey named the same starting eleven that defeated Chelsea at the quarter-final stage.
The South Yorkshire side had also beaten Liverpool 2-1 at Anfield in the fifth round during a dream FA Cup run.
However, they were immediately brought back down to earth in the ninth minute when Ledley dispatched his unstoppable volley.
Soon after, Steele knew little about a 22nd minute save when he blocked Trevor Sinclair's point-blank effort with his body after gifting the veteran midfielder the chance by spilling a weak shot from Jimmy Floyd Hasselbaink.
At the other end, Istvan Ferenczi saw a low shot cleared off the goalline by Sinclair following a dangerous Brian Howard corner.
Gavin Rae should have doubled the advantage at the beginning of the second half, but his bullet header from 12 yards after arriving late onto Ledley's cross was straight at the grateful Steele.
But the miss of the match came on 67 minutes when Tykes striker Kayode Odejayi was played through one-on-one with Peter Enckelman.
The powerful striker looked to pick his spot after latching onto a perfectly weighted through ball, but side-footed inches wide from ten yards before falling to his knees with his hands on his head.
That was to be as close as Barnsley would come to finding an equaliser and the Bluebirds played out the remaining time with little trouble.
In fact, Dave Jones' side should have killed off the tie as Peter Whittingham saw a 20-yard shot ruffle the roof of the net, before Steele charged down Rae's close-range effort.
However despite wasting those late chances, Cardiff ensured themselves of a first FA Cup Final appearance since 1927 as their defence held firm despite late Barnsley pressure.