It was the dominance of the home pack which proved decisive with Ryan Jones crossing and a powerful scrum earning two penalty tries before Williams' bonus point effort.
The Ospreys completed dominated the opening stages but a great tackle from Toby Faletau on Richard Hibbard stopped what looked like a certain try.
And on ten minutes it was the visitors who took the lead as Lewis Robling slotted a long-range penalty from just inside his own half after the Ospreys had been penalised for holding on in the tackle.
However Dan Biggar replied in kind before Robling slotted his second penalty of the night to restore the visitors' lead.
It would not last long however. Rhys Webb took a quick penalty and having made a break it was Ryan Jones who eventually crashed over for the first try of the night, Biggar's conversion making it 10-6 midway through the half.
The home side could have stretched the lead further when another quick tap from Webb led to Hanno Dirksen getting away but Aled Brew's desperate tackle stopped him just short of the line.
However from the ensuing scrum the Ospreys showed their dominance up front, and after Nathan Buck had been sent to the sin bin, the Dragons scrum folded and Nigel Owens had no hesitation in giving a penalty try which Biggar converted.
The home side went in 17-6 up at the break but the Dragons pulled three points back at the start of the half after a loose pass from the Ospreys led to Ian Evans holding on and Robling added his third kick of the night.
And despite the Ospreys edging closer to a third try they were dealt a blow when Biggar was sent to the sin bin when he prevented Brew from taking a quick penalty.
Just before the half hour the Dragons were back in it when Robling slotted his fourth penalty of the game to make it 17-12.
But with Biggar back the Dragons found themselves under pressure again and they suffered another blow when prop Dan Way was sinbinned as the Ospreys pack was in total control.
With just a minute remaining another set of scrums left with Owens with no option but to award a second penalty try which Biggar converted.
That left the Ospreys one try away from a bonus point and Williams duly obliged slipping through the Dragons defence for a trademark swallow dive finish before slotting the conversion for good measure.
|Ospreys Score Card|
|Newport Gwent Dragons Score Card|