His opposite number Ian Humphreys missed two chances to push Ulster clear again - he was off target with a 62nd minute penalty attempt and watched his last-gasp drop goal shot drift the wrong side of the posts.
Ulster welcomed back hooker Rory Best for his first appearance of the season, following his quicker-than-expected recovery from a serious knee injury.
His Ireland colleagues Andrew Trimble and Paddy Wallace also returned from Six Nations duty.
Wales lock Luke Charteris was back to skipper the Dragons, while the visitors paired Tom Riley and Rhodri Gomer Davies together in midfield.
Humphreys, who opened the scoring in the second minute, and Tovey shared out four penalty goals in the opening quarter before Ulster went up a notch with a 24th minute try from Wallace.
After a Dragons attack broke down, the home side countered through Humphreys and Darren Cave with the latter galloping forward before linking with supporting lock Dan Tuohy who fed Wallace and he was able to edge past Pat Palmer's tackle to score in the corner.
Humphreys missed the conversion attempt but tagged on a penalty, four minutes later, and matters improved further for Brian McLaughlin's men when a free-flowing move came to an end and Gomer Davies was dispatched to the sin-bin for a professional foul at a ruck.
Humphreys kicked the resulting penalty for a 17-6 lead but the 14-man Dragons dug their heels in before half-time and calmly played their way back into the game, with Tovey landing his third penalty.
Ulster then lost the services of the fit-again Best, who was yellow carded for his involvement in an off-the-ball incident, and Tovey made no mistake with his fourth penalty attempt - closing the gap to 17-12 - as Humphreys closed out the first half with a long range miss at the other end.
The Dragons got back on terms, just five minutes into the second half, when hooker Steve Jones stretched over to score his side's only try following a well-constructed lineout move.
Tovey added the extras to move the Welsh region ahead for the first time at 19-17, but with the sides levelled numerically by Best's return, Ulster were soon back on the attack in search of scores.
They were finding some joy through centres Wallace and Cave and one such move ended with full-back Jamie Smith flicking on for Scottish winger Simon Danielli to touch down in the corner for his fifth try of the league campaign.
Humphreys failed to add the conversion and Ulster's lead was a short-lived one as, amid a raft of replacements, Tovey coolly dropped a 54th minute goal to square up matters once again.
That proved to be the final scoring act of a well-contested tussle, a game of cat and mouse ensued. Ulster had the better of the closing quarter but could not turn their pressure into points and the draw extended their unbeaten run in 2010 to four matches.
But the Dragons will argue that they are the ones building form, following on from their two recent LV= Cup wins over the Ospreys and Saracens.
Dragons boss Paul Turner was clearly the happier coach afterwards, saying: "That's two points won, this is always a difficult place to come.
"We've had a good week, there was huge commitment from the players. They showed a lot of composure, that's great from us and we'll build off this.
"Luke Charteris and Dan Lydiate furthered their cause. Luke is a good man, he just gets on with it. That was another good performance from Jason Tovey too."
|Ulster Rugby Score Card|
|Newport Gwent Dragons Score Card|