A try in each half from Enrico Ceccato and Alberto di Bernardo put Treviso in pole position for the win but those efforts were matched by flanker Chris Henry and centre Paddy Wallace's tries in each half.
Ulster had only won once on the road in the RaboDirect PRO12 this season since September and that poor run looked to continue when hooker Ceccato snuck over for the hosts.
Ulster had barely hit a ruck in anger before Burton and full-back Brendan Williams combined to create space for the hooker and Burton converted for a flying 7-0 lead.
Fly-half Ian Humphreys got Ulster's first points after 17 minutes with a penalty from inside his own half.
That was followed up by Henry's try putting the visitors into the lead after good work from Wallace created space for the blindside. Scrum-half Ruan Pienaar converted.
Burton, who slotted a drop goal in Italy's victory over Scotland last Saturday, then repeated the trick to pull Treviso level but their parity was brief and Burton's ill-discipline was to blame.
The fly-half failed to release in the tackle and Welsh referee Leighton Hodges awarded Ulster a penalty, which Pienaar comfortably put over.
Burton made amends when Treviso worked their way into Ulster's 22 and an infringement gave him a straightforward penalty kick.
With the scores tied at 13-13 on the stroke of half-time, Pienaar spurned a penalty attempt and Burton booted Treviso ahead after the break with his second drop goal.
Both sides then rang the changes in an attempt to find a way through.
Ulster's returning duo of lock Johann Muller and centre Darren Cave were hauled off within five minutes of each after in the first game after two months of inactivity.
But Smith's decision to substitute Alberto di Bernardo with Williams in the 65th minute was immediately vindicated as the replacement crossed after just four minutes on the field.
With Burton converting Treviso were looking good for a first victory of the year but Wallace had other ideas and his try and Pienaar's conversion made it 23-22 before Whitten pounced for the winner.
|Benetton Treviso Score Card|
|Alberto di Bernardo||1||5|
|Ulster Rugby Score Card|