However the tides turned on the stroke of half time as Ospreys fly-half Dan Biggar was deemed to have deliberately slapped the ball down as Ulster pressed with a penalty try the consequence.
Biggar also received a yellow card and from thereon in Ulster took control with Wallace scoring eight minutes after the restart however a late Eli Walker try ensured it went to the wire in a thrilling conclusion.
Ospreys threatened a remarkable comeback in the immediate aftermath but in the end had to settle for a losing bonus point as Ulster breathed a huge sigh of relief.
Ulster started well and controlled the majority of play but they had little to show for it with scrum-half Ruan Pienaar sending a penalty wide after 12 minutes.
And it was to prove costly as the Ospreys ventured into the hosts' half at Ravenhill for the first time shortly after and duly scored the first try of the game.
Kahn Fotuali'i broke quickly from a ruck but saw his progress impeded by Wannenburg - however it didn't go unnoticed and the No.8 was shown a yellow card.
With one less forward on the pitch Ospreys fly-half Biggar kicked into the corner and the tactics paid dividends as Bearman barged his way over after the lineout.
Biggar converted to make it 7-0 but 13 minutes later Ulster finally got themselves on the scoreboard - Pienaar kicking a penalty after Walker was punished for failing to release.
And things soon went from bad to worse for the visitors as Ulster piled on the pressure towards the end of the first half and were eventually rewarded with a penalty try to take the lead.
Referee Neil Hennessey ruled that Biggar had committed a professional foul by deliberately slapping down the ball down with Ulster inches away from the Ospreys line.
The Ospreys fly-half was shown a yellow card and scrum-half Pienaar kicked a simple conversion to give the Irish outfit a 10-7 lead going into half-time.
And Ulster started the second half as they ended the first with fly-half Wallace completing a sweeping move involving John Afoa, Pienaar and Ian Whitten, by scoring in the corner.
Pienaar sent the conversion wide but it took a superb try saving tackle from his opposite number Fotuali'i on centre Whitten in the 53rd minute to keep the score at 15-7.
Ulster looked to have the win in the bag however Walker had other ideas scoring a superb solo try in the 79th minute with Biggar converting but despite some ferocious late pressure they were able to create a turnover that gives a huge fillip to their play-off hopes.
|Ulster Rugby Score Card|
|Ospreys Score Card|