Early tries from Dutchman Tim Visser for the visitors and Rhys Webb for the home side set the game up nicely, but that was the end of the try scoring as the kickers took charge.
Last season's top scorer, Visser, was first to cross after a bouncing ball eluded the Ospreys full-back Richard Fussell and the winger went over.
Five minutes later, the home side, without ten players on international duty at the World Cup, levelled through scrum-half Webb. The ball fell kindly to the Wales U20 international who side-stepped well to score his second try of the season.
Ospreys hero Tipuric then spurned a glorious opening ten minutes before the break after good interplay sent him racing towards the line.
The 22-year-old flanker received an untimely push but lost possession as he fell to the ground.
The end of the half developed into a disjointed affair, with both sides guilty of wasting possession and going in all square 13-13.
Kickers Greig Laidlaw and Dan Biggar continued to keep the scoreboard ticking over before Tipuric's dramatic finale.
With the scores level at 19-19 and the home side reduced to 14 men after Ian Evans' yellow card, they managed to force the ball the entire length of the field to snatch a remarkable victory.
Edinburgh, who have only beaten the Welsh region twice in league competition since the advent of regional rugby in 2003, lost 15-38 at home to Cardiff last time out and will be disappointed to only pick up a bonus point which, at least, gets them off the mark.
|Ospreys Score Card|
|Edinburgh Rugby Score Card|