Tito and Rhys Thomas linked well in midfield as Cardiff sensed a score, but strong tackling from Simon Danielli and Cave saw Ulster keep the hosts out with Martyn Williams held up short of the try-line.
The try did not come as Young's charges kept pressing close to half-time, and they had to settle for two more penalties from Blair to take a 9-3 advantage into the break.
Cardiff continued to take the game to Ulster on the restart but the visitors thwarted a number of attacks and produced some key turnovers, before O'Connor fired over a morale-boosting, monster penalty for his side, with half-an-hour remaining.
Leigh Halfpenny and Blair failed to convert penalty efforts, either side of Caldwell's sin-binning for hands in the ruck.
The Blues pack got on the front foot again and worked a penalty for their Kiwi full-back, but Blair's shot clipped the post and the scoreline remained delicately poised at 9-6.
The game's defining moments came at the start of the final quarter as Cave saw yellow for a trip on Halfpenny in midfield, putting Ulster down to 13 men and under a serious amount of pressure.
Cardiff immediately took advantage, kicking the penalty to touch and a well-executed lineout assault saw Molitika muscle over for the all-important try, which Blair converted.
Molitika then went close to adding a second try as the Blues went for the kill.
And while a collapsed scrum allowed Ulster, with Cave still in the sin-bin, close the gap with O'Connor's third penalty, Blair cancelled that out with the last of his haul.
The sides will meet again next Saturday at Ravenhill, where Cardiff will be hoping to record their seventh straight win over the Ulstermen.
|Cardiff Blues Score Card|
|Ulster Rugby Score Card|