The Italians are still searching for their first ever-win in the competition despite a brave performance after taking a 7-0 lead through Samuele Pace's try midway through the first half.
Prop Nathan Trevett scored his first try for Blues to drag them back into the game as the Welsh side moved up to seventh in the league with the win, leaving Zebre bottom with four points to their name.
And Sweeney was the main difference as he kicked three all-important penalties as Cardiff turned the screw and kept on the pressure in the second half.
The match started evenly but Zebre were given an opening 15 minutes in when Robin Copeland was sent to the sin bin.
The Italians took full advantage of the extra man as just three minutes later centre Pace, who was so impressive in their narrow loss against Ulster, ran through to score his second try of the season.
Fly-half Daniel Halangahu held his nerve to dispatch the conversion asCardiff inevitably attempted to force their way back into the game.
Cardiff completely restructured their front row for the trip to Italy and the change appeared to pay off as prop Trevett bundled over on 27 minutes although Sweeney failed to take the extra two points.
The rest of the half saw plenty of back and forth between the two teams as Zebre held their own against their more heralded opponents and they missed a great opportunity to open the gap as Gonzalo Garcia saw his penalty slide wide with the last kick of the half.
There was cheer for Blues fans as their reigning player of the year Mike Paterson came on for Macauley Cook five minutes after the restart for his first appearance of the season after returning from operations on both his shoulders.
And there was more to be happy about as a Sweeney penalty put them in front in the 56th minute although he missed another just five minutes later.
But the 32-year-old kicked two more in two minutes with the first in the 68th as the Blues opened up a seven point lead which a spirited Zebre side were unable to reverse.
|Zebre Score Card|
|Cardiff Blues Score Card|