Josh Adams' five-pointer was the only try in a game dominated by the kickers, with Handre Pollard slotting four penalty goals for South Africa and Dan Biggar and Gareth Anscombe getting a penalty goal each, but Anscombe's 78th minute conversion from the touchline was probably the most crucial kick of the match as it put his side ahead shortly before the end.
The halftime score was 3-3 with Biggar kicking the first points in the opening minute, while Pollard responded six minutes later.
Biggar's first three-pointer came after Wales kicked off to start the match and Warrick Gelant jumped to gather, but the Bok pack was deemed to seal off by the referee and the Welsh captain stepped up to take a straight forward penalty goal to put his side 3-0 up.
The Springboks stayed calm though and from their first attack - via a lineout both starting debutants, Kurt-Lee Arendse and Evan Roos, came very close to score after Gelant broke through. Wales were penalised for offside and Pollard slotted the penalty to draw the scores level six minutes in.
The next scoring opportunity came Wales' way. The Boks were pinged near the halfway line for offside at a ruck, but Biggar's effort came up short. Wales lost wingr Alex Cuthbert after 16 minutes and he was replaced by Adams.
A scrum penalty to South Africa, just inside the Welsh half, gave Pollard a chance to extend the lead, but his kick also went wide, although it had good distance on it.
It was clear that the South African scrum was the stronger of the two and would become a launchpad for attacks. From such an attack, with Roos having a first carry of the base, the Boks again came close, with Pollard making a dart to the line. Wales managed to keep the Boks out, but conceded a penalty for going offside.
Pollard kicked to the corner, but the lineout throw was poor and Wales could get away from their red zone. The Boks were putting pressure on Wales and enjoyed territorial advantage, but somehow were not as clinical as they should be in Wales' 22, and it gave the visitors ample opportunity to clear their decks.
The last 10 minutes of the first half was played between the two 10m-lines, with both sides using the box kick to try and gain the upper hand. Bok hooker Joseph Dweba had a strong run down the line, but could not find support in time.
Malcolm Marx started the second half as did Vincent Koch, and the Boks' replacement hooker was involved in an attacking play early on which finished when Pollard was brought down and then stripped of the ball illegally. The Bok captain called for the kicking tee and after 43 minutes of rugby, the Boks were ahead for the first time at 6-3.
That advantage was almost cancelled out immediately though. A Hendrikse box-kick went straight up in the air and Thomas du Toit was deemed offside. Biggar stepped up, but pushed the penalty attempt to the right, much to the delight of the sold out 46 000 crowd.
Deon Fourie came on and won a penalty at the first ruck available for him to compete. Coach Jacques Nienaber went sort of bomb squad early, as Fourie (for Roos), Ntuthuko Mchunu (for Du Toit) and Rynhardt Elstadt (for Marcell Coetzee) all came on before the 50th minute.
Pollard kicked a third penalty goal soon after as Wales were starting to make defensive errors and being penalised for poor discipline on the floor.
The Boks were starting to squeeze and the visitors did not always responded in a positive manner. Referee Angus Gardner pinged Dan Lydiate for trying to slow the ball down but Pollard kick went wide.
Arendse again ran nicely and when Wales were found hands in the ruck again, Pollard kicked to the corner and Alun Wyn Jones was yellow carded for a cynical play. The Boks drove well, but their maul was stopped, again illegally and this time Pollard made no mistake as Ruan Nortje made his Bok debut, replacing Marvin Orie with 20 minutes to go.
The penalty count was beginning to ring the alarm bells for the Welsh as it seemed that the only way they could stop the waves of Bok attacks was by doing so in conflict of the laws.
Anscombe, who replaced Biggar, cut back the lead with a 50m penalty goal 14 minutes from time to give Wales a sniff.
Eben Etzebeth was penalised twice in a row and from the second offence - playing his opponent in the air - Anscombe had another long-range effort to bring Wales right back into the game, but he pushed it wide though.
Grant Williams replaced a cramping Pollard seven minutes from time and soon after, Mchunu was penalised scrum time in a rare Bok mistake in that phase. A lineout drive and a few phases later, Wales sent the ball wide to Adams who went over in the corner, after which Anscombe was kicking to put Wales back in the lead and with a minute left to play.
He made no mistake and although the Boks had a minute to save the test match, from a scrum feed inside the Welsh half, the visitors held out and when Koch dropped a pass and the visitors could leave the field knowing they just made history.
South Africa 12 (3) - Penalty goals: Handre Pollard (4).
Wales 13 (3) - Try: Josh Adams. Conversion: Gareth Anscombe. Penalty goals: Dan Biggar, Anscombe.