Romelu Lukaku gave the visitors a shock second-half lead when Yannick Bolasie flicked on a long ball from deep into the Belgian’s path and he stormed forward from halfway, outpaced Gael Clichy and slammed a firm low shot past Claudio Bravo.
After De Bruyne saw his penalty saved just before half-time, Aguero experienced the same feeling after 70 minutes.
The Argentina striker missed two spot-kicks in one game against Steaua Bucharest earlier in the season and another for his country in midweek.
Stekelenburg was at it again moments later, clawing out another Aguero effort, but he was finally beaten when Nolito, on for Sane, headed home a Silva cross with his first touch after 72 minutes.
Bob Bradley’s first game as Swansea manager ended in defeat as his side were narrowly beaten by 10-man Arsenal in an entertaining clash at the Emirates Stadium.
The Swans had won there on their previous two visits and had not lost away to Arsenal in over five years but, despite threatening a comeback, they succumbed to a 3-2 loss that sent the Gunners level with Manchester City at the top of the table.
Theo Walcott was all-but gifted a brace to put Arsenal in charge but Granit Xhaka, who would later be sent off by referee Jon Moss, handed possession to Gylfi Sigurdsson to bring Bradley’s side back into the game.
Mesut Ozil restored Arsenal’s two-goal cushion with a well-taken volley but a first Swansea goal for record signing Borja Baston kept things nervy for the Gunners, who played with 10 men for the final 20 minutes after Xhaka was shown a straight red card for bringing down the impressive Modou Barrow.
Tottenham are just a point behind in third place but missed the chance to go top of the table after needing Dele Alli to rescue a late point at West Brom.
The England midfielder struck an 89th-minute equaliser to clinch a 1-1 draw after Nacer Chadli looked to have condemned his former club to defeat just seven minutes previously.
Earlier, Diego Costa scored his seventh Premier League goal of the season as Chelsea claimed a 3-0 win over champions Leicester at Stamford Bridge.
Eden Hazard doubled the lead before half-time and David Luiz hit the crossbar with a free-kick as Chelsea dominated, with former Foxes midfielder N’Golo Kante excelling, before Victor Moses struck a late third.
Bournemouth were the big winners of the day as they thumped Hull 6-1.
Junior Stanislas fired a brace to add to goals from Steve Cook, on his 200th appearance for the club, Charlie Daniels, Callum Wilson and Dan Gosling, as Eddie Howe’s men romped to their record Premier League victory. Ryan Mason’s 34th-minute strike briefly had Hull level.
Joe Allen’s scoring streak continued as his double in a 2-0 win gave Stoke a first Premier League victory of the season over winless Sunderland.