Aaron Ramsey equalised for Arsenal again in the 83rd minute before Giroud's crucial touch just two minutes later completed the quick turnaround at the Emirates.
The Gunners got off to the ideal start, with record-signing Alexandre Lacazette scoring after just 94 seconds, via his his first-ever shot in the Premier League.
Shinji Okazaki levelled for Leicester just minutes after Arsenal's opening goal with a header that did not paint the Gunners defence in a good light.
It may not be the role the Frenchman wants but he reinforced his position as the Premier League's top impact player with his late victor for Arsenal in Friday's 4-3 thriller with Leicester.
Wenger is keen for his team to make a strong start and banish memories of last season, when they lost their opening game 4-3 to Liverpool at the Emirates Stadium.
With this match, the English top-flight season will begin on a Friday for the first time in its 129-year history.
But perhaps the answer that baffled Wenger the most was when the German defender said his favourite pet animal was a guinea pig.
Alexandre Lacazette is the other that we all want to see out there, although having both him and Olivier Giroud still feels like the optimal way to go, especially with Alexis likely missing from the starting XI.
"He adds great value to the squad and I think as well he is a great lover of the club".
The Premier League is borrowing an idea long practiced in America, as they'll kick off their season with a single night game before the rest of the league kicks off. Following a short corner on the left, Marc Albrighton's deep cross was headed back across goal by Harry Maguire and Okazaki leapt to nod a header past Petr Cech. Alexis will not be available for example, so I can not go into the game with too many uncertainties.
Jamie Vardy inflicted further misery on the Gunners just before the half-hour mark, as he converted another well-executed Albrighton delivery.