Germany dominated a frenetic match for large spells but failed to capitalise on their pressure – before Japan caused the Khalifa International Stadium to erupt into a mix of celebration and surprise.
Substitute Takuma Asano squeezed in a thumping strike from a tight angle to make it 2-1 to Japan and ran over to the photographers in the corner of the stadium in sheer delight.
There was a collective pause in the arena as the ball rolled down from the roof of the net before supporters realised it had gone in.
Manchester City’s Ilkay Gundogan had given Germany the lead from the penalty spot in the first half, before Japan substitute Ritsu Doan equalised in the 75th minute.
The equaliser came after a string of stunning saves at each end – Japan’s Shuichi Gonda pulling off a double stop to deny Serge Gnabry and Jonas Hofmann, before Germany’s Manuel Neuer got his fingertips to Junya Ito’s effort.
Germany, who finished bottom of their group in Russia in 2018, had missed several chances to extend their lead when the score was 1-0.
They face stiff opposition in Group E with 2010 winners Spain to come on Sunday, then a final group game against Costa Rica.
Before kick-off, Germany’s players put their hands over their mouths during a team photo and in the stands a German government minister, Nancy Faeser, wore the OneLove armband.
It comes after news the German football federation (DFB) is investigating whether it is legal for Fifa to threaten players with sanction for wearing the OneLove armband in Qatar.
The DFB later tweeted that human rights are “non-negotiable” and “we stand by our position”.
Flashbacks for Germany as Japan celebrate memorable win
The misery of Germany’s early exit from the World Cup in Russia could not be forgotten as Hansi Flick’s side trudged off the field at full-time having suffered defeat at the hands of Asian opposition again.
Their supporters had arrived in hope more than expectation that their young stars would lead them into a new era.
Japan were relentless in energy and refused to give in, despite Germany controlling large spells.
There was a ripple of anxiety in the final moments from the Japan supporters at the opposite end to Gonda’s goal as Germany’s Leon Goretzka struck wide and goalkeeper Neuer came up for consecutive corners to try to salvage a dramatic equaliser.
However, Japan held on and, as the final whistle went, the players in the dugout raced on to the pitch to celebrate and flags were waved gleefully in the stands.
They had caused problems for Germany in the first half as Celtic forward Daizen Maeda had a goal disallowed for offside, before Germany’s Kai Havertz also one chalked off for the same reason.
Gundogan and Antonio Rudiger both missed the target with decent chances in the first half, while midfielder Joshua Kimmich was denied twice by Gonda.
Gundogan coolly slotted his penalty down the middle after David Raum had been pushed down by Gonda to give Germany a deserved lead, but they were made to pay for their missed chances.
Bayern Munich teenager Jamal Musiala beat four players in the box only to blast an effort over the bar early in the second half, before Goretzka shot wide in the final moments.
Squad number21Player nameGündogan
Squad number1Player nameNeuer
Squad number14Player nameMusiala
Squad number2Player nameRüdiger
Squad number13Player nameMüller
Squad number10Player nameGnabry
Squad number7Player nameHavertz
Squad number15Player nameSüle
Squad number6Player nameKimmich
Squad number3Player nameRaum
Squad number23Player nameSchlotterbeck
Squad number8Player nameGoretzka
Squad number26Player nameMoukoko
Squad number11Player nameGötze
Squad number18Player nameHofmann
Squad number9Player nameFüllkrug
Squad number9Player nameMitoma
Squad number8Player nameDoan
Squad number16Player nameTomiyasu
Squad number18Player nameAsano
Squad number25Player nameMaeda
Squad number10Player nameMinamino
Squad number6Player nameEndo
Squad number12Player nameGonda
Squad number14Player nameIto
Squad number4Player nameItakura
Squad number22Player nameYoshida
Squad number15Player nameKamada
Squad number17Player nameTanaka
Squad number5Player nameNagatomo
Squad number19Player nameH Sakai
Squad number11Player nameKubo
1Neuer15Süle2Rüdiger23Schlotterbeck3Raum6Kimmich21GündoganSubstituted forGoretzkaat 67’minutes10GnabrySubstituted forMoukokoat 90’minutes13MüllerSubstituted forHofmannat 67’minutes14MusialaSubstituted forGötzeat 79’minutes7HavertzSubstituted forFüllkrugat 79’minutes
12Gonda19H SakaiSubstituted forMinaminoat 74’minutes4Itakura22Yoshida5NagatomoSubstituted forMitomaat 57’minutes6Endo17TanakaSubstituted forDoanat 71’minutes14Ito15Kamada11KuboSubstituted forTomiyasuat 45’minutes25MaedaSubstituted forAsanoat 57’minutes
Match ends, Germany 1, Japan 2.
Second Half ends, Germany 1, Japan 2.
Corner, Germany. Conceded by Wataru Endo.
Attempt blocked. Niklas Süle (Germany) right footed shot from the left side of the six yard box is blocked.
Antonio Rüdiger (Germany) wins a free kick on the right wing.
Foul by Takumi Minamino (Japan).
Attempt missed. Leon Goretzka (Germany) right footed shot from outside the box is close, but misses to the left.
Mario Götze (Germany) wins a free kick in the defensive half.
Foul by Takuma Asano (Japan).
Attempt missed. Antonio Rüdiger (Germany) right footed shot from outside the box misses to the left. Assisted by Nico Schlotterbeck.
Attempt missed. Niclas Füllkrug (Germany) header from the centre of the box is close, but misses to the right. Assisted by Mario Götze with a cross.
Substitution, Germany. Youssoufa Moukoko replaces Serge Gnabry.
David Raum (Germany) wins a free kick in the defensive half.
Foul by Wataru Endo (Japan).
Corner, Germany. Conceded by Maya Yoshida.
David Raum (Germany) wins a free kick in the attacking half.
Foul by Wataru Endo (Japan).
Goal! Germany 1, Japan 2. Takuma Asano (Japan) right footed shot from the right side of the six yard box to the top right corner. Assisted by Ko Itakura.
Foul by Nico Schlotterbeck (Germany).
Wataru Endo (Japan) wins a free kick in the defensive half.
Get your daily dose of Fifa World Cup reaction, debate & analysis with World Cup Daily on BBC Sounds