Supporters in Tokyo screamed in despair when Japan's Yuichi Komano hit the crossbar with his penalty kick after extra time failed to produce a result.
Asia's final hope of reaching the World Cup quarter-finals was dashed as Paraguay's Oscar Cardozo scored the decisive spot-kick in Pretoria.
Some female fans wept as others crouched in both disappointment and fatigue at a Tokyo cafe run by former star striker Hidetoshi Nakata.
"I cannot take this," Akira Hayashi said at the cafe, where groans of dismay echoed as television footage showed the Paraguay team celebrate.
"But they really did their best," said the 22-year-old. "They really showed the Japanese spirit."
Taiki Nakazawa, 21, said: "They fought to the very last minute. Just an inch short in the penalty kick. Paraguay was only slightly stronger than us."
"Of course, it would have been better if they had won," Yamato Suekuni, 19, said. "But they really were impressive."
The streets of Shibuya, the buzzing centre of Japanese youth culture in Tokyo, were flooded by dejected fans, despite the late hour after a game that kicked off at 11pm local time.
At Saitama Football Stadium, north of Tokyo, some 5,000 fans, mostly wearing blue jerseys to match the team colours, gathered to see live television coverage of the match, despite heavy rain.
Bleached-blond goal ace Keisuke Honda dominated the front pages of Japanese sports newspapers on Tuesday, with the striker looking to continue his form to carry the team into the quarter-finals.
The striker, who plies his trade at CSKA Moscow, had become a national hero after scoring two goals and setting up another as underdogs Japan upset Cameroon and Denmark and went down fighting 1-0 to the Dutch in the group stage.
Japan had never won a World Cup game on foreign soil prior to this year's finals, with their only previous second-round appearance in 2002 when they co-hosted the tournament with South Korea.
Me: Japan did their very good job. Bravo Japan! Go!! You always my winner. Asia Legend!