Amine Gouiri scored the winner eight minutes from time when he poked home a loose ball, triggering jubilation on the pitch and in the stands, and condemning Jordan to an early World Cup exit.
"We had the mental strength to come back, and I think we're continuing to improve. This victory, earned with character and determination, will do us the world of good," Gouiri said.
"It could be a turning point. A win is always good, especially for confidence given the way the match unfolded. We're very happy, but we know nothing is done yet. We've taken an important step."