He thanked Germany for its support during Russia's invasion.
But it was uncomfortable listening for many MPs as he criticised German energy policy and business interests for contributing to that wall of division.
Over the past week, Mr Zelensky has given a master class in how to pitch his message to allies.
On Wednesday, he addressed US Congress, referring to Pearl Harbor and the 9/11 terror attacks. On Tuesday he asked Canada's parliament to envisage bombs and missiles falling on Toronto, Vancouver and Edmonton.
Last week, his address to the House of Commons in London was partly a homage to Winston Churchill's wartime speeches.
His emotive speech to the German parliament, the Bundestag, was equally tailored to appeal to his audience with well-chosen historical parallels. Some of his audience grew up in communist East Germany behind the Berlin Wall.
During the Cold War, Germany's dogged negotiation and engagement with Moscow contributed to the fall of the Berlin Wall and German reunification.
Since then Germany has tried to use trade, business and energy links to integrate a peaceful Russia into the West.
But Russian President Vladimir Putin's invasion of Ukraine three weeks ago has destroyed many of those aspirations. The general political consensus in Germany now is that this may have been the wrong approach with Mr Putin's increasingly authoritarian and kleptocratic government.
President Zelensky expressed outrage that some German businesses were still operating in Russia. The controversial, now cancelled, Russian-German gas pipeline Nord Stream 2 was "cement for that new wall" dividing Europe, he said. Germany's continued reluctance to allow Ukraine into the European Union was "another brick" in that new wall.
"Dear Mr Scholz, tear down this wall," he demanded of the German chancellor, repeating US President Ronald Reagan's historic demand to the Soviets in 1987 as he stood beside the Brandenburg Gate in Berlin.
Many German ministers listening will agree with him. The Green Party, which in the new governing coalition is now responsible for energy policy within the powerful Economy Ministry, has for years been campaigning against Nord Stream 2.
Even those who once supported the pipeline have changed their minds. Until recently Mr Scholz still referred to it as an non-political business matter.
But as Russian troops gathered on Ukraine's borders, shortly before the invasion, he unexpectedly announced the newly finished pipeline would not be opening after all.
Using another German Cold War analogy, in his speech President Zelensky said that Ukraine needed an airlift, and a no-fly zone to protect it, just as Cold War West Berlin had been saved by the Allied airlift in 1948-49.
He also reminded Germany of its responsibility to make up for Nazi crimes in Ukraine, saying Ukrainian towns and cities were being destroyed for the second time in 80 years.
This was a well-judged comment - Germany does guilt well. But for now there is no time for national bouts of repentance and soul searching.
President Zelensky's speech was greeted by long applause and standing ovations.
The main opposition party, the conservative CDU, expressed outrage that after the address the chamber went on to discuss normal parliamentary business, instead of debating the war in Ukraine. The transition certainly jarred as the parliamentary president announced, amid booing from opposition MPs, a scheduled birthday congratulation message.
Pressure is growing from Ukraine, and from German voters, to end German reliance on Russian energy imports.
The German government is planning to stop imports of Russian oil and coal this year, with an end to Russian gas as soon as other sources are found. Berlin argues that a sudden ban would harm Germany more than Russia, sparking mass unemployment here.
German Economy Minister Robert Habeck is now on a whirlwind global tour to drum up energy from elsewhere. After a visit to Norway yesterday, he announced a deal to boost imports of Norwegian gas and this week he'll fly to Qatar.
This may not satisfy German voters. In polls a majority say they want an instant embargo on Russian energy. But then it may be impossible to fulfil the electorate's demands - in other surveys an overwhelming majority of German voters also say that energy costs are already too high.