Johnson was born in Kammin in Pomerania. His father was a Swedish-descent peasant from Mecklenburg and his mother was from Pommern. At the end of World War II in 1945, he fled with his family to Anklam; his father died in a Soviet internment camp. The family eventually settled in Güstrow, where he attended John-Brinckman-Oberschule 1948–1952. He went on to study German philology, first in Rostock (1952–54), then in Leipzig (1954–56). His Diplomarbeit was on Ernst Barlach. Due to his lack of political support for the Communist regime of East Germany, he was suspended from the University on 17 June 1953 but was later reinstated.
Beginning in 1953, Johnson worked on the novel Ingrid Babendererde, rejected by various publishing houses and unpublished during his lifetime.
In 1956, Johnson's mother left for West Berlin. As a result, he was not allowed to work a normal job in the East. Unemployed for political reasons, he translated Herman Melville's Israel Potter: His Fifty Years of Exile (the translation was published in 1961) and began to write the novel Mutmassungen über Jakob, published in 1959 by Suhrkamp in Frankfurt am Main. Johnson himself moved to West Berlin at this time. He promptly became associated with Gruppe 47, which Hans Magnus Enzensberger once described as "the Central Café of a literature without a capital."
During the early 1960s, Johnson continued to write and publish fiction, and also supported himself as a translator, mainly from English-language works, and as an editor. He travelled to America in 1961; the following year he was married, had a daughter, received a scholarship to Villa Massimo, Rome, and won the Prix International.
1964 - for the Berliner Tagesspiegel, Reviews of GDR television programmes boycotted by the West German press (published under the title "Der 5. Kanal", "The Fifth Channel", 1987).
In 1965, Johnson travelled again to America. He then edited Bertolt Brecht's Me-ti. Buch der Wendungen. Fragmente 1933-1956 (Me-ti: the Book of Changes. Fragments, 1933-1956). From 1966 through 1968 he worked in New York City as a textbook editor at Harcourt, Brace & World and lived with his family in an apartment at 243 Riverside Drive (Manhattan). During this time (in 1967) he began work on his magnum opus, the Jahrestage and edited Das neue Fenster (The new window), a textbook of German-language readings for English-speaking students learning German.
On 1 January 1967 protesters from Johnson's own West Berlin apartment building founded Kommune 1. He first learned about it by reading it in the newspaper. Returning to West Berlin in 1969, he became a member of the West German PEN Center and of the Akademie der Künste (Academy of the Arts). In 1970, he published the first volume of his Jahrestage (Anniversaries). Two more volumes were to follow in the next three years, but the fourth volume would not appear until 1983.
Meanwhile, in 1972 Johnson became Vice President of the Academy of the Arts and was the editor of Max Frisch's Tagebuch 1966-1971. In 1974, he moved to Sheerness on the English Isle of Sheppey; shortly after, he broke off work on Jahrestage due partly to health problems and partly to writer's block.
This was not a completely unproductive period. Johnson published some shorter works and continued to do some work as an editor. In 1977, he was admitted to the Darmstädter Akademie für Sprache und Dichtung (Darmstadt Academy for Speech and Writing); two years later he informally withdrew. In 1979 he gave a series of lectures on poetics at the University of Frankfurt (published posthumously as Begleitumstände. Frankfurter Vorlesungen).
In 1983, the fourth volume of Jahrestage was published, but Johnson broke off a reading tour for health reasons. He died on 22 February 1984 in Sheerness in England.