Hildesheimer was born of Jewish parents in Hamburg. His grandfather was Azriel Hildesheimer, the modernizer of Orthodox Judaism in Germany. He was educated at Humanistische Gymnasium in Mannheim from 1926 to 1930. He then attended Odenwaldschule until 1933, when he left Germany. He was then educated at Frensham Heights School in Surrey, England. He studied carpentry in Mandatory Palestine, where his parents had emigrated, and underwent psychoanalysis in Jerusalem.
He studied painting and stage building in London. In 1946, he worked as a translator and clerk at the Nuremberg trials. Afterward, he worked as a writer and was a member of Group 47. In 1980, he gave the inaugural address at the Salzburg Festival (Was sagt Musik aus? — What does music say?). In addition to writing, Hildesheimer created collages which he collected in several volumes (the first Endlich allein, 1984), an activity he shared with other late 20th century writers Peter Weiss and Ror Wolf. The municipality of Poschiavo in Switzerland made Hildesheimer an honorary citizen in 1982; he died there in 1991.