Klemperer was born in Breslau, then in Prussia, now Wrocław, Poland. He took United States citizenship in 1937 and Israeli citizenship in 1970.
Klemperer studied music first at the Hoch Conservatory in Frankfurt and later in Berlin under Hans Pfitzner. In 1905 he met Gustav Mahler while conducting the off-stage brass at a performance of Mahler's Symphony No. 2, 'Resurrection'. The two became friends, and Klemperer became conductor at the German Opera in Prague in 1907 on Mahler's recommendation. Mahler wrote a short testimonial, recommending Klemperer, on a small card which Klemperer kept for the rest of his life.
Later, in 1910, Klemperer assisted Mahler in the premiere of his Symphony No. 8, Symphony of a Thousand.
Klemperer went on to hold a number of conducting posts, in Hamburg (1910-1912); in Barmen (1912-1913); the Strasbourg Opera (1914-1917); the Cologne Opera (1917-1924); and the State Opera in Wiesbaden (1924-1927).
From 1927 to 1931, he was conductor at the Kroll Opera in Berlin. In this post he enhanced his reputation as a champion of new music, playing a number of new works, including Leoš Janáček's From the House of the Dead, Arnold Schönberg's Erwartung, Igor Stravinsky's Oedipus Rex, and Paul Hindemith's Cardillac.
In 1933, once the Nazi Party had reached power, Klemperer, who was Jewish, left Germany and moved to the United States. Klemperer had previously converted to Catholicism, but eventually returned to Judaism. In the U.S. he was appointed Music Director of the Los Angeles Philharmonic Orchestra; there, also, he began to concentrate more on the standard works of the Germanic repertoire that would later bring him greatest acclaim, particularly the works of Beethoven, Brahms and Mahler, though he gave the Los Angeles premieres some of fellow Los Angeles resident Arnold Schoenberg's works with the Philharmonic. He also visited other countries, including England and Australia. While the orchestra responded well to his leadership, Klemperer had a difficult time adjusting to Southern California, a situation exacerbated by repeated manic-depressive episodes, reportedly as a result of severe cyclothymic bipolar disorder.
Then, after completing the 1939 Los Angeles Philharmonic summer season at the Hollywood Bowl, Klemperer was visiting Boston and was incorrectly diagnosed with a brain tumor, and the subsequent brain surgery left him partially paralyzed. He went into a depressive state and was placed in institution; when he escaped, The New York Times ran a cover story declaring him missing, and after being found in New Jersey, a picture of him behind bars was printed in the Herald Tribune. Though he would occasionally conduct the Philharmonic after that, he lost the post of Music Director. Furthermore, his erratic behavior during manic episodes made him an undesirable guest to US orchestras, and the late flowering of his career centered in other countries.
Following the end of World War II, Klemperer returned to Continental Europe to work at the Budapest Opera (1947-1950). Finding Communist rule in Hungary increasingly irksome, he became an itinerant conductor, guest conducting the Montreal Symphony Orchestra, WDR Orchestra Koln, Concertgebouw Orchestra, and the Philharmonia of London. He settled in Switzerland and became the first principal conductor of the Philharmonia in 1959.
Klemperer is less well known as a composer, but he wrote a number of pieces, including six symphonies, a Mass, nine string quartets and the opera Das Ziel. He seldom performed any of these himself and they have been almost entirely forgotten since his death.
A severe fall during a visit to Montreal forced Klemperer subsequently to conduct seated in a chair. A severe burning accident further paralyzed him; it was caused by his smoking in bed with a glass of whisky. Despite this, he continued conducting until his retirement in 1971. His career was turned around in 1954 by London-based producer Walter Legge, who recorded Klemperer in Beethoven, Brahms and much else with his hand-picked orchestra, the Philharmonia, for the premium EMI label.
Klemperer died in Zürich, Switzerland in 1973, aged 88, and was buried in the Israelitischer Friedhof-Oberer Friesenberg in that city.