Joe Haider performed as an amateur musician in the region Stuttgart between 1954 and 1959 and studied at Richard Strauss Conservatory in Munich from 1960 to 1965. In this time, he played also in the quintet of Fritz Münzer (Live in HR 1962, Jazz for young people).
From 1965 to 1968, he worked as a pianist and leader of a trio in jazz club Domicile in Munich and performed there with many European and American jazz musician such as Benny Bailey, Duško Gojković, Nathan Davis, Booker Ervin, Klaus Doldinger, Hans Koller, Leo Wright, Attila Zoller, George Mraz, Peter Trunk, Philly Joe Jones, Joe Nay, Kurt Bong, Klaus Weiss and Pierre Favre.
After leading the Radio Jazz Ensemble of the Bayerischer Rundfunk, he worked from 1970 with the quartet Four for Jazz (Heinz Bigler (alto saxophone), Isla Eckinger (bass) and Peter Giger, (percussion)) and founded his own trio with Eckinger and Favre and founded a Combo with Duško Gojković and lead a Big Band together with Slide Hampton, where Dexter Gordon performed also. In 1979, he founded his own label EGO to release records of his and his German colleagues. In 1988, he achieved the Deutscher Schallplattenpreis for recording of his next large size project Joe Haider Orchestra featuring Mel Lewis. His tours with Woody Shaw was followed by Eckinger and Wolfgang Haffner and various soloists such as Andy Scherrer, Roman Schwaller, Sandy Patton or Don Menza. From 2000 bis 2011, Haider worked with Brigitte Dietrich and with a double quartet (with bowed string instruments). In 2016, he published the album Keep It Dark.
From 1984 to 1995, he was the director of the Swiss Jazz School in Bern. In 1994, the Canton of Bern's government awarded him the Great Cultural Prize for his contribution in the field of music.