Khan got interested in sarod after a concert at Jagat Kishore Acharya's, where he listened to Ahmed Ali Khan, a student of Asghar Ali Khan. Alauddin became his student, and studied the sarod under him for five years. He went to Rampur in order to take lessons from Wazir Khan Beenkar, court musician of the Nawab there. Through him, Alauddin was able to access to the Senia gharana (Tansen School of music). He became the court musician of Brijnath Singh Maharaja of Maihar Estate in Central Province.
Ustad Allauddin Khan is the founder of the Senia-Maihar gharana, one of the greatest gharanas of instrumental music of the 20th century. Though a Muslim by birth, Baba's forebears, who hailed from Tripura, were Vaishnavites and vegetarians. His father was an ardent devotee of Lord Shiva as also a sitarist, who learnt under Qasim Ali Khan of Senia gharana.
In 1935, he toured Europe, along with Uday Shankar's ballet troupe as well as worked at his institute, 'Uday Shankar India Culture Centre' at Almora for a while. He composed several ragas and laid the foundation of a modern Maihar gharana.
He was a disciple of many great musicians, including Gopal Chandra Banerjee, Lobo, Munne Khan, and after a lot of struggle managed to become a shagird of the legendary Veena player, Wazir Khan of Rampur.
He was awarded the Padma Vibhushan in 1971, and prior to that in 1954, the Sangeet Natak Akademi awarded him with its highest honour, the Sangeet Natak Akademi Fellowship.
He reshaped the Maihar gharana of Indian classical music. He infused the beenbaj and dhrupad ang, surbahar (bass sitar) and sur-sringar (bass sarod), into the playing of many classical instruments. He played many instruments, something that shaped his pedagogy. He put together an orchestra with Indian instruments, the String Band now known as Maihar Band.
He married Madanmanjari Devi and had one son and sarod heir, Ali Akbar Khan, and three daughters, Sharija, Jehanara and Annapurna. He created many ragas of his own that includes Arjun, Bhagabati, Bhim, Bhuvaneshvari, Chandika, Dhabalashri, Dhankosh, Dipika, Durgeshvari, Gandhi, Gandhi Bilawal, Haimanti, Hem-Behag, Hemant, Hemant Bhairav, Imni Manjh, Jaunpuri Todi, Kedar Manjh, Komal Bhimpalasi, Komal Marwa, Madanmanjari, Madhabsri, Madhavgiri, Malaya, Manjh Khamaj, Meghbahar, Muhammed, Nat-Khamaj, Prabhakali, Raj Bijoy, Rajeshri, Shobhavati, Subhabati, Sugandha and Surasati. Manjh Khamaj is the best known.