The city of Ujjain has numerous shrines dedicated to Bhairava, a vicious manifestation of Shiva, the destroyer of the Universe. The Kalbhairava Temple is also believed to be associated with the cult of Tantra, an unorthodox secret cult with strong black magic overtones. Although Tantra practices on the sly are said to keep enemy at bay, missing the target may spell doom to the practitioner.

Many sadhus or holy men with ash-smudged bodies and long matted hair can easily be spotted around this temple. Pieces of sculpted stones depict deities from Hindu pantheon along with Vishnu. A Shiva lingam is enshrined under a banyan tree inside the fringe of the temple opposite Nandi the Bull. According to legends, the bull was given to Shiva and Parvati as a wedding present from Daksha, Parvati`s father.

Like the beautiful lampstands of the Harsiddhi Temple, there are related ones in Kalbhairav Temple too. The temple comes to life especially on the festival of Mahashivratri, when devotees congregate in great numbers.