As was the norm in those days, Taraknath got married in his teenage years. But with the consent of his young wife, he practiced brahmacharya. In later years, when Swami Vivekananda heard this, he referred to Taraknath as 'mahapurush.' From then on he, popularly, came to be known as 'Mahapurush Maharaj.' Young Taraknath took up a job in Gaziabad as his family was going through hard times financially. Later on he moved back to Kolkata with a job in an English firm. It was during this time that he had a fortunate opportunity to meet Ramakrishna at the residence of Ramchandra Dutta in May 1880. On the other hand, Ramakrishna, too, was pleasantly surprised to learn that Taraknath was the son of Ramkanai Ghosal, who was his old friend. It was this chance meeting that made Taraknath a direct disciple of Ramakrishna Paramhansa. He relinquished the worldly life and spent his days at Dakshineshwar and in a solitary place in Kolkata (Kankurgachi).
However after the passing away of Ramakrishna, the Baranagar Math was established. Taraknath was one of the first few men to join the Baranagar Math. While receiving the monastic orders, he was given the name 'Swami Shivananda.' As Swami Shivananda he travelled to north India and returned to the Math in 1896. A year later, Swami Vivekananda founded the Ramakrishna Mission in 1897. It was he who sent Swami Shivananda to Ceylon in order to spread Vedanta philosophy. He returned to the Baranagar Math in 1898. In 1902, Swami Shivananda went to Varanasi and established the Ramakrishna Advaita Math and acted as the head there for a span of seven years. However in the year 1910, he was elected the Vice President of Ramakrishna Mission.
Swami Shivananda, in 1922, became the second President of Ramakrishna Math and Mission, after the passing away of Brahmananda Maharaj. Meditation formed an integral part of his teachings. He laid great stress on Sadhana as a form of prayer. He always advocated meditation and study of scriptures alongwith work. He asked his followers to pray to God for strength, dispassion and knowledge. Since not everyone can practice the life of an ascetic, all they can do is pray with love, devotion and sincerity. During this time he also consecrated a large number of people and brought religious consolation to numerous devotees. The diligent and generous Swami Shivananda breathed his last on 20th February 1934. In 1933, he suffered a stroke and was paralysed on one side. Even during his illness he never forgot to remember his lord. A room was dedicated in his memory in Belur Math that is situated close to the old temple.