The 4th Regiment of Bengal Native Infantry was founded around the year 1798, when the unit was part of the 2nd Battalion, 16th Bengal Native Infantry. The regiment was consisted of a number of titles through out its existence, such as the 33rd Bengal Native Infantry from 1824 to 1861, the 4th Bengal Regiment of Bengal Native Infantry from the year 1861 to 1890, the 4th Prince Albert Victor's Bengal Infantry during the years 1890 and 1897, and the 4th Prince Albert Victor's Rajput Regiment from 1897 to 1901. The 4th Regiment of Bengal Native Infantry was renamed for the final time as the 4th Prince Albert Victor's Rajputs, when the names of the British Indian presidencies were omitted after the Kitchener reforms of the British Indian Army.
The 4th Regiment of Bengal Native Infantry participated in the Rawalpindi Parade held in the year 1905 to accolade the visit of the Prince and Princess of Wales in British India. The former British Indian Army unit also participated in the conquest to Bhurtpore (Bharatpur) from the year 1825 to 1826. The regiment as actively fought in the First Anglo Sikh War, the Second Anglo Afghan War, the Third Anglo Burmese War from 1885 to 1887 and the First World War.
After the culmination of the First World War, the British Government of India restructured and re-organised the army. The separate single battalion infantry regiments were merged together in order to develop more prominent multi battalion regiments. The 4th Regiment of Bengal Native Infantry (4th Prince Albert Victor's Rajputs) became the 2nd Battalion, 7th Rajput Regiment.
On 15th August 1947, the nation achieved independence from the rule of British Empire in India. As a result, the country was divided into the Union of India and the Dominion of Pakistan through the Partition of India. The 4th Regiment of Bengal Native Infantry was allocated to the Indian Army.