The incident occurred at Kotrupi along Pathankot-Mandi National Highway.
"The army and NDRF have been called while police, home guard and fire services have already been deployed in the rescue and relief operation", Kadam said.
A more than 150 metre stretch of the road was washed away in the mudslide.
"Three vehicles were stranded after the landslide".
One of the buses was going from Chamba to Manali and the other from Manali to Katra. Survivors of the disaster have been shifted to hospital and are undergoing emergency treatment. He added that 4 bodies had so far been recovered. The minister also posted helpline numbers of the HRTC and the district authorities.
The landslide in Urla village in Himachal Pradesh state, which was triggered on Sunday by heavy monsoon rains, buried part of the highway, trapping two cars and a motorbike as well as the buses.
Local media reported 30 people were feared dead in the disaster, while other reports suggested an even higher death toll.
Army and NDRF teams rushed to the rescue, with more than 100 Army personnel of 9 Grenadier reaching the site from Palampur.
Officials said 21 passengers travelling to Manali got their tickets booked online.
Prime Minister Narendra Modi today condoled the deaths of bus passengers in a landslide in Mandi district of Himachal Pradesh. He directed to assess the loss to the property at the earliest so that the sufferers could be provided compensation and other assistance. "My condolences with the families of the deceased", Modi tweeted. He also prayed for the quick recovery of those injured in the incident.
Chief Minister Virbhadra Singh, who visited the spot along with his Cabinet colleagues, announced an ex-gratia of Rs 5 lakh to the next of kin of those killed.