According to Kerala Mahatmyam, this temple was consecrated by Lord Parashurama - one of the incarnations of Vishnu. Yet hearsay evidences point out that this temple is about 2000 years old.

According to a legend this temple was consecrated by a sage named Swetharshi. The legend says Agasthya Muni on his way to River for offering his customary ablution was confronted by two demons named Neela and Swetha. Agasthya Muni cursed them. On their request for penance, Agasthya Muni advised them to perform Tapas at two Shiva Temples. The place where Neela performed Tapas came to be known as Neeleswaram. Swetha performed his Tapas at Thiruvangad (Tiru-van-kad, meaning the sacred dense forest). They were later known as Neelarshi and Swetharshi. It is said that Swetharshi consecrated a Shiva temple & Vishnu temple at the place where he performed his Tapas.