Construction workers in the coastal district of Mannar stumbled on at least 10 skeletal remains buried at a location where they were laying a new water pipe, said police spokesman Ajith Rohana.
"A judicial medical officer has gone to the site. Further forensic examinations are underway to determine the age of the mass grave," Rohana told AFP. "Additional staff are being inducted for the investigation."
There was no immediate indication who the victims were or how and when they had died.
However, it is the first time that evidence of a mass grave has emerged in the former war zone since troops declared victory over separatist Tamil Tiger guerrillas in May 2009.
Both government forces and Tamil Tiger rebels have been accused of killing civilians during the 37-year separatist war.
Sri Lanka has denied allegations that its troops killed up to 40,000 civilians in the final months of fighting.
Earlier this year construction workers stumbled on another mass grave in central Sri Lanka, hundreds of kilometres away from the conflict zone.
At least 154 people had been buried in the grave at the central district of Matale, a hotbed of an anti-government uprising between 1987 and 1990.
Local forensic experts said it dated back to a period when the then-government led a crackdown on leftist Sinhalese rebels.
The insurgency by the mainly Sinhalese JVP was unrelated to the separatist campaign in the north and east by Tamil rebels.
The UN has estimated that at least 100,000 people were killed in Sri Lanka's Tamil separatist war