Panama, Kumana, Helawa and Ragamwela residents who have tried to resettle after the war say they were prohibited by the SLN, which the villagers charge of land grabbing.
Residents of Ragamwela who were chased away by an armed group almost a year ago say they are now helpless and no progress has been made in investigations over the attack.
A farmer, S Upali Attanayake told journalist Wasantha Chandrapala that it is extremely difficult for the families to survive as they have no land to cultivate.
In Ragamwela, nobody is allowed to resettle or cultivate the lands. The only presence in the village is a police check point.
Lagoon fishing 'threatned'
K Jayawardene, a resident of Panama told BBC Sandeshaya: "When the terrorist problem was over, the navy came and grabbed our land. Now they have built their own houses just behind our lands."
Sri Lanka navy camp off Panama lagoon (photo: Wasantha Chandrapala)
And fishermen are too are complaining of another serious problem they face.
They fear of losing the Panama lagoon where the livelihood for centuries has been fishing.
A protesting fisherman told BBC Sinhala service, Sandeshaya "We have been living in this lagoon. Now for a few weeks navy does not allow us to come to the lagoon."
"Even our lands were forcefully grabbed. We were beaten and chased away from our lands," he added.
"They have built an electric wire surrounding our lands. All our lands were captured," said another resident.
MoD land
But the SLN denies accusations that it is grabbing residents land to establish a new camp.
Our lands were forcefully grabbed. We were beaten and chased away from our lands
Panama residents
SLN spokesman, Commander Kosala Warnakulasuriya told journalist Wasantha Chandrapala that the new camp is established in a land provided by the ministry of defence.
"The main reason for the new camp is to strengthen the security in the area. Grabbing land from residents is not happening at all," he said.
Panama is in the vicinity of the Yala wildlife sanctuary and the Arugambay surfing resort.
Tourism industry in the area, seeing a revival following decades of war, has attracted the interest of many local and international investors.
Many mega-projects, including new Tourism Promotion Zones in Kalpitiya, in north western Sri Lanka, are under way to welcome new arrivals.
In Kalpitiya, a new zone where fourteen islands are for lease is being developed with facilities for adventure tourism, as well as whale and dolphin-watching and an underwater visitor centre.
BBC Sinhala