Unnamed / Coxstill Rd Lagoon

Can someone enlighten me as to some fishing and boating rules in ga ? I'm on a small pond which has 7-8 houses on the banks and we obtained permission from one of the owners to fish. So we're in the boat and we went towards a corner of the pond and this dude comes out of his house and threatened to call the cops on us and charge us with trespassing if we didn't stay behind a certain tree on the far side of the pond and the corner of his house. Can he do that ? I thought if you had permission to fish a pond you could fish the whole pond as long as you didn't touch dry land on his shore line ? We were approximately where I placed the catch position when he threatened us about 45 yards from him

Caught around 11-12 LMB all caught on a white and chartreuse z-man chatterbait with a rage swimmer bait trailer. All weighed between 1.7-3.8. And one surprise crappie that snatched a spook from under a magnolia tree limb. Overall a very fun day on the water with @catch em all