As i see now, location is accessible thru map: "Lands" icon, left of Sakura Town.
It's some mistake i think, that u can't get there thru gate in town (it leads to a small location forest), next to a shrine, but when returning to town with direction arrows, these gate are used.