You can put a partially filled Memory into a Diary, but you can't add more Experiences to that Memory after you do so.
When you put a Memory into a Diary your character functionally forgets it. That Memory slot is now free for all new Memories, the Memory that was there is now a Diary entry.
You should only be putting Memories into a Diary when you are out of Memory slots. Don't casually add three Memories to a Diary, that's just not the way the game is supposed to work mechanically.
Does this help?