What makes you think that?
Maybe should have specified. You only get points if you take them. Just seeing them and not taking them does not give any points.
I'd like to second this. After completing 3 whole runs where I picked up every tf item, stepped on every magic circle, and took every cursed object offered, I only have about 19-21 TF mastery points for each of the starting species. Not to mention the dozens of runs where I failed before completing it.
It feels near impossible to level these masteries, and very unclear on how you are supposed to advance them.