As my skills level up for things like fishing, it shows that my catch chance is going down even when not changing what I am fishing for.
Hey, thanks for flagging that. That number is your chance relative to the fish you're on, not your skill overall: it starts at 50% at the fish's required level and climbs 1% per level above it, up to 95%.
So when you level into a new fish, it resets to 50% for that one. Shrimp at level 29 sits at 78%, but Trout at level 30 starts back at 50%. If a quest or a tap switched you to the newly unlocked fish, it can look like levelling up made you worse when it's actually a fresh, more valuable fish.
If you stay on the same fish it only ever goes up. I'll make the panel spell this out so it isn't so confusing.