Because technically there is more than one colour on screen but I don't know how to remove this. If someone does know how to fix this (in unity) please comment
For unity, you can change the UI canvas to be screenspace camera instead of overlay. This means any camera effects will also effect the UI, so you can use a quantisation shader on your camera to limit it to 2 colours
Thank you very much for commenting but I am extremely unfamiliar with shaders in unity and I have no clue how to setup a quantisation shader. I looked around on the internet and couldn't find anything that I would know to be useful (I'm gonna keep looking). Thank you again tho.
I figured out in a way that I could do it! I messed around with random things and figured out making this number really big removes the blur. (This image has the font I'm using)