UPDATE: I think I know what happened. When I tested it earlier, I didn't have my headphones/bluetooth mic on. It seems the issue arrives when I have my headphones connected. I'll try with some other stuff I have, but it looks like I need to invest in a proper mic :,)