Thanks a lot for your extensive response. I still need to spend more time digesting the pieces of your code and articles that you mentioned, but they seem to address most of my concerns.
My robot arm has also 5 DOF, very similar to Dexter. The way I handle inverse kinematics is by matching the orientation of the object that needs to be picked up with the last two DOF's (I think like what you have done for Dexter). For the rest of the 3 DOF's I've written down the transformations in terms of joint rotations and have solved them explicitly for a given target position. But I think the systematic representation based on quaternions is much cleaner for implementing the transform hierarchies and are easier to extend them later.
For now, I'm planning to stick to python, because of the other constrains including the interaction with the hardware (Arduinos, sensors,...) and also other available math libraries like opencv. But on the downside, creating an attractive and interactive 3d graphics seems to be a pain in python. But I think at some point I should find a way to either use external tools for handling the graphics or instead write the whole thing as a web application and use Unity for handling the graphics.