It is not necessary to use the SUGARManager to control UI, this is to help your custom UI work with the existing UI templates. If you intend to use only your own UI, then yes, it is perfectly fine to not use the manager, just use SUGARManager.Client in your code to use SUGAR features. This will help to manage requests and callbacks
Thanks