As per documentation, tj_encode accepts arrays, TJ objects, TJ booleans, and basic values. It does not accept maps, as there isn't even a way to tell that your value is a map, let alone tell which children of it might be maps.
So you'd want to do
var obj = tj_object("a",10, "b",50);
var json_string = tj_encode(map);
show_debug_message( "result: "+ json_string );
or
var obj = tj_object();if you can describe your use case in more detail, I may have further recommendations.
tj_set(obj, "a", 10);
tj_set(obj, "b", 50);
var json_string = tj_encode(map);
show_debug_message( "result: "+ json_string );