Looks like this is a quirk of how LayeredImageProxy handles transforms with arguments - you get a similar error if you try to use a transform like
transform test(a=0.5): alpha a
and use the test transform for LayeredImageProxy. It's expecting to receive the child as the first argument, which you can get around via
transform outline_test(child=None, width=1, color="#fff", smoothing=1.0, end_color=None, num_passes=1, gradient_smoothing=None, is_mesh=True, mesh_pad=False, drawable_res=False, offset=(0, 0), smoothstep=True, power=None): outline_transform(width, color, smoothing, end_color, num_passes, gradient_smoothing, is_mesh, mesh_pad, drawable_res, offset, smoothstep, power)(child)
and then use outline_test instead of outline_transform for your proxy. Hope that helps! Might be worth reporting to Ren'Py honestly since it seems unintuitive that it causes this issue.