Not 100% sure if I did this the right way based on what you said, but this ended up fixing it! Edit: removed the image because that code was wrong... but I adjusted it so that now the prefab paths are in the correct spot and the cursor marker retains the correct size and position. Thanks for the help!
Viewing post in MariaEngine 64 + Open World Edition comments
Hello again (Not sure if I should start a new thread since this one's getting pretty long...)
So as I'm using the Maria64 engine, I'm trying to clean/organize it a bit so that it'll be easier for me to work with as I make my game. This project of mine will be my next Steam release so I plan on going all out with 3D animated character models, realistic water, lighting, etc...
Recently though I'm having trouble with the collision relating to cylinder blocks. As you can see in the video here, the first cylinder's collision works as expected, but with the 2nd one (that's closer to the edge of the level) my bird character is able to clip through half of that cylinder. So the collision shape for that one is only a semicylinder. I have no idea why this is happening (and I tried checking if this happens in the original OpenWorld project too, but I'm not using the Skelani system for my character so maybe that messed something up in relation to hitboxes/collision?) Do you know what could be causing this?
Hmmm... my gut instinct is that it might be placed on a boundary for the collision checking grid, and is erroneously only added to ONE of the cells (objects that overlap a boundary are supposed to be added to all cells they touch).
I would check this by copypasting the weird cylinder and placing it in a bunch of different positions (altering both X and Y) and seeing if they are also glitchy or will work as intended. In particular if you move it sideways along the axis it's "cut off" at, EVERY cylinder at those coordinates would have half of its collision data missing, and if you move it "into the level"-wards, only the overlapping section would be missing collision data.
(And if this is actually the case, the next step would be adding some debug messages in terrain_buffer_add to investigate further, to e.g. print the "key" and ensure it gets added to every cell you expect)
Relatedly, is this line where the collision is broken the zero of any coordinate? I just realized terrain_buffer_get_key is using the "div" operator (integer division) and if GM interprets "round down" as "round towards zero" (instead of "round towards negative infinity" which is the correct definition) both the collision grid cells directly adjacent to the zero of any axis will map to the same key.
After trying out everything you said, checking the weird cylinder I realized that my sprite origin for the cylinder object was at 0,0 instead of in the middle center, fixing that seems to have solved this ... so I guess the sprite's offset somehow messes up the cylinder's collision boundary? Sorry, that was my bad! 😭(I'm still learning how this collision-checking grid works... :P)
I also have a question in regard to the player hitboxes - when I change the hitbox_radius variable in the player object, it doesn't change how close I can get to the terrain model, I can still kind of clip through it a bit (it stops at the player's exact x/y coordinate). How would I expand this to cover my entire bird character's model?
I'm working off of the OpenWorld project since that one is more up-to-date and includes extra stuff like the WMHM and footstep sounds, but since it uses a 2D flat sprite for the character, I want to bring back the 3D model collision that the legacy version uses. Did it use a 3D AABB/cylinder for Super Maria? How did you make sure that it avoided clipping through terrain?
Ah... yeah, the code assumes the origin for cylinders is in the center, so having it be in the top left corner will offset everything by the radius amount on both axes. Glad that got sorted out in the end at least.
The player movement (the vast majority of movement is done in psbp_inertia_horizontal / psbp_inertia_vertical) mainly uses two functions, terrain_hit_pilloid_cheap and slope_hit_optimized, the latter of which in turn uses terrain_buffer_lookup_verticalslice. Both of these intentionally just check 3 points (XY center @ Z bottom, Z top, and Z centerpoint) since collision checking turned out to be the most expensive operation in the game. But there's a dummied-out alternate version of terrain_hit_pilloid_cheap that does check the full 12 points (taking the radius into account). Replacing the 3-point version with this, and then using slope_hit_pilloid_cheap in the player inertia functions instead of slope_hit_optimized, should restore the old collision checking as it was. (Note that slope_hit_optimized uses a z height argument that slope_hit_pilloid_cheap doesn't use so you might need to remove that to avoid GM complaining about passing too many arguments)