A Moogle  🏳️‍🌈 💻 🎮 🎶 🇸🇪

it's super confusing that they have .x,.y .w as well as .p[0] to .p[3]

the mysterious 24 word packet that you originally posted, is that what these matrices look like typically or is it supposed to be weird for this game? what does it look like for a game that renders correctly?

Based on looking at the underlying game code, it appears to be a standard function that's used by each game to set up at least the projection matrix. Here's what it is in the scene I screenshotted: 1.0, 0.0, -0.414215, 0.0 0.0, 1.0, -0.310669, 0.0 0.0, 0.0, -1.0, 0.0

The other (rightmost) half of the matrix is the same, except the -1.0 in the bottom row is 0.0, and the lower-right value is 772.531250

this is being made even more confusing by the fact that you target MAME's render engine rather than GL or D3D

MAME doesn't *have* a render engine per se. It's all software rasterization. By the time geometry gets handed over to poly_manager, the verts must be in screen-space for the bitmap to which it will be drawn.