Fix the warning-flagged real bugs across the codebase (#691, 1 of 3) (#725)

* menu_items: fix the real-bug-tier compiler warnings (#691 batch 1)

Six fixes, all in menu_items.c, all verified by menu playtest on macOS:

- BSWAP16(*color0++) advanced the pointer twice per pixel on little-endian
  builds (the macro evaluates its argument twice) and mixed bytes from two
  different pixels. Read first, increment separately. Note: this function
  (func_8009A9FC) currently has no callers, so the corruption was latent,
  not live.
- func_8009B9D0 fell off the end on a lookup miss, returning garbage (the
  existing code comment already suspected this). The miss path now returns
  the display list head unchanged, i.e. draws nothing.
- update_ok_menu_item read an uninitialized stack slot for unknown menu
  item types; now selects an explicit no-animation value, matching what
  the garbage read did in practice.
- pause_menu_item_box_cursor: removed the three empty 'if (x2);' matching
  artifacts. The x/y/z spin state itself is untouched.
- func_800A1FB0: initialized var_s4/var_s5 against the guarded-but-warned
  switch default.
- func_800A54EC: initialized the pause cursor position pointer against its
  unreachable mode-switch default (would have been a null-deref class bug
  if mode values ever grew).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* collision: define the G_ENDDL opcode shifts, return 0 when no tyre surface found

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* replays: cast staff-ghost pointer comparisons, return 0 for empty ghost buffer

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* update_objects: return 0 from conditional step helpers, fix TLUT pointer comparison, init train draw distances

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* render_player: remove the impossible lamp range (vanilla bug, never glowed on N64 either)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* math_util_2: return the vector, not the address of the parameter slot

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Crab: remove inner declaration shadowing the initialized objectIndex

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* editor: honor InverseMatrix failure (bool was compared against 2, always true)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Track.h: drop dead null checks on array members

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* shells: drop always-true angle and surface gates (behavior unchanged, s16 made them tautological)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* particles: make the no-return particle setters void

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* effects/stubs/skybox/main: align signatures with functions that return nothing

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* menus/save: type the pak status variable as s32, return BAD_READ on the fall-off path

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* audio: remove uninitialized-read matching artifacts, init isSound

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* code_80005FD0/code_80086E70: init dead bomb kart pointer and the no_init variable

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* port/engine UI: fix printf-style format types and non-literal format string

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* render_objects: feed the unused texture param to the uninitialized img walker

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Format the changed lines per .clang-format

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Address review: strip explanatory comments, drop vec3f pointer returns

Comments moved to the PR record; the one flagged worth keeping stays.
vec3f_set_xyz/normalize/cross_product return void now since the out
argument is the interface and nothing used the pointer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Update actors_extended.c

* Add checkbox for 'Shells Shoot Straight' option

* Update render_player.c

* Update PortMenu.cpp

* Update render_player.c

* Update actors_extended.c

* Update render_player.c

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: MegaMech <MegaMech@users.noreply.github.com>

1 files changed, 3 insertions, 3 deletions

diff --git a/src/engine/editor/EditorMath.cpp b/src/engine/editor/EditorMath.cpp
index 635d68d59..486b86430 100644
--- a/src/engine/editor/EditorMath.cpp
+++ b/src/engine/editor/EditorMath.cpp
@@ -65,13 +65,13 @@ FVector ScreenRayTrace() {
     guPerspectiveF(perspMtx, &perspNorm, camera->fieldOfView, OTRGetAspectRatio(), CM_GetProps()->NearPersp, CM_GetProps()->FarPersp, 1.0f);
 
     Mat4 inversePerspMtx;
-    if (InverseMatrix((float*)&perspMtx, (float*)&inversePerspMtx) != 2) {
+    if (InverseMatrix((float*) &perspMtx, (float*) &inversePerspMtx)) {
         FVector4 rayEye = MultiplyMatrixVector(inversePerspMtx, (float*)&rayClip.x);
 
         Mat4 lookAtMtx;
         guLookAtF(lookAtMtx, camera->pos[0], camera->pos[1], camera->pos[2], camera->lookAt[0], camera->lookAt[1], camera->lookAt[2], camera->up[0], camera->up[1], camera->up[2]);
         Mat4 inverseViewMtx;
-        if (InverseMatrix((float*)&lookAtMtx, (float*)&inverseViewMtx[0][0]) != 2) {
+        if (InverseMatrix((float*) &lookAtMtx, (float*) &inverseViewMtx[0][0])) {
             rayEye.w = 0;
             FVector4 invRayWor = MultiplyMatrixVector(inverseViewMtx, (float*)&rayEye.x);
 
@@ -206,7 +206,7 @@ FVector TransformVecDirection(const FVector& dir, const float mtx[4][4]) {
 Ray RayToLocalSpace(MtxF mtx, const Ray& ray) {
     MtxF inverse;
 
-    if (InverseMatrix((float*)&mtx, (float*)&inverse) != 2) {
+    if (InverseMatrix((float*) &mtx, (float*) &inverse)) {
         FVector localRayOrigin = TransformVecByMatrix(ray.Origin, (float(*)[4])&inverse);
         FVector localRayDir = TransformVecDirection(ray.Direction, (float(*)[4])&inverse);
         return Ray{localRayOrigin, localRayDir.Normalize()};