VertexManagerBase: Skip drawing objects with mismatched xf/bp stages

Hardware tests have shown that if the number of texgens/channels do not
match, you get garbage rendering. Presumably because the output
registers from the XF stage are fed into the incorrect input registers
for TEV/BP.

Currently, this causes Dolphin to crash/generate invalid shaders with an
assertion failure in the hardware backends. Instead, we log an error.

Perhaps in the future we should just spit out all texgens/colors anyway
from both stages, and let cross-stage optimization take care of DCE'ing
it away. But doing so would require changing the UIDs and invalidating
everyone's shader caches.

1 files changed, 11 insertions, 0 deletions

diff --git a/Source/Core/VideoCommon/VertexManagerBase.cpp b/Source/Core/VideoCommon/VertexManagerBase.cpp
index 7fcd4935c0..56f9f68b26 100644
--- a/Source/Core/VideoCommon/VertexManagerBase.cpp
+++ b/Source/Core/VideoCommon/VertexManagerBase.cpp
@@ -344,6 +344,17 @@ void VertexManagerBase::Flush()
 
   m_is_flushed = true;
 
+  if (xfmem.numTexGen.numTexGens != bpmem.genMode.numtexgens ||
+      xfmem.numChan.numColorChans != bpmem.genMode.numcolchans)
+  {
+    ERROR_LOG(VIDEO,
+              "Mismatched configuration between XF and BP stages - %u/%u texgens, %u/%u colors. "
+              "Skipping draw. Please report on the issue tracker.",
+              xfmem.numTexGen.numTexGens, bpmem.genMode.numtexgens.Value(),
+              xfmem.numChan.numColorChans, bpmem.genMode.numcolchans.Value());
+    return;
+  }
+
 #if defined(_DEBUG) || defined(DEBUGFAST)
   PRIM_LOG("frame%d:\n texgen=%u, numchan=%u, dualtex=%u, ztex=%u, cole=%u, alpe=%u, ze=%u",
            g_ActiveConfig.iSaveTargetId, xfmem.numTexGen.numTexGens, xfmem.numChan.numColorChans,