Cleanup and refactor of zfreeze port

Based on the feedback from pull request #1767 I have put in most of
degasus's suggestions in here now.

I think we have a real winner here as moving the code to
VertexManagerBase for a function has allowed OGL to utilize zfreeze now
:)

Correct use of the vertex pointer has also corrected most of the issue
found in pull request #1767 that JMC47 stated.  Which also for me now
has Mario Tennis working with no polygon spikes on the characters
anymore!  Shadows are still an issue and probably in the other games
with shadow problems.  Rebel Strike also seems better but random skybox
glitches can show up.

1 files changed, 40 insertions, 0 deletions

diff --git a/Source/Core/VideoCommon/VertexManagerBase.cpp b/Source/Core/VideoCommon/VertexManagerBase.cpp
index 38cfd19630..80ea3b5bb9 100644
--- a/Source/Core/VideoCommon/VertexManagerBase.cpp
+++ b/Source/Core/VideoCommon/VertexManagerBase.cpp
@@ -241,3 +241,43 @@ void VertexManager::DoState(PointerWrap& p)
 {
 	g_vertex_manager->vDoState(p);
 }
+
+void VertexManager::CalculateZSlope(u32 stride)
+{
+	float vtx[9];
+	float out[12];
+
+	// Lookup vertices of the last rendered triangle and software-transform them
+	// This allows us to determine the depth slope, which will be used if zfreeze
+	// is enabled in the following flush.
+	for (unsigned int i = 0; i < 3; ++i)
+	{
+		u8* vtx_ptr = s_pCurBufferPointer - stride * (3 - i);
+		vtx[0 + i * 3] = ((float*)vtx_ptr)[0];
+		vtx[1 + i * 3] = ((float*)vtx_ptr)[1];
+		vtx[2 + i * 3] = ((float*)vtx_ptr)[2];
+
+		VertexShaderManager::TransformToClipSpace(&vtx[i * 3], &out[i * 4]);
+
+		// viewport offset ignored because we only look at coordinate differences.
+		out[0 + i * 4] = out[0 + i * 4] / out[3 + i * 4] * xfmem.viewport.wd;
+		out[1 + i * 4] = out[1 + i * 4] / out[3 + i * 4] * xfmem.viewport.ht;
+		out[2 + i * 4] = out[2 + i * 4] / out[3 + i * 4] * xfmem.viewport.zRange + xfmem.viewport.farZ;
+	}
+	float dx31 = out[8] - out[0];
+	float dx12 = out[0] - out[4];
+	float dy12 = out[1] - out[5];
+	float dy31 = out[9] - out[1];
+
+	float DF31 = out[10] - out[2];
+	float DF21 = out[6] - out[2];
+	float a = DF31 * -dy12 - DF21 * dy31;
+	float b = dx31 * DF21 + dx12 * DF31;
+	float c = -dx12 * dy31 - dx31 * -dy12;
+
+	float slope_dfdx = -a / c;
+	float slope_dfdy = -b / c;
+	float slope_f0 = out[2];
+
+	PixelShaderManager::SetZSlope(slope_dfdx, slope_dfdy, slope_f0);
+}