How to translate D3DXMatrixLookAtLH to Unity?

Hello everyone. I am currently trying to port a really old Game I have the source from to Unity as a fun project. And I'm currently stuck with the object rotation techniques in Unity compared to Direct 3D9 (DirectX).

The object displays all properly already, also having the correct scaling. Just the rotation is completely off. I'm having this DirectX Code that handles the object rotation:

 D3DXMATRIX myDXMatrix;
 D3DXVECTOR3 objectPos, objectVel, objectUp; // <- these are comming from the object properties
 D3DXMatrixLookAtLH( &myDXMatrix, &objectPos, &(objectPos+objectVel), &objectUp);
 D3DXMatrixInverse( &myDXMatrix, NULL, &myDXMatrix);

And I'm trying to produce the same in Unity. What I came up with:

1. Try

 // This function gets called with a GameObject objects transformation property
 private void RotateDirect3DObjectInUnity(Transform transform, D3DXVECTOR3 pos, D3DXVECTOR3 up, D3DXVECTOR3 vel)
 {
             // Convert DX to Unity vectors
             Vector3 _pos = new Vector3(pos.x, pos.y, pos.z);
             Vector3 _up = new Vector3(up.x, up.y, up.z);
             Vector3 _vel = new Vector3(vel.x, vel.y, vel.z);
 
             // Normalize up vector
             _up.Normalize();    // previously: D3DXVec3Normalize(&m_vUp,&m_vUp);
 
             // Looking at the target, the way I understand they called D3DXMatrixLookAtLH
             // specifying world and up position vectors
             transform.LookAt(_pos + _vel, _up);
 }

This caused all objects to point towards the sky like the target would be really far away.

2. Try This is the try that got my the furthest. Though it's a lot of converting around and very messy.

I tried to rewrite D3DXMatrixLookAtLH using the MSDN documentation as far as I could. They provided this pseudo code of how the function works:

 zaxis = normal(At - Eye)
 xaxis = normal(cross(Up, zaxis))
 yaxis = cross(zaxis, xaxis)
     
  xaxis.x           yaxis.x           zaxis.x          0
  xaxis.y           yaxis.y           zaxis.y          0
  xaxis.z           yaxis.z           zaxis.z          0
 -dot(xaxis, eye)  -dot(yaxis, eye)  -dot(zaxis, eye)  1

So I went writing that method:

 /// <summary>
 /// <para>Direct X method D3DXMatrixLookAtLH</para>
 /// </summary>
 /// <param name="eye">Eye Vector</param>
 /// <param name="at">At Vector</param>
 /// <param name="up">Up vector</param>
 /// <returns>Unity Matrix of the calculation</returns>
 public static Matrix4x4 MatrixLookAtLH(Vector3 eye, Vector3 at, Vector3 up)
 {
     // Generate matrix
     Matrix4x4 matrix = new Matrix4x4();
 
     // Get axes
     Vector3 zAxis = (at - eye).normalized;
     Vector3 xAxis = Vector3.Cross(up, zAxis).normalized;
     Vector3 yAxis = Vector3.Cross(zAxis, xAxis);
 
     // Set up the matrix
     matrix.m00 = xAxis.x;   matrix.m01 = yAxis.x;   matrix.m02 = zAxis.x;   matrix.m03 = 0;
     matrix.m10 = xAxis.y;   matrix.m11 = yAxis.y;   matrix.m12 = zAxis.y;   matrix.m13 = 0;
     matrix.m20 = xAxis.z;   matrix.m21 = yAxis.z;   matrix.m22 = zAxis.z;   matrix.m23 = 0;
     matrix.m30 = -Vector3.Dot(xAxis, eye);  matrix.m31 = -Vector3.Dot(yAxis, eye);  matrix.m32 = -Vector3.Dot(zAxis, eye);  matrix.m33 = 1;
 
     // Return the calculated matrix
     return matrix;
 }

And as I could not find a way to directly specify a Matrix4x4 to a Transform object, I converted that to a Quaternion object using a method I found browsing the net:

 public static Quaternion QuaternionFromMatrix(Matrix4x4 m)
 {
     return Quaternion.LookRotation(m.GetColumn(2), m.GetColumn(1));
 }

Then I rotate the object like this:

 // This function gets called with a GameObject objects transformation property
 private void RotateDirect3DObjectInUnity(Transform transform, D3DXVECTOR3 pos, D3DXVECTOR3 up, D3DXVECTOR3 vel)
 {
             // Convert DX to Unity vectors
             Vector3 _pos = new Vector3(pos.x, pos.y, pos.z);
             Vector3 _up = new Vector3(up.x, up.y, up.z);
             Vector3 _vel = new Vector3(vel.x, vel.y, vel.z);
             
             // From previous Direct X code:
             // D3DXMatrixLookAtLH(&m_mMatrix, &m_vPos, &(m_vPos + m_vVel), &m_vUp);
             // D3DXMatrixInverse(&m_mMatrix, NULL, &m_mMatrix);
             Matrix4x4 matrix = MatrixLookAtLH(_pos, _pos + _vel, _up).inverse;
             transform.rotation = QuaternionFromMatrix(matrix);
 }

This rotates 80% of the objects on the map absolutely inch perfect. Though 20% remain twisted around randomly and I cannot understand why.

So that is what I tried so far. Now...

How can I translate D3DXMatrixLookAtLH properly in to Unity compatible code?

Sorry for the long post and thanks to anyone who is bothering reading this!