koobas.hobune.stream
can anybody help with script that takes webcam texture pixel information to deform mesh vertices?
i'm trying to use a webcam texture -- get its pixel information- to then assign to mesh vertices so that while the image color/alpha changes, the mesh will deform at the same time.
i am using a mac, so things are slightly different, but i have managed to get a mesh to create on play and the webcam to work.
i now need to figure out how to get the pixel info and assign it to vertices, and this is where i'm stuck, because i'm not sure how Unity uses textures and shaders. its pretty mysterious to me.
anyway, any help would be GREATLY appreciated. here's the code as it is now:
using UnityEngine;
public class WebCamMeshTest : MonoBehaviour { public bool getDeviceByName = false; public int deviceIndex = 0; public string deviceName = ""; public bool setRequestResolution = false; public int resolutionWidth = 800; public int resolutionHeight = 600;
private int resolution = 80 ;
public bool setFPS = false;
public int requestedFPS = 30;
public bool mirrorHorizontal = false;
[HideInInspector]
public bool running = false;
private WebCamTexture wcTexture;
private bool foundDevice = false;
private Mesh mesh;
private int totalPixels;
private Vector3[] vertices;
private Color[] colors;
private Vector3[] normals;
private Vector2[] uv;
//public Gradient coloring;
void Start()
{
totalPixels = resolution * resolution;
//totalPixels = resolution * resolution;
mesh = new Mesh();
mesh.name = "Surface Mesh";
GetComponent<MeshFilter>().mesh = mesh;
vertices = new Vector3[(resolution+1) * (resolution+1)];
colors = new Color[vertices.Length];
Vector2[] uv = new Vector2[vertices.Length];
Vector3[] normals = new Vector3[vertices.Length];
int vertexIndex = 0;
float stepSize = 1f / resolution;
for (int v = 0, z = 0; z <= resolution; z++) {
for (int x = 0; x <= resolution; x++, v++) {
vertices[v] = new Vector3(x * stepSize - 0.5f, 0f, z * stepSize - 0.5f);
colors[v] = renderer.material.mainTexture.GetPixels();
//colors[v] = coloring.Evaluate(sample);
normals[v] = Vector3.up;
uv[v] = new Vector2(x * stepSize, z * stepSize);
}
}
mesh.vertices = vertices;
mesh.colors = colors;
mesh.normals = normals;
mesh.uv = uv;
int[] triangles = new int[resolution * resolution * 6];
for (int t = 0, v = 0, y = 0; y < resolution; y++, v++) {
for (int x = 0; x < resolution; x++, v++, t += 6) {
triangles[t] = v;
triangles[t + 1] = v + resolution + 1;
triangles[t + 2] = v + 1;
triangles[t + 3] = v + 1;
triangles[t + 4] = v + resolution + 1;
triangles[t + 5] = v + resolution + 2;
}
}
mesh.triangles = triangles;
if(WebCamTexture.devices.Length > 0)
{
wcTexture = new WebCamTexture();
if(setRequestResolution)
{
wcTexture.requestedWidth = resolution;
wcTexture.requestedHeight = resolution;
//wcTexture.requestedWidth = resolutionWidth;
//wcTexture.requestedHeight = resolutionHeight;
}
if(setFPS)
{
wcTexture.requestedFPS = requestedFPS;
}
if(getDeviceByName)
{
foreach (WebCamDevice a in WebCamTexture.devices)
{
if(a.name == deviceName)
{
foundDevice = true;
}
}
if(!foundDevice)
{
Debug.LogWarning("WebCam - Cannot find named device");
}
else
{
wcTexture.deviceName = deviceName;
}
}
else
{
if(deviceIndex < WebCamTexture.devices.Length)
{
wcTexture.deviceName = WebCamTexture.devices[deviceIndex].name;
deviceName = WebCamTexture.devices[deviceIndex].name;
foundDevice = true;
}
else
{
Debug.LogWarning("WebCam - Index out of range. Webcams Detected: " + WebCamTexture.devices.Length.ToString());
}
}
if(foundDevice)
{
renderer.material.mainTexture = wcTexture;
wcTexture.Play();
if(!wcTexture.isPlaying)
{
Webcam[] wcs = GameObject.FindObjectsOfType<Webcam>();
foreach(Webcam a in wcs)
{
if(a.running)
{
if(a.deviceName == deviceName)
{
renderer.material.mainTexture = a.renderer.material.mainTexture;
}
}
}
}
else
{
running = true;
}
if(mirrorHorizontal)
{
Vector2 tempScale = renderer.material.mainTextureScale;
Vector2 tempOffset = renderer.material.mainTextureOffset;
tempOffset.x += tempScale.x;
tempScale.x *= -1;
renderer.material.mainTextureScale = tempScale;
renderer.material.mainTextureOffset = tempOffset;
}
}
}
else
{
Debug.LogWarning("WebCam - No Webcam Device Detected");
}
}
void OnDestroy()
{
if(running)
{
wcTexture.Stop();
}
}
}
Though I did not verify it for accuracy, this is fairly sophisticated code. Is it yours or did you find it somewhere? You also don't define how you want to deform the texture...what color attribute makes what happen with the mesh. Have you taken a look at the (free) procedural examples in the Asset store?
i pieced it together from two seperate scripts. one was a webcam script and the other creates a mesh and then deformes it based on a perlin noise.
i want to deform the mesh, not the texture. so that as the image moves/pixels change in the video, the corresponding vertices will raise or lower (like a terrain moving underneath a player).
i'm not sure which procedural examples specifically you're referring to. i've looked, but not found anything that seems to apply.
in the mesh script i was looking at, it assigned colors to the vertices based on noise, then had an attached shader that needed to be able to read that. do i also need to mess with shaders, or can it directly in this script? shaders are completely foreign to me...
Thanks for your help! --j
I was referring to these procedural examples:
http://u3d.as/content/unity-technologies/procedural-examples/3zu
Again, you don't give the criteria for defor$$anonymous$$g the mesh. As you will see in the examples, you can just change the local coordinate of the vertices and reassign the vertices array to the mesh when done. I can give you a bit more help, you you need to define the critera for the deform...how will the color information be used to deform the mesh.
so, i asked a friend who has more coding experience than me, but who still wasn't sure how to do it. and they sugested adding something like this at the end above the last curly bracket, but i'm getting these error messages:
No overload for method GetPixel' takes 4' arguments
and: NullReferenceException: Object reference not set to an instance of an object WebCam$$anonymous$$eshTest.Update () (at Assets/WebCam$$anonymous$$eshTest/WebCam$$anonymous$$eshTest.cs:184)
here's that additional script:
void Update() { if (renderer.material.mainTexture == null) return;
Vector3[] vertices = mesh.vertices;
int vi=0;
// Iterate through texture pixels and set height
for(int x=0; x<resolution; x++) {
for(int y=0; y<resolution; y++) {
Texture2D mainTexture = renderer.material.mainTexture as Texture2D;
Color pixel = mainTexture.GetPixel(x,y,resolution, resolution);
float pixelValue = pixel.grayscale;
Vector3 currentVertex = vertices[vi];
currentVertex.y = pixelValue;
vertices[vi] = currentVertex;
vi++;
}
}
mesh.vertices = vertices;
}
GetPixel has just two parameters:
Color pixel = mainTexture.GetPixel(x,y);
But calling GetPixel() for every pixel will be very slow. The faster way would be to use GetPixels() or better yet GetPixels32() to get the full array of colors. The array is 1D, so there is a bit of math involved in addressing it as 2D. GetPixels32() does not have a 'grayscale' so you would need to do it by hand, but it is the most efficient method of getting the color data.
Edit, actually the 1D vertices array and the 1D color array either are the same, or can be made to be the same, so you can just cycle through the arrays in parallel.
