ARCore for Unity save camera image

Question

Using ARCore for Unity, trying to save Frame.CameraImage.AcquireCameraImageBytes() as image and scanning the image for QR code. But the converted image is not in actual scale and it is repeating, so not able to deduct the QR code correctly.

Here is my code

 void Update()
 {
 using (var image = Frame.CameraImage.AcquireCameraImageBytes())
     {
         if (image.IsAvailable)
         {               
 
             byte[] m_EdgeImage = null;
             Color32[] pixels = null;
             IParser Parser = new ZXingParser();
             if (_texture == null || m_EdgeImage == null || _texture.width != image.Width || _texture.height != image.Height)
             {
                 _texture = new Texture2D(image.Width, image.Height, TextureFormat.RGBA32, false, false);                    
                 m_EdgeImage = new byte[image.Width * image.Height*4];           
 
             }
             System.Runtime.InteropServices.Marshal.Copy(image.Y, m_EdgeImage, 0, image.Width * image.Height);               
             _texture.LoadRawTextureData(m_EdgeImage);
             _texture.Apply();       
 
             ParserResult Result = Parser.Decode(pixels, _texture.width, _texture.height);
             if (Result != null)
             {
                 Debug.Log("QRCODE");
 
             }
             else
             {               
                 var encodedJpg = _texture.EncodeToJPG();
                 var path = Application.persistentDataPath;
                 File.WriteAllBytes(path + "/test.jpg", encodedJpg);             
                 Debug.Log("NOQRCODE");
                 Application.Quit();
             }
         }
     }
 }

Here is the converted image

alt text

What is wrong here and why the image is repeating

Answer 1

You've probably figured this out by now but I thought I'd answer this to help anyone who stumbles across this.

You don't actually have to convert the CameraImageBytes into a Texture2D to scan for barcodes - you can use a byte[] instead.

I've also spun off a Task to do the image acquisition in this example to avoid the frame stutter you get from getting the camera data on the main thread.

Because of this, the callback should avoid using the Unity APIs.

To pass control back to the main thread while performing your desired action you can start a coroutine inside of the callback - this will then be executed on the main Unity thread.

Be aware that you would probably want to modify this example to not overlap tasks (there is no guarantee that the task will be done in the 0.5 second interval I've set here), and could perhaps look into using Unity Jobs for the multi-threading.

 using System;
 using System.Runtime.InteropServices;
 using System.Threading.Tasks;
 using BarcodeScanner;
 using BarcodeScanner.Parser;
 using GoogleARCore;
 using UnityEngine;
 using ZXing;
 
 namespace AndroidQRCodeScanExample
 {
     class ExampleClass : MonoBehaviour
     {
         private ZXingParser Parser { get; set; }
 
         /// <summary>
         ///  Set up the ZXingParser and set up our scanner to invoke every 0.5 seconds
         /// </summary>
         void Start()
         {
             var scannerSettings = new ScannerSettings();
             Parser = new ZXingParser(scannerSettings);
 
             InvokeRepeating("Scan", 0, 0.5f);
         }
 
         /// <summary>
         /// Capture and scan the current frame 
         /// </summary>
         void Scan()
         {
             Action<byte[], int, int> callback = (bytes, width, height) =>
             {
                 if (bytes == null)
                 {
                     // No image is available.
                     return;
                 }
 
                 // Decode the image using ZXing parser
                 var result = Parser.Scanner.Decode(bytes, width, height, RGBLuminanceSource.BitmapFormat.Gray8);
                 var resultText = result.Text;
 
                 // Do whatever you would like with the result here
                 StartCoroutine(DoSomethingWithThisInformation());
             };
 
             CaptureScreenAsync(callback);
         }
 
         /// <summary>
         /// Capture the screen using CameraImage.AcquireCameraImageBytes.
         /// </summary>
         /// <param name="callback"></param>
         void CaptureScreenAsync(Action<byte[], int, int> callback)
         {
             Task.Run(() =>
             {
                 byte[] imageByteArray = null;
                 int width;
                 int height;
 
                 using (var imageBytes = Frame.CameraImage.AcquireCameraImageBytes())
                 {
                     if (!imageBytes.IsAvailable)
                     {
                         callback(null, 0, 0);
                         return;
                     }
 
                     int bufferSize = imageBytes.YRowStride * imageBytes.Height;
 
                     imageByteArray = new byte[bufferSize];
 
                     Marshal.Copy(imageBytes.Y, imageByteArray, 0, bufferSize);
 
                     width = imageBytes.Width;
                     height = imageBytes.Height;
                 }
 
                 callback(imageByteArray, width, height);
             });
         }        
     }
 }

ARCore for Unity save camera image

1 Reply

Your answer

Follow this Question

Related Questions