Wayback Machinekoobas.hobune.stream
May JUN Jul
Previous capture 14 Next capture
2021 2022 2023
2 captures
12 Jun 22 - 14 Jun 22
sparklines
Close Help
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
  • Asset Store
  • Get Unity

UNITY ACCOUNT

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account
  • Blog
  • Forums
  • Answers
  • Evangelists
  • User Groups
  • Beta Program
  • Advisory Panel

Navigation

  • Home
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
    • Blog
    • Forums
    • Answers
    • Evangelists
    • User Groups
    • Beta Program
    • Advisory Panel

Unity account

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account

Language

  • Chinese
  • Spanish
  • Japanese
  • Korean
  • Portuguese
  • Ask a question
  • Spaces
    • Default
    • Help Room
    • META
    • Moderators
    • Topics
    • Questions
    • Users
    • Badges
  • Home /
avatar image
1
Question by RetroGeek46 · Jul 09, 2018 at 03:37 PM · androidpluginnative pluginwebcamtexturenative

Pass camera stream from unity and android plugin

I'm trying to use Firebase's MLKit with unity, trying to use the face detection in my project. While sending a single image works, there is a high ping associated with the plugin call from unity and the time by which the callback actually returns the value. Is there any way to achieve a low latency solution for this, so that a reliable connection can be made?

Unity Code

   public void AndroidRender() {
         tex = new Texture2D(webcamTexture.width, webcamTexture.height);
         textureHolder.GetComponent<RawImage>().texture = tex;
         tex.SetPixels32(webcamTexture.GetPixels32());
         tex.Apply();
     
         byteObject.Call("passByteArray", colorBytes);
         coordinates = byteObject.Call<string>("getData");
     }

Java code

 public passByteArray(byte[] byteArray) {
             if(gotData) {
                 return;     //create new values only if previous one's have been consumed
             }
             //create bitmap
             BitmapFactory.Options bitmapOptions = new BitmapFactory.Options();
             bitmapOptions.inMutable = true;
             Bitmap bmp = BitmapFactory.decodeByteArray(byteArray, 0, byteArray.length, bitmapOptions);
             bmp = rotateBitmap(bmp, 90);
     
             FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(bmp);
             //start detection
             FirebaseVisionFaceDetector detector = FirebaseVision.getInstance()
                     .getVisionFaceDetector(options);
             Task<List<FirebaseVisionFace>> result =
                     detector.detectInImage(image)
                             .addOnSuccessListener(
                                     new OnSuccessListener<List<FirebaseVisionFace>>() {
                                         @Override
                                         public void onSuccess(List<FirebaseVisionFace> faces) {
                                             Log.i(TAG, "detecting faces");
                                             for (FirebaseVisionFace face : faces) {
     
                                                 nose = face.getLandmark(6);
                                                 leftEye = face.getLandmark(4);
                                                 rightEye = face.getLandmark(10);
                                                 if (nose != null) {
                                                     FirebaseVisionPoint point = nose.getPosition();
                                                     x_nose = point.getX();
                                                     y_nose = point.getY();
                                                 }
                                                 if (leftEye != null) {
                                                     FirebaseVisionPoint point = leftEye.getPosition();
                                                     x_lEye = point.getX();
                                                     y_rEye = point.getY();
                                                 }
                                                 if (rightEye != null) {
                                                     FirebaseVisionPoint point = rightEye.getPosition();
                                                     x_rEye = point.getX();
                                                     y_rEye = point.getY();
                                                 }
                                             }
                                             if(faces.size() == 0) {
                                                 coordinates = "0.0-0.0-0.0-0.0-0.0-0.0";
                                             } else {
                                                 coordinates = "" + x_nose + "-" + y_nose + "-" + x_lEye + "-" + y_lEye + "-" +  x_rEye + "-" + y_rEye;          //actual value to be returned
                                             }
                                             gotData = true;
                                             //Log.i(TAG,faces.size() + " -> " + x + "-" + y);
                                         }
                                     })
                             .addOnFailureListener(
                                     new OnFailureListener() {
                                         @Override
                                         public void onFailure(@NonNull Exception e) {
                                             Log.i(TAG, "some error");
                                         }
                                     });
         }

The boolean gotData at the start of Java plugin is used to wait till the callback actually returns value before passing back to Unity

Comment
Add comment · Show 1
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image zippopalermo · Aug 03, 2018 at 08:10 AM 0
Share

Did you have any luck solving this by any chance? I'm running into the same problem...

1 Reply

· Add your reply
  • Sort: 
avatar image
0

Answer by tony_jadeja · Dec 15, 2021 at 09:36 AM

@RetroGeek46 I am trying to do the same thing but I am not able to covert WebCamTexture to byte[] to send in Java function can you please help me to resolve this?

Comment
Add comment · Show 1 · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image RetroGeek46 · Dec 15, 2021 at 09:45 AM 0
Share

Hi, I think I did manage to make it work somehow. Will have to check old code as I worked on it 3 years ago. If would be great if you could ping me on discord, retrogeek46#4047

Your answer

Hint: You can notify a user about this post by typing @username

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Follow this Question

Answers Answers and Comments

200 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

How to access an Android native plugin in unity ? 0 Answers

Out of ideas on how to make this plugin work again, please HELP! 1 Answer

Access photos, videos & music from iOS and Android library? 0 Answers

Android Native Plugin (*.SO) 2 Answers

android time cheat plugin problem with broadcastreceiver 1 Answer


Enterprise
Social Q&A

Social
Subscribe on YouTube social-youtube Follow on LinkedIn social-linkedin Follow on Twitter social-twitter Follow on Facebook social-facebook Follow on Instagram social-instagram

Footer

  • Purchase
    • Products
    • Subscription
    • Asset Store
    • Unity Gear
    • Resellers
  • Education
    • Students
    • Educators
    • Certification
    • Learn
    • Center of Excellence
  • Download
    • Unity
    • Beta Program
  • Unity Labs
    • Labs
    • Publications
  • Resources
    • Learn platform
    • Community
    • Documentation
    • Unity QA
    • FAQ
    • Services Status
    • Connect
  • About Unity
    • About Us
    • Blog
    • Events
    • Careers
    • Contact
    • Press
    • Partners
    • Affiliates
    • Security
Copyright © 2020 Unity Technologies
  • Legal
  • Privacy Policy
  • Cookies
  • Do Not Sell My Personal Information
  • Cookies Settings
"Unity", Unity logos, and other Unity trademarks are trademarks or registered trademarks of Unity Technologies or its affiliates in the U.S. and elsewhere (more info here). Other names or brands are trademarks of their respective owners.
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Default
  • Help Room
  • META
  • Moderators
  • Explore
  • Topics
  • Questions
  • Users
  • Badges