Wayback Machinekoobas.hobune.stream
May JUN Jul
Previous capture 14 Next capture
2021 2022 2023
2 captures
13 Jun 22 - 14 Jun 22
sparklines
Close Help
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
  • Asset Store
  • Get Unity

UNITY ACCOUNT

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account
  • Blog
  • Forums
  • Answers
  • Evangelists
  • User Groups
  • Beta Program
  • Advisory Panel

Navigation

  • Home
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
    • Blog
    • Forums
    • Answers
    • Evangelists
    • User Groups
    • Beta Program
    • Advisory Panel

Unity account

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account

Language

  • Chinese
  • Spanish
  • Japanese
  • Korean
  • Portuguese
  • Ask a question
  • Spaces
    • Default
    • Help Room
    • META
    • Moderators
    • Topics
    • Questions
    • Users
    • Badges
  • Home /
avatar image
0
Question by lireas · Dec 06, 2021 at 04:34 PM · networkinghttpwebrequest

Downloading multiple textures with a single http request (UnityWebRequest)

Getting a one single texture is not a problem. i've figured out how to do that. But i need a few thousands for which i dynamically generate url..

The problem is that by making an http request for each one, after maybe 50 of them load i start getting http error 429 "too many requests".

How should i deal with this? I can't find in unity documentation anything for downloading multiple files with a single request. (maybe i just don't understand it)

Comment
Add comment
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

2 Replies

· Add your reply
  • Sort: 
avatar image
0

Answer by fffMalzbier · Dec 06, 2021 at 05:37 PM

Unity Does not provide a method for Requesting and downloading Multiple files with one Request. You can control how many request you do have running at the same time in order not to overload the server.

If you control the server another option is the Request a list of images via a small PHP script that then can package the requested images into a container like a zip file , then unity gets back the URL for the container , downloads it , extracts the images and the load it via multiple webreqeuest from disk.

Comment
Add comment · Show 1 · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image lireas · Dec 06, 2021 at 06:14 PM 0
Share

I unfortunately don't control the server. I guess the best solution would then be to divide the map in chunks and load images for each chunk individually, am i right?

avatar image
0

Answer by Bunny83 · Dec 07, 2021 at 01:01 AM

You can use a load balancer like this one which I've quickly thrown:together:

 using System.Collections;
 using System.Collections.Generic;
 using UnityEngine;
 using UnityEngine.Networking;
 
 public class LoadBalancer<T>
 {
     public class Request
     {
         public UnityWebRequest request;
         public UnityWebRequestAsyncOperation response = null;
         public T data;
         public System.Action<UnityWebRequestAsyncOperation, T> callback;
         public Request(UnityWebRequest aRequest, T aData,  System.Action<UnityWebRequestAsyncOperation, T> aCallback)
         {
             request = aRequest;
             callback = aCallback;
             data = aData;
         }
     }
     private Queue<Request> m_Queue = new Queue<Request>();
     private List<Request> m_ActiveList = new List<Request>();
     private MonoBehaviour m_Host;
     private Coroutine m_Coroutine = null;
     private int m_MaxTasks;
     public bool IgnoreMissingCallback = false;
     public System.Action<UnityWebRequestAsyncOperation, T> defaultCallback;
     public LoadBalancer(MonoBehaviour aHost, int aMaxTasks, System.Action<UnityWebRequestAsyncOperation, T> aDefaultCallback = null)
     {
         m_Host = aHost;
         m_MaxTasks = aMaxTasks;
         defaultCallback = aDefaultCallback;
     }
 
     public void AddTask(UnityWebRequest aTask, T aData = default, System.Action<UnityWebRequestAsyncOperation, T> aCallback = null)
     {
         if (aCallback == null)
         {
             aCallback = defaultCallback;
             if (!IgnoreMissingCallback && aCallback == null)
                 Debug.LogWarning("Added task without callback");
         }
         m_Queue.Enqueue(new Request(aTask, aData,  aCallback));
         if (m_Coroutine == null)
             m_Coroutine = m_Host.StartCoroutine(RunRequests());
     }
 
     public void AbortAllRequests()
     {
         m_Host.StopCoroutine(m_Coroutine);
         foreach (var request in m_ActiveList)
             request.request.Dispose();
         m_ActiveList.Clear();
         m_Queue.Clear();
     }
 
     private IEnumerator RunRequests()
     {
         while (m_Queue.Count > 0 || m_ActiveList.Count > 0)
         {
             while (m_ActiveList.Count < m_MaxTasks && m_Queue.Count > 0)
             {
                 var request = m_Queue.Dequeue();
                 request.response = request.request.SendWebRequest();
                 m_ActiveList.Add(request);
             }
             for(int i = 0; i < m_ActiveList.Count; i++)
             {
                 var request = m_ActiveList[i];
                 if (request.response.isDone)
                 {
                     m_ActiveList[i] = m_ActiveList[m_ActiveList.Count - 1];
                     m_ActiveList.RemoveAt(m_ActiveList.Count - 1);
                     if (request.callback != null)
                         request.callback(request.response, request.data);
                 }
             }
             yield return null;
         }
         m_Coroutine = null;
     }
 }

You can create an instance of this loader, specify the max number of parallel tasks and just add as many tasks you like. Each task can have its own callback method as well as a custom data instance. The callback receives the web response as well as the custom data instance. When no custom callback is provided, it will use the default callback specified at the load balancer itself.


Here's an example script that uses the load balancer:

 public class LoadImages : MonoBehaviour
 {
     LoadBalancer<GameObject> loader;
     int xPos = 0;
     void Start()
     {
         loader = new LoadBalancer<GameObject>(this, 10, HandleResponse);
         for (int i = 0; i < 100; i++)
         {
             loader.AddTask(LoadImage("https://upload.wikimedia.org/wikipedia/commons/thumb/c/c4/Unity_2021.svg/1200px-Unity_2021.svg.png"), CreateCube(xPos++));
             loader.AddTask(LoadImage("https://upload.wikimedia.org/wikipedia/commons/thumb/1/19/Unity_Technologies_logo.svg/800px-Unity_Technologies_logo.svg.png"), CreateCube(xPos++));
             loader.AddTask(LoadImage("https://i1.wp.com/joyofandroid.com/wp-content/uploads/2019/01/powered-by-unity1-1024x587.jpg?w=696&ssl=1"), CreateCube(xPos++));
         }
     }
     UnityWebRequest LoadImage(string aURL)
     {
         var request = UnityWebRequest.Get(aURL);
         request.downloadHandler = new DownloadHandlerTexture();
         return request;
     }
 
     GameObject CreateCube(int aXPos)
     {
         var cube = GameObject.CreatePrimitive(PrimitiveType.Cube);
         cube.transform.position = new Vector3(1.1f * aXPos, 0, 0);
         return cube;
     }
 
     void HandleResponse(UnityWebRequestAsyncOperation aResponse, GameObject aGO)
     {
         Debug.Log($"Completed: {aResponse.webRequest.url} data: {aResponse.webRequest.downloadedBytes} bytes");
         aGO.GetComponent<Renderer>().material.mainTexture = DownloadHandlerTexture.GetContent(aResponse.webRequest);
     }
 }

This will load the same 3 images 100 times and assigns them to cube objects. So the custom data object in this examples is simply a cube GameObject. However you could use whatever class you like. As you can see I simply use a single callback that will handle each request. Since the callback gets the request as well as the custom data object you know what request you're handling.


The LoadBalancer only runs a single coroutine which handles all the requests. The internal queue holds the queued tasks that has not yet been started and the active list holds the requests that are currently processed. The balancer would only schedule "MaxTasks" requests at the same time. Once a request has finished, if will load the next from the queue. The coroutine autostarts and terminates as necessary. The required MonoBehaviour "host" argument is used to actually run the coroutine.


pps: I'm wondering when you load thousands of images, how large are those images? Because loading that many textures you could easily run into memory issues.

Comment
Add comment · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Your answer

Hint: You can notify a user about this post by typing @username

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Follow this Question

Answers Answers and Comments

179 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

Unity networking tutorial? 6 Answers

UnityWebRequest timeout not containing 1 Answer

HTTP Requests removed on Android Pie (API 28+) 2 Answers

Is HttpWebRequest supported for desktop and editor? 2 Answers

why does UnityWebRequest want Dispose() ? 1 Answer


Enterprise
Social Q&A

Social
Subscribe on YouTube social-youtube Follow on LinkedIn social-linkedin Follow on Twitter social-twitter Follow on Facebook social-facebook Follow on Instagram social-instagram

Footer

  • Purchase
    • Products
    • Subscription
    • Asset Store
    • Unity Gear
    • Resellers
  • Education
    • Students
    • Educators
    • Certification
    • Learn
    • Center of Excellence
  • Download
    • Unity
    • Beta Program
  • Unity Labs
    • Labs
    • Publications
  • Resources
    • Learn platform
    • Community
    • Documentation
    • Unity QA
    • FAQ
    • Services Status
    • Connect
  • About Unity
    • About Us
    • Blog
    • Events
    • Careers
    • Contact
    • Press
    • Partners
    • Affiliates
    • Security
Copyright © 2020 Unity Technologies
  • Legal
  • Privacy Policy
  • Cookies
  • Do Not Sell My Personal Information
  • Cookies Settings
"Unity", Unity logos, and other Unity trademarks are trademarks or registered trademarks of Unity Technologies or its affiliates in the U.S. and elsewhere (more info here). Other names or brands are trademarks of their respective owners.
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Default
  • Help Room
  • META
  • Moderators
  • Explore
  • Topics
  • Questions
  • Users
  • Badges