Wayback Machinekoobas.hobune.stream
May JUN Jul
Previous capture 12 Next capture
2021 2022 2023
1 capture
12 Jun 22 - 12 Jun 22
sparklines
Close Help
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
  • Asset Store
  • Get Unity

UNITY ACCOUNT

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account
  • Blog
  • Forums
  • Answers
  • Evangelists
  • User Groups
  • Beta Program
  • Advisory Panel

Navigation

  • Home
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
    • Blog
    • Forums
    • Answers
    • Evangelists
    • User Groups
    • Beta Program
    • Advisory Panel

Unity account

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account

Language

  • Chinese
  • Spanish
  • Japanese
  • Korean
  • Portuguese
  • Ask a question
  • Spaces
    • Default
    • Help Room
    • META
    • Moderators
    • Topics
    • Questions
    • Users
    • Badges
  • Home /
avatar image
1
Question by TheShadyColombian · Oct 20, 2015 at 08:34 PM · wwwurlwww class

WWW not getting site that doesn't end on .html

I am using WWW to download a web page. The problem is that right now, the website URL doesn't end in .html The URL is https://shopifycounter.monstercat.com/counter/realtime

Comment
Add comment
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

1 Reply

· Add your reply
  • Sort: 
avatar image
2
Best Answer

Answer by Bunny83 · Oct 20, 2015 at 10:30 PM

Well, this is an endless event stream which uses a custom format. It doesn't represent a normal webpage. You can't use the WWW class here. WWW expects the stream to end and will only complete when when the stream finished. This stream is never closed. It periodically sends a "heartbeat" event to keep the connection open. You have to use System.Net classes instead.

For example with such an helper class:

 using UnityEngine;
 using System.Collections.Generic;
 using System.Net;
 
 public class WebStreamReader : System.IDisposable
 {
     volatile bool running = false;
     string url = "";
     System.Threading.Thread thread = null;
     Queue<string> buffer = new Queue<string>();
     object lockHandle = new object();
 
     ~WebStreamReader()
     {
         Dispose();
     }
 
     public void Start(string aURL)
     {
         if (!running)
         {
             url = aURL;
             thread = new System.Threading.Thread(Run);
             thread.Start();
         }
     }
     private void Run()
     {
         running = true;
         ServicePointManager.ServerCertificateValidationCallback = (a, b, c, d) => { return true; };
         var request = System.Net.HttpWebRequest.Create(url);
         var response = request.GetResponse();
         var stream = response.GetResponseStream();
         byte[] data = new byte[2048];
         while (running)
         {
             int count = stream.Read(data, 0, 2048);
             if (count > 0)
             {
                 lock(lockHandle)
                 {
                     string message = System.Text.UTF8Encoding.UTF8.GetString(data, 0, count);
                     buffer.Enqueue(message);
                 }
             }
         }
     }
 
     public string GetNextBlock()
     {
         string tmp = "";
         lock(lockHandle)
         {
             if (buffer.Count > 0)
             {
                 tmp = buffer.Dequeue();
             }
         }
         return tmp;
     }
 
     public void Dispose()
     {
         running = false;
         thread.Abort();
     }
 }

You can read the stream data using a coroutine like this:

 public class ReadStream : MonoBehaviour
 {
     WebStreamReader request = null;
 
     void Start()
     {
         StartCoroutine(WRequest());
     }
 
     IEnumerator WRequest()
     {
         request = new WebStreamReader();
         request.Start("http://shopifycounter.monstercat.com/counter/realtime");
         string stream = "";
         while (true)
         {
             string block = request.GetNextBlock();
             if (!string.IsNullOrEmpty(block))
             {
                 stream += block;
                 string[] data = stream.Split(new string[] { "\n\n" }, System.StringSplitOptions.None);
                 stream = data[data.Length - 1];
                 for (int i = 0; i < data.Length - 1; i++)
                 {
                     if (!string.IsNullOrEmpty(data[i]))
                     {
                         string msg = data[i].TrimStart('\n');
                         int index = msg.IndexOf("dollarAmount");
                         if (index > 0)
                         {
                             msg = msg.Substring(index + 14);
                             int index2 = msg.IndexOf('}');
                             msg = msg.Substring(0, index2);
                             decimal amount = decimal.Parse(msg);
                             OnDataUpdate(amount);
                         }
                     }
                 }
             }
             yield return new WaitForSeconds(1);
         }
     }
 
     void OnApplicationQuit()
     {
         if (request != null)
             request.Dispose();
     }
 
     void OnDataUpdate(decimal aAmount)
     {
         Debug.Log("Received new amount: " + aAmount);
         // Do whatever you want with the value
     }
 }

Each message seems to be in the format:

 event: EVENTTYPE
 data: DATA

The only two events i've seen is heartbeat and update. The heartbeat event has just "1" as data while update has a JSON string which looks like this:

 {"sales":3502,"dollarAmount":17341.71}

edit
I added the method "OnDataUpdate" to seperate the whole parsing from the point where you can use the value. Since you only want the dollar amount value, that's what this method get passed as parameter everytime when a new update arrives, I used the "decimal" type since we seem to deal with currency here. The decimal type eliminates most errors that comes from a floating point representation of a number.

Note: It's extremely important that you call Dispose on the WebStreamReader object to stop the thread it started. If you don't do that the thread will keep running, even when you stop playmode in the editor. This will not only add a new thread each time you press play, but might also make that website refuse your new connection since the old one is still alive.

Comment
Add comment · Show 4 · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image TheShadyColombian · Oct 20, 2015 at 11:51 PM 0
Share

Is there a more simple way? I just need the Dollar amount, that's it.

avatar image Bunny83 TheShadyColombian · Oct 21, 2015 at 02:52 AM 0
Share

Uhm, no. You can't just "load" a site here since it's an endless stream. Even the WWW class uses a thread behind the scenes. You can't avoid it when dealing with network connections. Since it's a stream it's possible that only a part of a message arrives and the other part arrives later. That's why most "protocols" which are embedded in an endless stream have some sort of "marker" to indicate when a new message begins. In this case it's simply two line breaks in a row. $$anonymous$$y example already gives you the stitched and cropped raw message. It also takes care of synchronising the threads.

About the parsing of the message that's up to you. If the data always just looks like this you can of course manually parse the number out of the string. I can add this to my example as well. I'll edit my answer.

avatar image TheShadyColombian Bunny83 · Oct 21, 2015 at 03:15 AM 0
Share

Cool! Thank you so much!

avatar image EdgarasArt · Apr 22, 2016 at 01:58 PM 0
Share

This was really useful! Thanks! I've searched the internet for about two weeks for such solution.

Your answer

Hint: You can notify a user about this post by typing @username

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Follow this Question

Answers Answers and Comments

33 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

UriFormatException: Invalid URI: Invalid port number for VirtualBox server using IPv6 Address 1 Answer

C# Unity function to escape URL querystring parameters 2 Answers

How to use POST service using www class? 0 Answers

Help searching google! 1 Answer

How to encode german umlaute when laoding local stored data via WWW class? 0 Answers


Enterprise
Social Q&A

Social
Subscribe on YouTube social-youtube Follow on LinkedIn social-linkedin Follow on Twitter social-twitter Follow on Facebook social-facebook Follow on Instagram social-instagram

Footer

  • Purchase
    • Products
    • Subscription
    • Asset Store
    • Unity Gear
    • Resellers
  • Education
    • Students
    • Educators
    • Certification
    • Learn
    • Center of Excellence
  • Download
    • Unity
    • Beta Program
  • Unity Labs
    • Labs
    • Publications
  • Resources
    • Learn platform
    • Community
    • Documentation
    • Unity QA
    • FAQ
    • Services Status
    • Connect
  • About Unity
    • About Us
    • Blog
    • Events
    • Careers
    • Contact
    • Press
    • Partners
    • Affiliates
    • Security
Copyright © 2020 Unity Technologies
  • Legal
  • Privacy Policy
  • Cookies
  • Do Not Sell My Personal Information
  • Cookies Settings
"Unity", Unity logos, and other Unity trademarks are trademarks or registered trademarks of Unity Technologies or its affiliates in the U.S. and elsewhere (more info here). Other names or brands are trademarks of their respective owners.
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Default
  • Help Room
  • META
  • Moderators
  • Explore
  • Topics
  • Questions
  • Users
  • Badges