- Home /
Get redirection URL from UnityWebRequest
I am issuing a UnityWebRequest with a URL that has redirection involved. The redirection is being logged as a warning by the compiler. Is there any way of accessing that redirection URL?
I've tried the method of using ResponseHeaders["Location"]. That does not work as the Response Headers don't have such a field.
Can someone help me log that warning URL? I don't want to write a dump of the console into a file and parse it from there. Is there a cleaner approach?
TIA
Answer by Bunny83 · Apr 09, 2017 at 03:02 AM
Unfortunately no. The WWW class as well as the UnityWebRequest class follow a redirection response transparently. So they don't provide any information about that redirect. The UnityWebRequest has a redirectLimit, however it doesn't provide informations of the redirection but only generates an error if the redirection amount exceeds that limit.
I never was in need of getting the redirection URL manually. However if the URL is dumped as warning you could intercept it by using a log-callback.
A cleaner approach would be to use a more low-level API. The .NET / Mono WebRequest class does allow to disable the auto-redirect. That way you get the actual response and you have to follow the new URL manually.
You're right, it is also possible to do it with WebRequest (and it is simpler to write, thank you ^^)
However, I still have my certificate validation problem with it, and here, I have to desactive safe validation globally, not just for the request. I tried to find a better custom validator (see the referenced answers.unity3d.com question for details), but I'm still not sure that this solution can be considered safe.
Anyway, even if unsecure, it "works" this way too :) :
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System;
using System.Net;
using System.Net.Security;
using System.IO;
using System.Security.Cryptography.X509Certificates;
public class HttpWebRequestTest : $$anonymous$$onoBehaviour {
// Use this for initialization
void Start () {
StartCoroutine(SSLPostRequest ("<login>", "<password>"));
}
// Update is called once per frame
void Update () {
}
// Source: http://answers.unity3d.com/questions/50013/httpwebrequestgetrequeststream-https-certificate-e.html
public bool CustomCertificateValidationCallback(System.Object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors) {
bool isOk = true;
// If there are errors in the certificate chain, look at each error to deter$$anonymous$$e the cause.
if (sslPolicyErrors != SslPolicyErrors.None) {
for (int i=0; i<chain.ChainStatus.Length; i++) {
if (chain.ChainStatus [i].Status != X509ChainStatusFlags.RevocationStatus$$anonymous$$) {
chain.ChainPolicy.RevocationFlag = X509RevocationFlag.EntireChain;
chain.ChainPolicy.Revocation$$anonymous$$ode = X509Revocation$$anonymous$$ode.Online;
chain.ChainPolicy.UrlRetrievalTimeout = new TimeSpan (0, 1, 0);
chain.ChainPolicy.VerificationFlags = X509VerificationFlags.AllFlags;
bool chainIsValid = chain.Build ((X509Certificate2)certificate);
if (!chainIsValid) {
isOk = false;
}
}
}
}
return isOk;
}
IEnumerator SSLPostRequest(string username, string password){
RemoteCertificateValidationCallback defaultCertificateValidationCallback = ServicePoint$$anonymous$$anager.ServerCertificateValidationCallback;
ServicePoint$$anonymous$$anager.ServerCertificateValidationCallback = CustomCertificateValidationCallback;
string body = "username=" + WWW.EscapeURL(username) + "&password=" + WWW.EscapeURL(password) + "&login=1";
string host = "example.com";
string authorizationQuery = "/authentificationWithRedirect.html";
var request = (HttpWebRequest)WebRequest.Create("https://"+host+authorizationQuery);
request.AllowAutoRedirect = false;
var data = System.Text.Encoding.UTF8.GetBytes(body);
request.$$anonymous$$ethod = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = data.Length;
using (var stream = request.GetRequestStream())
{
Debug.Log ("Sending:\n" + data);
stream.Write(data, 0, data.Length);
}
var response = (HttpWebResponse)request.GetResponse();
Debug.Log ("Location header: " + response.GetResponseHeader("Location"));
var responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
Debug.Log ("response: "+ responseString);
ServicePoint$$anonymous$$anager.ServerCertificateValidationCallback = defaultCertificateValidationCallback;
yield return null;
}
}
Certificates should be validated automatically. However if you have an unregistrated certificate you have to do the validation yourself. $$anonymous$$eep in $$anonymous$$d that SSL build on top of the chain of trust. If you use a certificate that is not registrated at a certificate authority a client can never validate the certificate "securely". This is the case when a webbrowser requires you to add a manual exception to simply accept the certificate locally. You could ship your public certificate with your build so you may be able to simply compare them.
ps: you did not URL encode your username and password in your post data. This can cause problems when those contain restricted characters (like "=", "%", "&", ...)
Thanks, you're right about escaping the params ! I updated my snippets to add the escaping
Concerning the certificate of the API I'm connecting too, it seems valid (based on Let's Encrypt). I think that the certificate store is not loaded when using TcpClient or HttpWebRequest: I should find a way to manually trigger the store loading, but I've not yet found it.
Anyway, thanks for the advices :)
Thanks, helped me fixing an error loading a vimeo / akamaized.net HTTP live strea$$anonymous$$g link
When I was trying to load the url using Unity web libraries (ObservableWebRequest, UnityWebRequest) and even with the native c# libraries (HttpClient, WebClient) the libraries were trying to follow the redirect link but they were corrupting it probably by URLDecoding and then URLEncoding some parameters. I noticed that those libraries are converting "%28" into "(" , "%29" into ")" , "%2A" in "*" and possibly others. Then the libraries were trying to load the corrupt url and get a forbidden error. I only needed a fraction of the code:
using System.Net;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(httpLiveStrea$$anonymous$$gLink);
request.Credentials = CredentialCache.DefaultCredentials;
request.AllowAutoRedirect = false; //don't continue to load the final URL, just get me the redirect link
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
redirectURL = response.GetResponseHeader("Location");
response.Close();
Answer by gizmhail · Apr 09, 2017 at 05:35 AM
I also tried with UnityWebRequest and WWW to find the Location header in the responses, and I didn't find a proper way to do so.
So, I ended using directly TcpClient, and it kinda works. Remark: it looks like my implementation is not very efficient/stable, as you'll notice that the request take an usual time and sometimes fail. I'll investigate it later.
If you simply want to reach an http server, with a GET request, it is pretty straighforward.
If you want to POST a form, you have to build the request body.
And if you want to use an https connection, it becomes a bit tricky :
you have to use an SslStream
as certificates don't seem to be properly loaded, you have to handle their manipulation manually
I still haven't found how to do it securely, so for the moment, I completely disable the certificate check. Please keep in mind that it is not safe at all.
So, first, for the simple case (an http GET request, whose we want to find the redirection) :
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.Net.Sockets;
using System.IO;
public class TCPTest : MonoBehaviour {
public string redirection = null;
public string result = null;
public bool connected = false;
// Use this for initialization
void Start () {
fetchGetHttpRedirection ();
}
// Update is called once per frame
void Update () {
if (!this.connected) {
if (this.redirection != null) {
Debug.Log ("Redirection:\n" + this.redirection);
this.redirection = null;
} else if(this.result != null) {
Debug.Log ("Answer:\n" + this.result);
this.result = null;
}
}
}
void fetchGetHttpRedirection(){
StartCoroutine (GetTCPClientRedirect ("example.com","/"));
}
IEnumerator GetTCPClientRedirect(string host, string query){
TcpClient client = new TcpClient(host, 80);
NetworkStream networkStream = client.GetStream();
StreamReader reader = new StreamReader(networkStream);
StreamWriter writer = new StreamWriter(networkStream);
// Basic GET request:
// GET /foo.html HTTP/1.1\r\nHost: example.com\r\n
writer.WriteLine("GET "+ query + " HTTP/1.1");
writer.WriteLine("Host: " + host);
writer.WriteLine("");
writer.Flush ();
this.redirection = null; // variable where we'll store the redirection
string lastLine = null;
this.result = "";
this.connected = true;
while (true) {
if (!client.Connected) {
break;
}
if (networkStream.CanRead) {
string line = reader.ReadLine ();
if (line != null) {
string redirectionHeader = "Location: ";
if(line.StartsWith(redirectionHeader)){
this.redirection = line.Substring (redirectionHeader.Length, line.Length - redirectionHeader.Length);
}
if (this.result != "") {
this.result = this. result + "\n";
}
if (lastLine == "" && line == "0") {
break;
} else {
lastLine = line;
this.result = this.result + line;
}
} else {
break;
}
}
// To avoid blocking the thread
yield return null;
}
// Disconnection
networkStream.Close();
client.Close();
this.connected = false;
yield return null;
}
}
Then, an example with both a POST and the (unsecure) https handling :
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.Net.Sockets;
using System.IO;
using UnityEngine.Networking;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;
using System.Security.Authentication;
public class TCPPostSSLTest:MonoBehaviour
{
SampleAuthentificationRequest request = null;
void Start () {
string host = "example.com";
string authorizationQuery = "/foo/login.html";
request = new SampleAuthentificationRequest (host, authorizationQuery);
StartCoroutine(request.Launch ("<login>", "<password>"));
}
// Update is called once per frame
void Update () {
if (!this.request.connected) {
if (this.request.redirection != null) {
Debug.Log ("Redirection:\n" + this.request.redirection);
this.request.redirection = null;
} else if(this.request.result != null) {
Debug.Log ("Answer:\n" + this.request.result);
this.request.result = null;
}
}
}
public class SampleAuthentificationRequest
{
public string result;
public string redirection = null;
public bool connected = false;
string host;
string authorizationQuery;
public SampleAuthentificationRequest(string host, string authorizationQuery){
this.host = host;
this.authorizationQuery = authorizationQuery;
}
public static bool CertificateValidationCallback(
object sender,
X509Certificate certificate,
X509Chain chain,
SslPolicyErrors sslPolicyErrors)
{
Debug.Log ("[Warning] Using unsafe certification: always accepting server certificats !");
return true;
}
public IEnumerator Launch(string username, string password) {
string body = "username=" + WWW.EscapeURL(username) + "&password=" + WWW.EscapeURL(password) + "&login=1";
int contentLength = System.Text.Encoding.UTF8.GetBytes(body).Length;
TcpClient client = new TcpClient(this.host, 443);
NetworkStream networkStream = client.GetStream();
SslStream sslStream = new SslStream(networkStream
,false
,new RemoteCertificateValidationCallback(CertificateValidationCallback)
);
// Debug.Log("Authenticating...");
sslStream.AuthenticateAsClient (host);
// Debug.Log("Authent done");
while (!sslStream.IsAuthenticated) {
// Debug.Log("Not yet authenticated...");
yield return null;
}
this.connected = true;
StreamReader reader = new StreamReader(sslStream);
StreamWriter writer = new StreamWriter(sslStream);
string requestData = "POST " + this.authorizationQuery + " HTTP/1.1\r\n";
requestData = requestData + "Host: " + this.host + "\r\n";
requestData = requestData + "User-Agent: UniNoco\r\n";
requestData = requestData + "Accept: */*\r\n";
requestData = requestData + "Content-Length: " + contentLength + "\r\n";
requestData = requestData + "Content-Type: application/x-www-form-urlencoded\r\n";
requestData = requestData + "\r\n";
requestData = requestData + body + "\r\n";
requestData = requestData + "\r\n";
requestData = requestData + "\r\n";
writer.Write (requestData);
writer.Flush ();
// Debug.Log ("Request:");
Debug.Log(requestData);
result = "";
string lastLine = null;
while (true) {
if (!client.Connected) {
// Debug.Log ("Disconnected");
break;
}
if (sslStream.CanRead) {
string line = reader.ReadLine ();
if (line != null) {
string redirectionHeader = "Location: ";
if(line.StartsWith(redirectionHeader)){
this.redirection = line.Substring (redirectionHeader.Length, line.Length - redirectionHeader.Length);
}
if (result != "") {
result = result + "\n";
}
if (lastLine == "" && line == "0") {
break;
}
lastLine = line;
// Debug.Log ("Received: >" + line + "<");
result = result + line;
} else {
break;
}
}
// Debug.Log ("No data ...");
yield return null;
}
// Debug.Log("Closing...");
sslStream.Close();
client.Close();
// Debug.Log("Closed.");
this.connected = false;
yield return null;
}
}
}
Your answer
Follow this Question
Related Questions
Super slow network speed in editor with some routers 0 Answers
Set Content-Length header for UnityWebRequest POST requests in 2017.3? 1 Answer
Webrequest put bodyData? 1 Answer
Get Contents of Folder Using UnityWebRequest 0 Answers
How to get UnityWebRequest data early when using php flush() 1 Answer