- Home /
What's the best way to implement file compression?
Edit: Clarification after answer: I chose Lucas's answer as the correct answer for this question. DotNetZip, when implemented with the I18N.dlls works like a champ in the Editor and built .exes. My specific implementation details are located in the "Potential Answer: DotNetZip" answer below.*
Edit 1/30/2011: Just a quick note to say that I have been using the DotNetZip solution in 3.x without any issues.
Using the System.IO.Compression.GZipStream classes results in the dreaded DllNotFoundException: MonoPosixHelper exception (http://answers.unity3d.com/questions/3147/dllnotfoundexception-monoposixhelpersystem-io-compression-deflatestream), so until Unity 3.0 comes out and the later version of Mono it ships with supports a fully managed compression scheme, we have to rely on a third party compression library.
There seem to be two libraries that might fit the bill. SharpZipLib (http://www.icsharpcode.net/opensource/sharpziplib/) is a venerable tool for zip compression, but the API is painful to use.
Some have reported that DotNetZip (http://dotnetzip.codeplex.com) works with Unity. I keep receiving an "IBM437 codepage not supported" (or similar) error when trying to work with this library. It is a much better library than SharpZipLib from a usability perspective.
There is a mono compiled version of SharpZipLib that ships with Unity (as discussed in this forum post http://forum.unity3d.com/viewtopic.php?t=10699&highlight=sharpziplib), which seems like a possibility. However, I am hesitant to use an older version of the library.
Should we just wait and hope? Can someone confirm if later versions of Mono will support this out of the box? Or will we have to roll our own in the 3.0 time frame, too?
Has anyone tried using this for iOS? I'm concerned firstly whether it will work and secondly about speed!
This is an old post but no new replies. Do anyone have thoughts for Unity 2017+ and also that would work well on iOS and Android?
Thanks.
Answer by Lucas Meijer 1 · May 22, 2010 at 09:59 PM
Nobody said that Unity3 will ship with a mono that has a fully managed GZipStream implementation. In fact, being the guy that is working on this currently, I can tell you that that would be extremely unlikely. So yes, also for the Unity3.0 timeframe, you'll have to rely on a managed implementation.
I went and tried the 3rd party library you are having trouble with: DotNetZip. I downloaded their distribution package, and grabbed Ionic.Zip.dll from it, and placed it into the Assets folder of an empty Unity2.6.1 project.
I then wrote a quick testscript:
using UnityEngine; using Ionic.Zip;
public class NewBehaviourScript : MonoBehaviour { // Use this for initialization void Start () { using (ZipFile zip = new ZipFile()) { // add this map file into the "images" directory in the zip archive zip.AddFile("c:\\mono.patch", "images"); // add the report into a different directory in the archive zip.Save("MyZipFile.zip"); } } }
(copied almost verbatim from their website). c:\mono.patch is a file that happened to live in my c:\ drive. When I ran this script, it succesfully wrote MyZipFile.zip.
Naturally the .AddFile() won't work from a webplayer, but this library has entrypoints for creating new files in the zipfile from a stream as well.
If this does not work for you, please describe the exact steps you're taking, and the exact error you get.
I wish it were this easy, Lucas. But the behavior when reading from a zipfile differs between running in the editor and running a built exe. When run in the editor, unity is able to retrieve data from the zipfile, but when run in a built binary the following exception is generated:
ArgumentException: Encoding name 'IB$$anonymous$$437' not supported
Parameter name: name
Demeter.DataArchive.GetItemFromArchive[StringLibrary] (System.String folder, System.String fileName) [0x00000] Demeter.DataArchive.GetStringLibrary (System.String filename, StringLibraryFields language) < snip />
Try grabbing the I18N*.dll assemblies from the mono framework that unity ships, and put those next to "mscorlib.dll" in the built exe. Good chance that would make it work.
I can confirm that the solution proposed by Lucas fixes this problem (tested in Unity 3.0 b7, but presumably works in older versions, too).
Though, actually, all you have to do is drop them into Assets/Plugins in your Unity project to make this work. On windows, the source files needed were in C:\Program Files (x86)\Unity\Editor\Data\$$anonymous$$ono\lib\mono\unity\
The mdb files are NOT needed, you can skip those. Using DotNetZip with these extra dll files included seems to be the best solution all around by far.
Answer by Zyxil · May 25, 2010 at 11:35 PM
The Answer: DotNetZip
I know it's bad juju to use an answer to post a comment, but I need to show some detail. Please accept my apologies.
Here are the implementation details that were answered by Lucas and confirmed by x4000:
Using DotNetZip version 1.9, place the Ionic.Zip.reduced.dll in your Assets folder.
Then, from your (Windows) C:\Program Files (x86)\Unity\Editor\Data\Mono\lib\mono\unity folder, copy the I18N*.dll files to your Assets folder. There should be 6. These files will prevent the IBM487 errors.
My methods for reading and writing to data files are:
/// <summary> /// Puts an item in the archive, overwrites if already exists /// </summary> /// <typeparam name="T"></typeparam> /// <param name="folder"></param> /// <param name="fileName"></param> /// <param name="item"></param> public void AddItemToArchive<T>(string folder, string fileName, ref T item) { if (!item.GetType().IsSerializable) throw new ArgumentException("item must be serializable");
using (ZipFile zipFile = new ZipFile())
{
zipFile.UseUnicodeAsNecessary = true;
//serialize item to memorystream
using (MemoryStream m = new MemoryStream())
{
IFormatter formatter = new BinaryFormatter();
formatter.Serialize(m, item);
m.Position = 0;
ZipEntry zipEntry = zipFile.AddEntry("entry", m);
zipEntry.UseUnicodeAsNecessary = true;
zipEntry.Password = _password;
zipFile.Save(filePath);
}
}
}
/// <summary>
/// Retrieves an item from the archive
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="folder"></param>
/// <param name="fileName"></param>
/// <returns></returns>
public T GetItemFromArchive<T>(string folder, string fileName)
{
//get the stream from the archive
using (MemoryStream m = new MemoryStream())
{
using (ZipFile zipFile = new ZipFile(filePath))
{
zipFile.UseUnicodeAsNecessary = true;
ZipEntry e = zipFile["entry"];
e.UseUnicodeAsNecessary = true;
e.Password = _password;
e.Extract(m);
m.Position = 0;
//now serialize it back to the correct type
IFormatter formatter = new BinaryFormatter();
T item = (T)formatter.Deserialize(m);
return item;
}
}
}
I hope this helps. ;o)
Just FYI, you can edit your original answer if you need to clarify anything. ;)
Did not work with iOS, I'm still getting IB$$anonymous$$437 errors. :(
Answer by Tray · Jun 08, 2010 at 04:30 AM
Have you tried looking at 7Zip? Has a completely managed C# library that I know works in Unity. It gives you compress/uncompress to and from memory streams/byte arrays, in LZMA format, so it's better than zip and actually what Unity uses to compress assets.
If you need something compatible with .zip format you might have some trouble.
holy macaroni, if you thought SharpZipLib was painful to use, you need to look at the 7zip libs. Fake CO$$anonymous$$ calls, zero documentation, cryptic code... Just terrible.
There are several wrappers for this on CodePlex, but they hide this confusion with extra dependencies. $$anonymous$$ostly dealing with the original problem with the 7zip dll's. 64/32 bit dependency issues, lack of implemented features (streams or encryption, etc).
Are there any samples for use in Unity?
We're using 7zip, a completely managed C# implementation, no CO$$anonymous$$, etc. I wrote a 20 line wrapper to handle streams so not a huge deal. I haven't looked at encryption for it cause we don't need it, but then again you could just encrypt or hash the resulting compressed byte array using what ever you want. Works in web player and stand alone.
Answer by pahe · Sep 05, 2013 at 02:13 PM
Thanks for this post, was very useful. I would like to add that on iOS webrequests are automatically uncompressed when setting the requestheader to "gzip", so on iOS is no need for an extra uncompression library.
Answer by Zyxil · Jun 13, 2010 at 03:54 AM
Potential Answer: SharpZipLib
There has been no progress on this question so I will post my research into this potential answer.
http://www.icsharpcode.net/opensource/sharpziplib/
The current version of SharpZipLib seems to work. Code will compile and run in the Unity editor. But the same problem with DotNetZip arises:
NotSupportedException: CodePage 437 not supported
is thrown when the project is built and running on the mono bits.
Another SharpZipLib/Unity developer banged his head on the wall last year and was able to get the code to work with the Unity version of the SharpZipLib library: http://community.sharpdevelop.net/forums/p/9762/27065.aspx
His bug demo code was still posted, so I grabbed the .dll from it and used it in my code. It WORKS! But, I won't immediately call this an acceptable answer to this question. This is the version of the library that ships with Unity (located here: Unity\Editor\Data\Frameworks\Mono.framework), so whatever bugfixes or improvements may have happened after this version was published, or may happen in the future, are lost to me.
This is the closest answer yet, but we're still not there yet. If others post their experiences with this solution, then I may feel better about calling this the answer.
Your answer
Follow this Question
Related Questions
DotNetZip in Webplayer 2 Answers
Manage zip with unity 1 Answer
Need Unity Pro to zip/unzip? 3 Answers
System.IO.Compression not working in JS? 1 Answer
Script to Zip files on Build? 0 Answers