JAVA 压缩和序列化

Python015

JAVA 压缩和序列化,第1张

压缩和序列化主要用在数据的存储和传输上,二者都是由IO流相关知识实现,这里统一介绍下。

全部章节传送门:

Java I/O类支持读写压缩格式的数据流,你可以用他们对其他的I/O流进行封装,以提供压缩功能。

GZIP接口比较简单,适合对单个数据流进行压缩,在Linux系统中使用较多。

ZIP格式可以压缩多个文件,而且可以和压缩工具进行协作,是经常使用的压缩方法。

JAR(Java Archive,Java 归档文件)是与平台无关的文件格式,它允许将许多文件组合成一个压缩文件。为 J2EE 应用程序创建的 JAR 文件是 EAR 文件(企业 JAR 文件)。

JAR 文件格式以流行的 ZIP 文件格式为基础。与 ZIP 文件不同的是,JAR 文件不仅用于压缩和发布,而且还用于部署和封装库、组件和插件程序,并可被像编译器和 JVM 这样的工具直接使用。在 JAR 中包含特殊的文件,如 manifests 和部署描述符,用来指示工具如何处理特定的 JAR。

如果一个Web应用程序的目录和文件非常多,那么将这个Web应用程序部署到另一台机器上,就不是很方便了,我们可以将Web应用程序打包成Web 归档(WAR)文件,这个过程和把Java类文件打包成JAR文件的过程类似。利用WAR文件,可以把Servlet类文件和相关的资源集中在一起进行发布。在这个过程中,Web应用程序就不是按照目录层次结构来进行部署了,而是把WAR文件作为部署单元来使用。

一个WAR文件就是一个Web应用程序,建立WAR文件,就是把整个Web应用程序(不包括Web应用程序层次结构的根目录)压缩起来,指定一个.war扩展名。下面我们将第2章的Web应用程序打包成WAR文件,然后发布

要注意的是,虽然WAR文件和JAR文件的文件格式是一样的,并且都是使用jar命令来创建,但就其应用来说,WAR文件和JAR文件是有根本区别的。JAR文件的目的是把类和相关的资源封装到压缩的归档文件中,而对于WAR文件来说,一个WAR文件代表了一个Web应用程序,它可以包含 Servlet、HTML页面、Java类、图像文件,以及组成Web应用程序的其他资源,而不仅仅是类的归档文件。

在命令行输入jar即可查看jar命令的使用方法。

对象转换为字节序列的过程称为对象的序列化。把字节序列恢复为对象的过程称为对象的反序列化。

对象的序列化主要有两种用途:

java.io.ObjectOutputStream代表对象输出流,它的writeObject(Object obj)方法可对参数指定的obj对象进行序列化,把得到的字节序列写到一个目标输出流中。

java.io.ObjectInputStream代表对象输入流,它的readObject()方法从一个源输入流中读取字节序列,再把它们反序列化为一个对象,并将其返回。

只有实现了Serializable的对象才能被序列化。对象序列化包括如下步骤:

对象反序列化的步骤如下:

创建一个可以可以序列化的对象。

然后进行序列化和反序列化测试。

s​e​r​i​a​l​V​e​r​s​i​o​n​U​I​D​:​ ​字​面​意​思​上​是​序​列​化​的​版​本​号​,凡是实现Serializable接口的类都有一个表示序列化版本标识符的静态变量。

JAVA序列化的机制是通过判断类的serialVersionUID来验证的版本一致的。在进行反序列化时,JVM会把传来的字节流中的serialVersionUID于本地相应实体类的serialVersionUID进行比较。如果相同说明是一致的,可以进行反序列化,否则会出现反序列化版本一致的异常,即是InvalidCastException。

为了提高serialVersionUID的独立性和确定性,强烈建议在一个可序列化类中显示的定义serialVersionUID,为它赋予明确的值。

控制序列化字段还可以使用Externalizable接口替代Serializable借口。此时需要定义一个默认构造器,否则将为得到一个异常(java.io.InvalidClassException: PersonPersonno valid constructor);还需要定义两个方法(writeExternal()和readExternal())来控制要序列化的字段。

如下为将Person类修改为使用Externalizable接口。

transient修饰符仅适用于变量,不适用于方法和类。在序列化时,如果我们不想序列化特定变量以满足安全约束,那么我们应该将该变量声明为transient。执行序列化时,JVM会忽略transient变量的原始值并将默认值(引用类型就是null,数字就是0)保存到文件中。因此,transient意味着不要序列化。

静态变量不是对象状态的一部分,因此它不参与序列化。所以将静态变量声明为transient变量是没有用处的。

1. 使用更高效的序列化技术,比如Protobuf,Thrift等;

2. 尽量减少序列化的数据量,比如使用压缩算法压缩数据;

3. 尽量减少序列化的次数,比如使用缓存机制减少对象的序列化次数;

4. 尽量减少序列化的复杂性,比如尽量避免在序列化过程中进行复杂的计算;

5. 尽量使用异步序列化,比如使用多线程来处理序列化和反序列化任务;

6. 尽量使用缓冲区,比如在序列化和反序列化过程中使用缓冲区来减少IO操作;

7. 尽量使用序列化框架,比如使用Jackson,Gson等序列化框架来提高序列化和反序列化的性能。

序列化与反序列化

先在webserver端序列化,然后再客户端反序列化,需要注意的是两次序列化方式要一致,从下面选择一种方式来实现吧,一般都是把对象序列化成字符串,然后再把字符串反序列化成相应的对象,当然根据需要你也可以序列化成byte数组

/****************************************

* 作者:张江松

* 创始时间:2008-5-29

* 功能:

*

* 修改人:

* 修改时间:

* 描述:

****************************************/

namespace MultiMedia.Common

{

using System

using System.Collections.Specialized

using System.Globalization

using System.IO

using System.Runtime.Serialization.Formatters.Binary

using System.Security

using System.Security.Permissions

using System.Text

using System.Xml

using System.Xml.Serialization

using System.Runtime.Serialization

using System.Runtime.Serialization.Formatters.Soap

/// <summary>

/// SerializeHelper 用于简化序列化和反序列化操作 。

/// 作者:朱伟 [email protected]

/// 2004.05.12

/// </summary>

public static class SerializeHelper

{

#region BinaryFormatter

#region SerializeObject

public static byte[] SerializeObject(object obj) //obj 可以是数组

{

IFormatter formatter = new BinaryFormatter()

MemoryStream memoryStream = new MemoryStream()//此种情况下,mem_stream的缓冲区大小是可变的

formatter.Serialize(memoryStream, obj)

byte[] buff = memoryStream.ToArray()

memoryStream.Close()

return buff

}

public static void SerializeObject(object obj, ref byte[] buff, int offset) //obj 可以是数组

{

byte[] rude_buff = SerializeHelper.SerializeObject(obj)

for (int i = 0i <rude_buff.Lengthi++)

{

buff[offset + i] = rude_buff[i]

}

}

#endregion

#region DeserializeBytes

public static object DeserializeBytes(byte[] buff, int index, int count)

{

IFormatter formatter = new BinaryFormatter()

MemoryStream stream = new MemoryStream(buff, index, count)

object obj = formatter.Deserialize(stream)

stream.Close()

return obj

}

#endregion

#endregion

#region SoapFormatter

#region SerializeObjectToString

/// <summary>

/// SerializeObjectToString 将对象序列化为SOAP XML 格式。

/// 如果要将对象转化为简洁的xml格式,请使用ESBasic.Persistence.SimpleXmlConverter类。

/// </summary>

public static string SerializeObjectToString(object obj)

{

IFormatter formatter = new SoapFormatter()

MemoryStream stream = new MemoryStream()

formatter.Serialize(stream, obj)

stream.Position = 0

StreamReader reader = new StreamReader(stream)

string res = reader.ReadToEnd()

stream.Close()

return res

}

#endregion

#region DeserializeString

public static object DeserializeString(string str,System.Text.Encoding ecnoding)

{

byte[] buff = ecnoding.GetBytes(str)

IFormatter formatter = new SoapFormatter()

MemoryStream stream = new MemoryStream(buff, 0, buff.Length)

object obj = formatter.Deserialize(stream)

stream.Close()

return obj

}

#endregion

#endregion

#region XmlSerializer

#region XmlObject

public static string XmlObject(object obj)

{

XmlSerializer xmlSerializer = new XmlSerializer(obj.GetType())

MemoryStream stream = new MemoryStream()

xmlSerializer.Serialize(stream, obj)

stream.Position = 0

StreamReader reader = new StreamReader(stream)

string res = reader.ReadToEnd()

stream.Close()

return res

}

#endregion

#region ObjectXml

public static T ObjectXml<T>(string str, System.Text.Encoding ecnoding)

{

return (T)SerializeHelper.ObjectXml(str, typeof(T), ecnoding)

}

public static object ObjectXml(string str, Type targetType, System.Text.Encoding ecnoding)

{

byte[] buff = ecnoding.GetBytes(str)

XmlSerializer xmlSerializer = new XmlSerializer(targetType)

MemoryStream stream = new MemoryStream(buff, 0, buff.Length)

object obj = xmlSerializer.Deserialize(stream)

stream.Close()

return obj

}

#endregion

#endregion

#region SaveToFile

/// <summary>

/// SaveToFile 将对象的二进制序列化后保存到文件。

/// </summary>

public static void SaveToFile(object obj, string filePath)

{

FileStream stream = new FileStream(filePath, FileMode.CreateNew)

IFormatter formatter = new BinaryFormatter()

formatter.Serialize(stream, obj)

stream.Flush()

stream.Close()

}

#endregion

#region ReadFromFile

/// <summary>

/// ReadFromFile 从文件读取二进制反序列化为对象。

/// </summary>

public static object ReadFromFile(string filePath)

{

byte[] buff = FileHelper.ReadFileReturnBytes(filePath)

return SerializeHelper.DeserializeBytes(buff, 0, buff.Length)

}

#endregion

}

/// <summary>

/// 序列化类

/// </summary>

public static class Serializer

{

/// <summary>

/// Static Constructor is used to set the CanBinarySerialize value only once for the given security policy

/// </summary>

static Serializer()

{

SecurityPermission sp = new SecurityPermission( SecurityPermissionFlag.SerializationFormatter )

try

{

sp.Demand()

CanBinarySerialize = true

}

catch ( SecurityException )

{

CanBinarySerialize = false

}

}

/// <summary>

/// Readonly value indicating if Binary Serialization (using BinaryFormatter) is allowed

/// </summary>

public static readonly bool CanBinarySerialize

/// <summary>

/// Converts a .NET object to a byte array. Before the conversion happens, a check with

/// Serializer.CanBinarySerialize will be made

/// </summary>

/// <param name="objectToConvert">Object to convert</param>

/// <returns>A byte arry representing the object paramter. Null will be return if CanBinarySerialize is false</returns>

public static byte[] ConvertToBytes( object objectToConvert )

{

byte[] byteArray = null

if ( CanBinarySerialize )

{

BinaryFormatter binaryFormatter = new BinaryFormatter()

using ( MemoryStream ms = new MemoryStream() )

{

binaryFormatter.Serialize( ms, objectToConvert )

// Set the position of the MemoryStream back to 0

//

ms.Position = 0

// Read in the byte array

//

byteArray = new Byte[ ms.Length ]

ms.Read( byteArray, 0, byteArray.Length )

ms.Close()

}

}

return byteArray

}

/// <summary>

/// Saves an object to disk as a binary file.

/// </summary>

/// <param name="objectToSave">Object to Save</param>

/// <param name="path">Location of the file</param>

/// <returns>true if the save was succesful.</returns>

public static bool SaveAsBinary( object objectToSave, string path )

{

if ( objectToSave != null &&CanBinarySerialize )

{

byte[] ba = ConvertToBytes( objectToSave )

if ( ba != null )

{

using ( FileStream fs = new FileStream( path, FileMode.OpenOrCreate, FileAccess.Write ) )

{

using ( BinaryWriter bw = new BinaryWriter( fs ) )

{

bw.Write( ba )

return true

}

}

}

}

return false

}

/// <summary>

/// Converts a .NET object to a string of XML. The object must be marked as Serializable or an exception

/// will be thrown.

/// </summary>

/// <param name="objectToConvert">Object to convert</param>

/// <returns>A xml string represting the object parameter. The return value will be null of the object is null</returns>

public static string ConvertToString( object objectToConvert )

{

string xml = null

if ( objectToConvert != null )

{

//we need the type to serialize

Type t = objectToConvert.GetType()

XmlSerializer ser = new XmlSerializer( t )

//will hold the xml

using ( StringWriter writer = new StringWriter( CultureInfo.InvariantCulture ) )

{

ser.Serialize( writer, objectToConvert )

xml = writer.ToString()

writer.Close()

}

}

return xml

}

public static void SaveAsXML( object objectToConvert, string path )

{

if ( objectToConvert != null )

{

//we need the type to serialize

Type t = objectToConvert.GetType()

XmlSerializer ser = new XmlSerializer( t )

//will hold the xml

using ( StreamWriter writer = new StreamWriter( path ) )

{

ser.Serialize( writer, objectToConvert )

writer.Close()

}

}

}

/// <summary>

/// Converts a byte array to a .NET object. You will need to cast this object back to its expected type.

/// If the array is null or empty, it will return null.

/// </summary>

/// <param name="byteArray">An array of bytes represeting a .NET object</param>

/// <returns>The byte array converted to an object or null if the value of byteArray is null or empty</returns>

public static object ConvertToObject( byte[] byteArray )

{

object convertedObject = null

if ( CanBinarySerialize &&byteArray != null &&byteArray.Length >0 )

{

BinaryFormatter binaryFormatter = new BinaryFormatter()

using ( MemoryStream ms = new MemoryStream() )

{

ms.Write( byteArray, 0, byteArray.Length )

// Set the memory stream position to the beginning of the stream

//

ms.Position = 0

if ( byteArray.Length >4 )

convertedObject = binaryFormatter.Deserialize( ms )

ms.Close()

}

}

return convertedObject

}

/// <summary>

/// Converts a string of xml to the supplied object type.

/// </summary>

/// <param name="xml">Xml representing a .NET object</param>

/// <param name="objectType">The type of object which the xml represents</param>

/// <returns>A instance of object or null if the value of xml is null or empty</returns>

public static object ConvertToObject( XmlNode node, Type objectType )

{

object convertedObject = null

if ( node != null )

{

using ( StringReader reader = new StringReader( node.OuterXml ) )

{

XmlSerializer ser = new XmlSerializer( objectType )

convertedObject = ser.Deserialize( reader )

reader.Close()

}

}

return convertedObject

}

public static object ConvertFileToObject( string path, Type objectType )

{

object convertedObject = null

if ( path != null &&path.Length >0 )

{

using ( FileStream fs = new FileStream( path, FileMode.Open, FileAccess.Read ) )

{

XmlSerializer ser = new XmlSerializer( objectType )

convertedObject = ser.Deserialize( fs )

fs.Close()

}

}

return convertedObject

}

/// <summary>

/// Converts a string of xml to the supplied object type.

/// </summary>

/// <param name="xml">Xml representing a .NET object</param>

/// <param name="objectType">The type of object which the xml represents</param>

/// <returns>A instance of object or null if the value of xml is null or empty</returns>

public static object ConvertToObject( string xml, Type objectType )

{

object convertedObject = null

if ( !string.IsNullOrEmpty( xml ) )

{

using ( StringReader reader = new StringReader( xml ) )

{

XmlSerializer ser = new XmlSerializer( objectType )

convertedObject = ser.Deserialize( reader )

reader.Close()

}

}

return convertedObject

}

public static object LoadBinaryFile( string path )

{

if ( !File.Exists( path ) )

return null

using ( FileStream fs = new FileStream( path, FileMode.Open, FileAccess.Read ) )

{

BinaryReader br = new BinaryReader( fs )

byte[] ba = new byte[ fs.Length ]

br.Read( ba, 0, ( int )fs.Length )

return ConvertToObject( ba )

}

}

/// <summary>

/// Creates a NameValueCollection from two string. The first contains the key pattern and the second contains the values

/// spaced according to the kys

/// </summary>

/// <param name="keys">Keys for the namevalue collection</param>

/// <param name="values">Values for the namevalue collection</param>

/// <returns>A NVC populated based on the keys and vaules</returns>

/// <example>

/// string keys = "key1:S:0:3:key2:S:3:2:"

/// string values = "12345"

/// This would result in a NameValueCollection with two keys (Key1 and Key2) with the values 123 and 45

/// </example>

public static NameValueCollection ConvertToNameValueCollection( string keys, string values )

{

NameValueCollection nvc = new NameValueCollection()

if ( keys != null &&values != null &&keys.Length >0 &&values.Length >0 )

{

char[] splitter = new char[ 1 ] { ':' }

string[] keyNames = keys.Split( splitter )

for ( int i = 0i <( keyNames.Length / 4 )i++ )

{

int start = int.Parse( keyNames[ ( i * 4 ) + 2 ], CultureInfo.InvariantCulture )

int len = int.Parse( keyNames[ ( i * 4 ) + 3 ], CultureInfo.InvariantCulture )

string key = keyNames[ i * 4 ]

//Future version will support more complex types

if ( ( ( keyNames[ ( i * 4 ) + 1 ] == "S" ) &&( start >= 0 ) ) &&( len >0 ) &&( values.Length >= ( start + len ) ) )

{

nvc[ key ] = values.Substring( start, len )

}

}

}

return nvc

}

/// <summary>

/// Creates a the keys and values strings for the simple serialization based on a NameValueCollection

/// </summary>

/// <param name="nvc">NameValueCollection to convert</param>

/// <param name="keys">the ref string will contain the keys based on the key format</param>

/// <param name="values">the ref string will contain all the values of the namevaluecollection</param>

public static void ConvertFromNameValueCollection( NameValueCollection nvc, ref string keys, ref string values )

{

if ( nvc == null || nvc.Count == 0 )

return

StringBuilder sbKey = new StringBuilder()

StringBuilder sbValue = new StringBuilder()

int index = 0

foreach ( string key in nvc.AllKeys )

{

if ( key.IndexOf( ':' ) != -1 )

throw new ArgumentException( "ExtendedAttributes Key can not contain the character \":\"" )

string v = nvc[ key ]

if ( !string.IsNullOrEmpty( v ) )

{

sbKey.AppendFormat( "{0}:S:{1}:{2}:", key, index, v.Length )

sbValue.Append( v )

index += v.Length

}

}

keys = sbKey.ToString()

values = sbValue.ToString()

}

}

}