Compressing text is a procedure that can reduce the size of content representation
up to 80%. This means that compressed text can be stored using 80% less space
than uncompressed text. It also means that the content needs less time to be
transferred over a network, which translates into higher performance for client-server
applications that communicate with text, like XML Web services.
The majority of project architectures seek to minimize the size of the data
that is exchanged between consumers and servers. While some experienced developers
are using advanced techniques to optimize the transfer of data through a network,
and especially the Internet, the overhead associated with data transfer remains
a bottleneck in many distributed systems. One solution to this problem is to
acquire more bandwidth, but this is not always practical. Another solution is
to minimize the amount of data to be transferred by compressing it.
When the content is text, its space can be reduced up to 80%. This means that
the bandwidth requirements between consumers and servers decrease at an analogous
percentage. The trade-off is the extra CPU resources both consumer-side and
server-side. Since upgrading a server's CPUs is generally less expensive than
increasing bandwidth, data compression is often the best way to improve performance.
XML/SOAP on a Wire
Let's have a closer look at what is traveling on a network as
a SOAP request/response to an XML Web service. We build an XML Web service that
implements the add method. This method takes two integers as input and returns
the sum of them:
<WebMethod()> Public Function add(ByVal a As Integer, ByVal b As
_
Integer) As Integer
add = a + b
End Function |
When the consumer of this XML Web service invokes the add method,
he actually sends a SOAP request to the server:
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Body><add xmlns="http://tempuri.org/"><a>10</a><b>20</b></add>
</soap:Body></soap:Envelope> |
The server responds to this SOAP request with a SOAP response:
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Body><addResponse xmlns="http://tempuri.org/">
<addResult>30</addResult></addResponse>
</soap:Body></soap:Envelope> |
This is the exact information that travels on the wire after
invoking the XML Web service's method. On more complex XML Web services, the
SOAP response could be a very large dataset. For instance, when it is serialized
to XML, a dataset that contains all columns of the table "orders"
from Northwind database is about 454 KB. If we create an application that retrieves
this dataset by invoking an XML Web service, the SOAP response would contain
all of that data.
To improve performance, we can compress the text content before it travels on
the network. How can we do this? By using SOAP extensions of course!
SOAP Extensions
SOAP extensions is a Microsoft ASP.NET WebMethod interception
mechanism that can be used to manipulate SOAP requests/responses before they
are sent on the wire. Developers can write code that executes before and after
the serialization and deserialization of messages. (SOAP extensions provide
a low level API for implementing all sorts of features.)
Using SOAP extensions, we can reduce the size of SOAP messages traveling on
the wire when a consumer invokes a method from an XML Web service. In most cases
SOAP requests are much smaller in size than SOAP responses (for example, a large
dataset), so we will only compress SOAP responses in our example. As you can
see in Figure 1 on server side and AfterSerialize stage the SOAP Response is
compressed and travels on the wire as a compressed SOAP message and on client
side and BeforeDeserialize stage the compressed SOAP message is decompressed
in order deserialization process to follow successfully:
Figure 1. The SOAP Message is compressed on AfterSerialize
stage (server side) and decompressed on BeforeDeserialize stage (client side)
We could also compress SOAP requests, but in this case the increase
in performance would be insignificant.
In order to compress our Web service's SOAP responses, we need to do two things:
· Compress the SOAP response message after serialization
on the server.
· Decompress the SOAP response message before deserialization on the
client.
This work will be done with a SOAP extension. In the following
paragraphs you can see the full code listing both on the server and the client.
First, here is the XML Web service that returns a large dataset:
| Imports System.Web.Services
<WebService(Namespace := "http://tempuri.org/")> _
Public Class Service1
Inherits System.Web.Services.WebService
<WebMethod()> Public Function getorders() As DataSet
Dim OleDbConnection1 = New System.Data.OleDb.OleDbConnection()
OleDbConnection1.ConnectionString = "Provider=SQLOLEDB.1;
_
Integrated Security=SSPI;Initial Catalog=Northwind; _
Data Source=.;Workstation ID=T-MNIKIT;"
Dim OleDbCommand1 = New System.Data.OleDb.OleDbCommand()
OleDbCommand1.Connection = OleDbConnection1
OleDbConnection1.Open()
Dim OleDbDataAdapter1 = New System.Data.OleDb.OleDbDataAdapter()
OleDbDataAdapter1.SelectCommand = OleDbCommand1
OleDbCommand1.CommandText = "Select * from orders"
Dim objsampleset As New DataSet()
OleDbDataAdapter1.Fill(objsampleset, "Orders")
OleDbConnection1.Close()
Return objsampleset
End Function
End Class
|
On the consumer side, we build a Microsoft Windows application
that invokes the above XML Web service, retrieves the dataset, and presents
it on the UI with a datagrid:
Public Class Form1
Inherits System.Windows.Forms.Form
Private Sub Button1_Click(ByVal sender As System.Object, _
ByVal e As System.EventArgs) Handles Button1.Click
Dim ws As New wstest.Service1()
Dim test1 As New ClsTimer()
test1.StartTiming()
DataGrid1.DataSource = ws.getorders()
test1.StopTiming()
TextBox5.Text = "Total time: " &
test1.TotalTime.ToString & "msec"
End Sub
Private Sub Button2_Click(ByVal sender As System.Object, _
ByVal e As System.EventArgs) Handles Button2.Click
Dim ws As New wstest2.Service1()
Dim test1 As New ClsTimer()
test1.StartTiming()
DataGrid1.DataSource = ws.getorders()
test1.StopTiming()
TextBox4.Text = "Total time: " &
test1.TotalTime.ToString & "msec"
End Sub
End Class
|
The consumer invokes two identical XML Web services, only one
of which uses SOAP compression. The Timer class below is used to measure invocation
time:
Public Class ClsTimer
Private Declare Function timeGetTime Lib "winmm"
() As Long
Private lngStartTime As Integer
Private lngTotalTime As Integer
Private lngCurTime As Integer
Public ReadOnly Property TotalTime() As String
Get
TotalTime = lngTotalTime
End Get
End Property
Public Sub StartTiming()
lngTotalTime = 0
lngStartTime = timeGetTime()
End Sub
Public Sub StopTiming()
lngCurTime = timeGetTime()
lngTotalTime = (lngCurTime - lngStartTime)
End Sub
End Class
|
Server-Side SOAP Extension
On the server side, the SOAP response from the server is compressed
in order to reduce its size. This section walks you through the process.
Step 1
Using Microsoft Visual Studio .NET, we create a new Microsoft
Visual Basic .NET class library project (using "ServerSoapExtension"
as the name of the project) and add the following class:
Imports System
Imports System.Web.Services
Imports System.Web.Services.Protocols
Imports System.IO
Imports zipper
Public Class myextension
Inherits SoapExtension
Private networkStream As Stream
Private newStream As Stream
Public Overloads Overrides Function GetInitializer(ByVal
_
methodInfo As LogicalMethodInfo, _
ByVal attribute As SoapExtensionAttribute) As Object
Return System.DBNull.Value
End Function
Public Overloads Overrides Function GetInitializer(ByVal
_
WebServiceType As Type) As Object
Return System.DBNull.Value
End Function
Public Overrides Sub Initialize(ByVal initializer As Object)
End Sub
Public Overrides Sub ProcessMessage(ByVal message As SoapMessage)
Select Case message.Stage
Case SoapMessageStage.BeforeSerialize
Case SoapMessageStage.AfterSerialize
AfterSerialize(message)
Case SoapMessageStage.BeforeDeserialize
BeforeDeserialize(message)
Case SoapMessageStage.AfterDeserialize
Case Else
Throw New Exception("invalid
stage")
End Select
End Sub
Public Overrides Function ChainStream(ByVal stream As Stream) As Stream
networkStream = stream
newStream = New MemoryStream()
Return newStream
End Function
Public Sub AfterSerialize(ByVal message As SoapMessage)
newStream.Position = 0
Dim fs As New FileStream("c:\temp\server_soap.txt",
_
FileMode.Append, FileAccess.Write)
Dim w As New StreamWriter(fs)
w.WriteLine("-----Response at " + DateTime.Now.ToString())
w.Flush()
Comp(newStream, fs)
w.Close()
newStream.Position = 0
Comp(newStream, networkStream)
End Sub
Public Sub BeforeDeserialize(ByVal message As SoapMessage)
Copy(networkStream, newStream)
Dim fs As New FileStream("c:\temp\server_soap.txt",
_
FileMode.Create, FileAccess.Write)
Dim w As New StreamWriter(fs)
w.WriteLine("----- Request at " + DateTime.Now.ToString())
w.Flush()
newStream.Position = 0
Copy(newStream, fs)
w.Close()
newStream.Position = 0
End Sub
Sub Copy(ByVal fromStream As Stream, ByVal toStream As Stream)
Dim reader As New StreamReader(fromStream)
Dim writer As New StreamWriter(toStream)
writer.WriteLine(reader.ReadToEnd())
writer.Flush()
End Sub
Sub Comp(ByVal fromStream As Stream, ByVal toStream As Stream)
Dim reader As New StreamReader(fromStream)
Dim writer As New StreamWriter(toStream)
Dim test1 As String
Dim test2 As String
test1 = reader.ReadToEnd
'
test2 = zipper.Class1.Compress(test1)
writer.WriteLine(test2)
writer.Flush()
End Sub
End Class
<AttributeUsage(AttributeTargets.Method)> _
Public Class myextensionattribute
Inherits SoapExtensionAttribute
Public Overrides ReadOnly Property ExtensionType() As Type
Get
Return GetType(myextension)
End Get
End Property
Public Overrides Property Priority() As Integer
Get
Return 1
End Get
Set(ByVal Value As Integer)
End Set
End Property
End Class
|
Step 2
We add the ServerSoapExtension.dll assembly as reference and
declare the SOAP extension on web.config of the XML Web service:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.web>
<webServices>
<soapExtensionTypes>
<add type="ServerSoapExtension.myextension,
ServerSoapExtension" priority="1" group="0"/>
</soapExtensionTypes>
</webServices>
...
</system.web>
</configuration>
|
As you can see in the code listing, we use a temporary folder
("c:\temp") with the appropriate permissions, to capture as text file
("c:\temp\server_soap.txt") the SOAP request and the compressed SOAP
response.
Consumer-Side SOAP Extension
On the consumer side, the SOAP response from the server is uncompressed
in order to retrieve the original content. This section walks you through the
process.
Step 1
Using Visual Studio .NET, we create a new Visual Basic .NET
class library project (using "ClientSoapExtension" as the name of
the project) and add the following class:
Imports System
Imports System.Web.Services
Imports System.Web.Services.Protocols
Imports System.IO
Imports zipper
Public Class myextension
Inherits SoapExtension
Private networkStream As Stream
Private newStream As Stream
Public Overloads Overrides Function GetInitializer(ByVal
_
methodInfo As LogicalMethodInfo, _
ByVal attribute As SoapExtensionAttribute) As Object
Return System.DBNull.Value
End Function
Public Overloads Overrides Function GetInitializer(ByVal
_
WebServiceType As Type) As Object
Return System.DBNull.Value
End Function
Public Overrides Sub Initialize(ByVal initializer As Object)
End Sub
Public Overrides Sub ProcessMessage(ByVal message As SoapMessage)
Select Case message.Stage
Case SoapMessageStage.BeforeSerialize
Case SoapMessageStage.AfterSerialize
AfterSerialize(message)
Case SoapMessageStage.BeforeDeserialize
BeforeDeserialize(message)
Case SoapMessageStage.AfterDeserialize
Case Else
Throw New Exception("invalid
stage")
End Select
End Sub
Public Overrides Function ChainStream(ByVal stream As Stream)
_
As Stream
networkStream = stream
newStream = New MemoryStream()
Return newStream
End Function
Public Sub AfterSerialize(ByVal message As SoapMessage)
newStream.Position = 0
Dim fs As New FileStream("c:\temp\client_soap.txt",
_
FileMode.Create, FileAccess.Write)
Dim w As New StreamWriter(fs)
w.WriteLine("----- Request at " + DateTime.Now.ToString())
w.Flush()
Copy(newStream, fs)
w.Close()
newStream.Position = 0
Copy(newStream, networkStream)
End Sub
Public Sub BeforeDeserialize(ByVal message As SoapMessage)
DeComp(networkStream, newStream)
Dim fs As New FileStream("c:\temp\client_soap.txt",
_
FileMode.Append, FileAccess.Write)
Dim w As New StreamWriter(fs)
w.WriteLine("-----Response at " + DateTime.Now.ToString())
w.Flush()
newStream.Position = 0
Copy(newStream, fs)
w.Close()
newStream.Position = 0
End Sub
Sub Copy(ByVal fromStream As Stream, ByVal toStream As Stream)
Dim reader As New StreamReader(fromStream)
Dim writer As New StreamWriter(toStream)
writer.WriteLine(reader.ReadToEnd())
writer.Flush()
End Sub
Sub DeComp(ByVal fromStream As Stream, ByVal toStream As
Stream)
Dim reader As New StreamReader(fromStream)
Dim writer As New StreamWriter(toStream)
Dim test1 As String
Dim test2 As String
test1 = reader.ReadToEnd
test2 = zipper.Class1.DeCompress(test1)
writer.WriteLine(test2)
writer.Flush()
End Sub
End Class
<AttributeUsage(AttributeTargets.Method)> _
Public Class myextensionattribute
Inherits SoapExtensionAttribute
Public Overrides ReadOnly Property ExtensionType() As Type
Get
Return GetType(myextension)
End Get
End Property
Public Overrides Property Priority() As Integer
Get
Return 1
End Get
Set(ByVal Value As Integer)
End Set
End Property
End Class
|
As you can see in the code listing, we use a temporary folder
("c:\temp") with the appropriate permissions to capture as text file
("c:\temp\client_soap.txt") the SOAP request and the uncompressed
SOAP response.
Step 2
We add the ClientSoapExtension.dll assembly as reference and
declare the SOAP extension on the XML Web service reference of our application:
|
Option Strict Off
Option Explicit On
Imports System
Imports System.ComponentModel
Imports System.Diagnostics
Imports System.Web.Services
Imports System.Web.Services.Protocols
Imports System.Xml.Serialization
Namespace wstest2
<System.Diagnostics.DebuggerStepThroughAttribute(), _
System.ComponentModel.DesignerCategoryAttribute("code"),
_
System.Web.Services.WebServiceBindingAttribute(Name:="Service1Soap",
_
[Namespace]:="http://tempuri.org/")> _
Public Class Service1
Inherits System.Web.Services.Protocols.SoapHttpClientProtocol
Public Sub New()
MyBase.New
Me.Url = "http://localhost/CompressionWS/Service1.asmx"
End Sub
<System.Web.Services.Protocols.SoapDocumentMethodAttribute
_
("http://tempuri.org/getproducts",RequestNamespace:= _
"http://tempuri.org/", ResponseNamespace:="http://tempuri.org/",
_
Use:=System.Web.Services.Description.SoapBindingUse.Literal,_
ParameterStyle:=System.Web.Services.Protocols.SoapParameterStyle _
.Wrapped), ClientSoapExtension.myextensionattribute()> _
Public Function getproducts() As System.Data.DataSet
Dim results() As Object = Me.Invoke("getproducts",
_
New Object(-1) {})
Return CType(results(0), System.Data.DataSet)
End Function
Public Function Begingetproducts(ByVal callback As _
System.AsyncCallback, ByVal asyncState As Object) As System.IAsyncResult
_
Return Me.BeginInvoke("getproducts", New Object(-1)
{}, _
callback, asyncState)
End Function
Public Function Endgetproducts(ByVal asyncResult As _
System.IAsyncResult) As System.Data.DataSet
Dim results() As Object = Me.EndInvoke(asyncResult)
Return CType(results(0), System.Data.DataSet)
End Function
End Class
End Namespace
|
Here is source code for the zipper class, which is implemented
using the freeware NZIPLIB library:
using System;
using NZlib.GZip;
using NZlib.Compression;
using NZlib.Streams;
using System.IO;
using System.Net;
using System.Runtime.Serialization;
using System.Xml;
namespace zipper
{
public class Class1
{
public static string Compress(string uncompressedString)
{
byte[] bytData = System.Text.Encoding.UTF8.GetBytes(uncompressedString);
MemoryStream ms = new MemoryStream();
Stream s = new DeflaterOutputStream(ms);
s.Write(bytData, 0, bytData.Length);
s.Close();
byte[] compressedData = (byte[])ms.ToArray();
return System.Convert.ToBase64String(compressedData,
0, _
compressedData.Length);
}
public static string DeCompress(string compressedString)
{
string uncompressedString="";
int totalLength = 0;
byte[] bytInput = System.Convert.FromBase64String(compressedString);;
byte[] writeData = new byte[4096];
Stream s2 = new InflaterInputStream(new
MemoryStream(bytInput));
while (true)
{
int size = s2.Read(writeData, 0, writeData.Length);
if (size > 0)
{
totalLength += size;
uncompressedString+=System.Text.Encoding.UTF8.GetString(writeData,
_
0, size);
}
else
{
break;
}
}
s2.Close();
return uncompressedString;
}
}
}
|
Metrics
Software & Hardware
· Consumer side: Intel Pentium III 500 MHz, 512 MB RAM,
Windows XP.
· Server side: Intel Pentium III 500 MHz, 512 MB RAM, Windows 2000 Server,
Microsoft SQL Server 2000.
On the consumer side a Windows application invokes an XML Web service. The XML
Web service returns a dataset and fills a datagrid on the consumer side.
Figure 2. This is a sample application which invokes
the same XML Web Service with and without using SOAP Compression. The XML Web
Services returns a very large dataset.
CPU Load Time and History
As shown in Figure 3, the CPU load time without using compression
in our case was 29903 milliseconds.
Figure 3. CPU load history & time without compression
The CPU load time using compression in our case, shown in Figure
4, was 15182 milliseconds.
Figure 4. CPU load history & time with compression
As you can see, we've achieved 50% less time in retrieving the
dataset on the consumer side with very little impact on the CPU load. SOAP Compression
can significantly increase the XML Web Services' performance when consumers
and servers exchange large data. There are a lot of solutions for improving
performance on the web. Writing code is always the cheapest solution for achieving
great results. SOAP Extensions is the way to improve XML Web Services' performance
by compressing the exchanged data with small impact on CPU load. Also it is
worth mention that someone could use more powerful and less demanding compression
algorithms to achieve even greater results.