Introduction
SOAP messages generated by Web Services may become very large, reducing
scalability by consuming expensive network bandwidth. CPU power is often much
cheaper and easier to extend than network bandwidth.
HTTP 1.1 protocol introduced standard ways of compression using gzip or
deflate algorithms supported by web servers. Initial tests shown that average
compression ratio 10:1 can be easily achieved with large SOAP messages (for
instance ADO.NET DataSets with multiple rows). Such reduction of data size
cannot be ignored - it's not 10 nor 20 percent but 90% of data
size that can be reduced.
Problems
We usually generate proxy class using wsdl.exe
utility using known WSDL URI. Unfortunately this class is not
compression aware as there is no built-in support for handling gzip/deflate
encoding of HTTP response. I assume we still want to use proxy class (in order
to use web service using cenvenient method calls, without being aware of SOAP
internals and so on).
There are two problem with the proxy class:
-
Proxy class doesn't send Accept-Encoding: gzip, deflate
HTTP header to the web server which tells web server to send compressed stream
instead of raw uncompressed data.
-
Proxy class is unable to decompress incoming (compressed) stream. It's ignoring
Encoding attribute of web response (for instance
Encoding: gzip meaning gzip compressed data). XML deserializer
invoked internally by proxy class (method Invoke
from class SoapHttpClientProtocol) expects
well-formed XML stream and it receives binary compressed data - exception
occurs.
Here comes the solution
Solution for the problems described above requires applying some
manual modifications to the wsdl-generated proxy class. Fortunately proxy class
(derived from class) has two methods that enable to access
WebRequest object used internally by proxy class to send HTTP
requests and GetWebResponse used to read
response stream from the web server. Overriding them solves both problems.
Overriding GetWebRequest
protected override
WebRequest GetWebRequest(Uri uri)
{
WebRequest request = base.GetWebRequest(uri);
request.Headers.Add("Accept-Encoding",
"gzip, deflate");
return request;
}
|
Custom GetWebRequest method enables proxy
class to add custom header to the request sent to web server. If web server
supports compression it will send compressed HTTP stream (gzip or deflate
encoding). First problem is solved.
Overriding GetWebResponse
protected override WebResponse
GetWebResponse(WebRequest request)
{
HttpWebResponseDecompressed response =
new HttpWebResponseDecompressed(request);
return response;
}
|
We have to use our own response "decompression filter" - HttpWebResponseDecompressed
and return it instead of default raw HTTP stream. This solves second problem.
My class takes compressed HTTP data stream, checks for the encoding and
generates decompressed MemoryStream used later
by proxy class.
HttpWebResponseDecompressed class
This class uses NZlib library port which is freely available at
http://www.icsharpcode.net/OpenSource/NZipLib/default.asp. The core of
the class is overridden method GetResponseStream. Encoding is being detected
and appropriate decompressor used (or no decompressor if stream is not
compressed).
public override Stream
GetResponseStream(){
Stream compressedStream = null;
// select right
decompression stream (or null if content is not compressed)
if
(response.ContentEncoding=="gzip")
{
compressedStream =
new GZipInputStream(response.GetResponseStream());
}
else if
(response.ContentEncoding=="deflate")
{
compressedStream =
new InflaterInputStream(response.GetResponseStream());
}
if (compressedStream
!= null)
{
//
decompress
MemoryStream
decompressedStream = new MemoryStream();
int
size = 2048;
byte[]
writeData = new byte[2048];
while
(true)
{
size
= compressedStream.Read(writeData, 0, size);
if
(size > 0)
{
decompressedStream.Write(writeData,
0, size);
}
else
{
break;
}
}
decompressedStream.Seek(0,
SeekOrigin.Begin);
return
decompressedStream;
}
else
return response.GetResponseStream();
}
|
Example - small proof of concept.
I created simple web service which returns all the customers from
Northwind database. Only one method - GetCustomers which returns
simple DataSet (file Customers.asmx.cs).
[WebMethod]
public DataSet GetCustomers()
{
SqlConnection conn = new SqlConnection(ConfigurationSettings.AppSettings["ConnectionString"]);
SqlDataAdapter da = new SqlDataAdapter("select
* from Customers", conn);
DataSet ds = new DataSet();
da.Fill(ds);
return ds;
}
|
Compression is done by PipeBoost
ISAPI fillter for IIS 5.0 (there are probably other products that work well
with MS .NET and enable dynamic dompression of asmx generated output but this
one was the first that Google returned for me as a search result). It was
necessary to turn compression on for file/script extension .asmx (disabled
by default in PipeBoost).
Next step is generation of web service proxy: wsdl
http://localhost/CustomersWS/Customers.asmx/GetCustomers?wsdl.
Proxy class Customers.cs has to be modified the way I've shown
above.
I also created very simple web service client (console
application, file CmdClient.cs) that verifies that compressed web service
response is correctly decoded and deserialized. Of course, I had to modify
proxy class generated by wsdl utility and use HttpWebResponseDecompressed
class. Core of the application is simple invocation of web service using
customized proxy class. Note! Client application is not aware of any
compression, encoding, etc. All the "magic stuff" is encapsulated within
modified proxy class.
Customers customers = new Customers();
// get DataSet
DataSet dset = customers.GetCustomers();
// display result in xml format
Console.WriteLine(dset.GetXml());
|
You will see xml text appearing on the standard concole output -
it works!
Summary
-
Compression of web services can reduce bandwidth even ten times or more.
-
It's all within standard protocols (HTTP 1.1, SOAP)
-
You don't have to get rid of proxy class generated by wsdl -
it's enough to slightly modify it to be able to get compressed response.
-
Web service client code requires no changes at all.