Not so long ago I wrote a story which was born out of some frustration. Being an object oriented programmer with a background in RAD tools like Delphi and VBA I saw the great possibilities in .NET. But, very often found myself translating documentation and examples based on quite another background. In the story, I described an ASP.NET page to edit a dataset in a grid using all RAD and OO possibilities available in .NET. The reactions to that article have inspired me to continue on this path.

In this article, I will work the same way with Web Services and will concentrate on the exchange of complex XML data. There have been a number of great articles on how this can be done describing a lot of the details which are going on behind the scenes. The final release of Visual Studio .NET has a great number of features with which it is easy to build powerful Web Services and their consumers without having to know that much about XML or serialization.

What is a Web Service?

A Web Service is an object which can be created remotely over the HTTP protocol. An object has methods, the methods of a Web Service provide the functionality of the Web Service. Methods are called following the Simple Object Access Protocol (SOAP), the client calling these methods is said to consume the Web Service. As Web Services are so easy to access and provide their results in such an easy way they are the perfect building blocks for integration of software services over the Internet.

What is RAD?

In the earlier article on working with the ASP.NET datagrid, I spent some words on Rapid Application Development. I mentioned three points which I consider to be an essential part of RAD. Let's take a look what VS.NET has to offer when building Web Services.

• Visual designers. A Web Service in VS.NET has a working surface on which you can drag components from the palette. This surface will not be visible to the user of the Web Service but is a good design time placeholder for things like a database connection, queries on the database or windows-services; to name just a few of the components a Web Service could use. The properties of these components are set using the property editor, without writing any code they are tailored to need.

The SQL for the database queries is constructed using the query builder, here you just click the fields you need. The XML in the Web Service is described by an XML schema. To edit a schema you use the VS schema editor where it boils down again to clicking fields and setting properties in the property editor.

• Wizards. A wizard collects some information and generates code or property values out of this. After specifying a location the Web Service wizard generates the complete skeleton of the Web Service. When dropping a database table or view on the designer a wizard will generates SQL statements to work with the data. When adding a reference to a Web Service a wizard generates the code to work with the service.

Wizards and visual designers work tightly together. The wizard generates the SQL statements which can be fine-tuned with the query builder. Wizards translate the visual representation to pure C# (or VB) code. When this code is executed it will create objects whose field values will be as set in the property editor.

• Strongly typed code and schema's. When creating a dataset VS generates a class to wrap up the dataset. This class has members  for every data field. Accessing fields in code is  done using these members, which is more reliable then using strings to identify fields and will execute faster as well.

  When the wizards creates a dataset it also creates an XML schema. This schema describes the fields in the dataset. When the dataset is returned to a consumer this schema is very useful and so VS will include it in the description of the Web Service. When building a Web Service client the developer can now also code against strongly typed fields.

Let's start using all these tools to build a Web Service with a consumer and investigate the XML data these two exchange.

Creating a Web Service

The Web Service will do the same thing as the ASP.NET page I had built before--it browses and updates the same SQL server table. As an extra option it will be able to work with selections on the rows. To create a service, I choose a new project in the File menu of Visual Studio. A dialog pops up where I have to specify the location of the service. For development purposes it is a good idea to choose //LocalHost, here I have full control for debugging.

After choosing a name and clicking OK, VS.NET generates a complete skeleton for the Web Service, all the parts can be seen in the solution explorer.

Most of these can be taken for granted for this moment. They will do their work behind the scenes, we will meet some of them in the process of building the service.

Accessing the Database

When VS.NET is ready to create the service I am confronted with the design pane of the Web Service. My service uses the same table as last time, it has an autonumber, a text and a date field. Using the server explorer I look for the database and its table and I drag the table on the form.

When the table is dropped VS generates two components: a sqlConnection  to manage the connection between the Web Service and the database and a sqlDataAdapter which uses this connection and has SQL statements to retrieve, update, insert and delete data from the table.

The sqlConnection  has a connection-string property. This string contains all parameters SQL Server needs to open the database. When I dropped the table on the design pane VS.NET generated this connection-string. The sqlDataAdapter has four sqlCommands, a SelectCommand., an UpdateCommand, an InsertCommand and a  DeleteCommand. Each command has a CommandText property containing the SQL to execute the desired action. Again VS.NET has generated these SQL statements for me. To fine-tune the statement I click the button next to the CommandText and the Querybuilder pops up.

The generated DataAdapter selects all rows in the table. To make a selection on the text field I start with dropping the table on the pane a second time. VS.NET will create another sqlDataAdapter. There already is a connection to the database, the wizard reuses this one for the adapter's connection property. To fine tune the select statement I click the button and do my work in the querybuilder.

For the JustText column I fill in LIKE @Param1 + '%' as criteria. This means that I want to return all rows which start with the string passed in the first parameter of the query. To order the result I select Ascending as Sort Type for the column. When clicking OK the query builder will notice that I have used a parameter and will ask permission to update the parameters collection. The result is that the parameters collection of the SelectCommand automatically got a member.

You can see this collection by clicking the button in the Parameters property.

The Data as XML 

Now, I have access to the data but the .NET components do not know yet how to work with it. NET only understands data in XML format and the DataSet component is the way to wrap up XML. Right clicking a DataAdapter starts the "Generate dataset" wizard. After filling in DataSetRAD  as name I click the OK button. The wizard generates two things. The most important one is the XSD file which gets added to the solution. It contains the XML schema which describes the dataset. Doubleclicking this item pops up the schema editor.

Here I can browse and edit the schema in a visual way. The wizard has not just created elements for every field in the table but has enriched the schema with interesting information like that the idField is a key field, has an auto-incrementing value and is read-only. The editor has an XML tab which shows the schema in its textual XML representation. With the property editor I have to set two properties of the schema. As the localization settings on my PC are set to Dutch, the locale of the dataset is set to Dutch. I will not tire you with the peculiarities of this language and set the locale to (Default) so the dataset will use the settings of the PC used.

The other property I have to change is the target namespace. A namespace can be described as vaguely as a collection of related things. The .NET class libraries are organized in namespaces, the namespace System.XML bundles all classes related to XML. Web Services and XML schemas use namespaces as well. By default VS sets the namespace to http://www.tempuri.org/DataSetRAD.xsd. This looks like a URL but you will not find a site named tempuri.org on the web. The string is actually meant as an unique identifying string, it is called an Universal Resource Identification. tempuri.org stands for all schemas who are still in development. When publishing a Web Service and the schemas defined in there, it is suggested to use the address of your Website. So I will use http://www.Gekko-Software.nl/DataSetRAD.xsd. It could be an idea to physically store your XML schema's there but when it comes to consuming Web Services nobody will really take a look there. I will come back on these schema's when writing the consumer code.

The second thing VS.NET generates is the representation of the dataset as a C# class. The file containing this code is hidden by the solution explorer but you can reach it following the class viewer. This is code which you are not supposed to touch. It would not even make sense to change anything here, every time the schema is changed this code will be regenerated from scratch. But this code is perfect to be used by other code, it is the place where all tables in the dataset and all fields of the tables are declared as true C# fields and properties. The class viewer gives a nice overview.

This XML schema can be used for both DataAdapters. The only difference between the adapters is the number of rows they return. The rows themselves are of the same type. Now all the building blocks are ready and I can start creating the functionality of the Web Service.

Simple WebMethods

Behind the design pane of the Web Service lies the code VS.NET has generated, it is a System.Web.Services.WebService derive class with one commented example method. The consumer of the Web Service can use all public methods of the class which are marked with the [WebMethod] attribute.

Just like the XML schema the service needs a namespace so that any consumer can identify it over the Internet. It is again advised to use a name based on the URL of the Website where the service will be deployed. If you do not specify a name the service will also end up in the tempuri.org namespace, together with all other services under development. The namespace's URI is set using an attribute in the code.

The Web Service is going to work with database data, to be able to do that the connection to the database has to be opened, this is done in the constructor of the service by making a call to the Open() method of the componenet.

It is a waste of resources to keep database connections open when no longer needed. By convention resources are freed when an object is disposed. The WebService class even has an event dedicated to the dispose event, you can reach it via the property editor. In the event handler I can close the connection.

Having the database at hand I can start writing my own WebMethods. As a first exercise I will create a method which returns the number of rows matching a selection string.

The method takes a string parameter and returns an integer. It uses the DataSetRAD type to create a new typed dataset object. The string is passed to the named parameter of the DataAdapter which uses its Fill method to fill the dataset with data. The dataset has a TestTable property, it is the table which the wizard imported from the database. The table has a Rows property, which in turn has a Count property which now stands for the number of rows matching the string.

I can fully exploit the dataset being strongly typed, there is no mistake of which table I want to count the rows. To identify the parameter of the DataAdapter I had to use a string, as DataAdapters are not subclassed. They are all of the same type and so every Parameters property is of the same type, not with a separate member for every parameter, like the dataset. This string is a source for errors which will not show up until run time.

WebMethods Using Complex Types

The consumer of the Web Service is going to work with complete datasets. To give him or her the data I will create a WebMethod which returns a complete dataset. The simplest is to return all data.

The method returns a dataset which follows the scheme of DataSetRAD so the return type of the method is DataSetRAD. The implementation of the method could not be simpler, all it has to do is create a new DataSetRAD, use the DataAdapter to fill it with data and return the DataSet.

Returning selected data uses sqlDataAdapter2, the Adapater with the parameter.

The implementation is even simpler than returning the count of selected rows, the method just returns the whole dataset.

What these methods do is serialize the contents of the datasets. Which means that the contents of the dataset is written to one long XML string of characters which can be transported over the (inter)net using a simple protocol like HTTP(S) or even SMTP. The .NET FCL (Framework Class Libraries) offer a lot of support for serializing objects. The nice thing about datasets is that they know how to serialize themselves, there is no need for us having to know anything at all about serialization at this moment.

WebMethods do not only know how to return datasets, they know how to receive them as well. The consumer of the dataset can modify the contents and return the result to the Web Service, which will commit the changes to the database. As there is a reasonable chance of something going wrong in an update, I will create a WebMethod which catches exceptions and returns an informative string.

The method accepts a strongly typed DataSetRAD in the parameter. It passes the dataset directly to the UpDate method of the DataAdapter. The update statement is guarded by a try catch block. In case an exception is thrown its message is returned to the consumer, in case all went well the consumer will receive OK.

Examining the Web Service with a Browser

Now the service is ready and it can be run from the VS IDE. What VS will do is start Internet Explorer which will open the asmx page on the localhost, just like running an ASP.NET page. A Web Service has no visual representation but here Web Services Description Language (WSDL) comes into action. Each Web Service contains a description of itself written in WSDL. This description is comparable to a type library in COM, it has a list of all methods in the service with their signature and description. VS has generated the WSDL for the service behind the scenes. Internet explorer will use this WSDL to list all methods in the service and you can call them directly using the HTTP GET protocol.

Let's try this for the SelectedRows WebMethod. After entering a parameter value and clicking the invoke button a new explorer with the URL http://localhost/RADService/Service1.asmx/SelectedRows?forQueryString=S  will open. In the URL are the location of the service, the name of the method and the method's parameter. The result is quite a chunk of XML data. The nice thing about the latest versions of IE is that they are very XML aware, you can expand and collapse nodes to get a good overview of the result.

What you cannot do here is edit any of the XML data. And even if you could do that you cannot send the result back to the service, IE will report that the Update method cannot be used using the HTTP GET protocol, due to the complex type of its parameter.

Creating a Client for the Web Service

To fully exploit the service I will create a client using VS.NET. This does not have to be a Web page, it is very well possible to create a Windows application. What I will do is create a Windows form which reads its data from the Web Service, edit this data in a datagrid and send the modified data back to the Web Service.

In the same solution I create a new project. After creating the form with the datagrid I have to add a reference to the service. Right clicking the project in the solution explorer starts the Add Web Reference wizard. The first question is where the service is located. A webservice can publish its availability on the internet using UDDI. On the web are a couple of "yellow pages" for webservices, one of them is http://uddi.microsoft.com/. My experimental service is assigned a public namespace but is not registered anywhere yet. I will not dive deeper into UDDI here, the link will provide you with more info. I have to type in the location of my service myself, this is http://localhost/radservice/service1.asmx. The wizard now displays the same screen as Internet Explorer, actually this wizard is a built around a browser component. Clicking View Contract displays the WSDL which is another chunk of XML.

This WSDL is extracted from the webservice using http://localhost/radservice/service1.asmx?wsdl. Special attention deserves the import part, here you see a reference to the schema of the dataset DataSetRAD.xsd. The way to get this schema is using the URL http://localhost/radservice/service1.asmx?schema=DataSetRAD.

So when you have the location of a Web Service you can extract all the information on the service using URL's, you could do this from your browser as well. When you have installed the demo application on your localhost you can even do this from this article by just clicking the links. Visual Studio uses these URL's to generate a new namespace in the client application's namespace. The namespace is named after the server where the service is located, that will be localhost. The namespace is filled with classes to wrap the service itself and all the types it defines, including DataSetRAD.

Using the Service

The client form has a datagrid, this grid needs a dataset to populate it. From the data tab of the toolbox I drag a dataset component. When I drop it on the form a dialog pops up asking what kind of dataset I want to create, it suggests a typed dataset of type Localhost.DataSetRAD and that is exactly what I want.

Now I can set the Dataset property of the grid to this dataSetRAD1 and the DataMember property to TestTable. The grid will now show columns matching the fields of TestTable.

To populate the grid I give the form two buttons and a textbox, one button to read all rows and one button to read only rows matching the contents of the textbox. Let's start with reading all rows:

First I create a new Web Service object then I clear the contents of the dataset by calling its Clear method. The AllRows method of the dataset will return the data, it is transferred to the local dataset using the Merge method. After clicking the button it takes some time to start up the Web Service but the result is that the grid will be populated with the data sent by the Web Service.

The second button populates the grid with only selected data, the code to get this done is quite similar.

The Merge method is quite powerful, as its name suggests it can merge data already in the dataset with new data. To demonstrate this I have added a checkbox, when this is not checked the dataset will be cleared when new data comes in. Now I can collect rows for more than one selection string and view them all at once. The Merge method is very good in this, it will not just append all new rows but checks first if the row is already in the dataset to prevent duplicates.

You can modify the rows in the grid, add new rows or delete rows. Committing these updates to the database is quite straightforward.

The local dataset is passed as a parameter to the UpdateRows method of the service, the result is displayed on a label. To get an idea of the power of the service you should play some time with it. You should notice some nice behavior. The idField will get a new autonumber automatically and can not be modified. The JustDate field will recognize 4-07 as a date in the Netherlands and 7/4 in the United States. And after clicking the update button OK should appear.

Viewing the XML

Now I have created a very simple Client Server database application, which does updates, that can work over the Internet. To get an idea of all the magic going on behind the scenes I will peek inside the local dataset. Any dataset can be viewed as XML. To do that I add a multi-line textbox and a button to make XML snapshots of the data.

The GetXML method of the dataset returns the dataset's contents in a simple XML format, just like you see it in the datagrid.

But this is not the format in which the DataAdapter fills the dataset. If you examine the results of the WebMethods in a browser you will notice the DiffGr tags. The data is returned in the XML diffgram format, this is the same format which MS SQLserver uses when returning data in XML format. The DataSet class can also give a representation of its contents in that format:

The dataset needs a StringWriter object. The StringWriter class can be found in the FCL and is one of the ways to construct complex strings. The WriteXML method of the dataset uses this stringwriter and in the second parameter the format of the desired XML is  specified, a DiffGram being one of the possibilities. To see the XML resulting I use the ToString method of the StringWriter.

The resulting XML now looks quite different, specially after editing some data. 

The most remarkable thing is that each changed row will appear twice in the XML. On the bottom is a section between <Diffgr:before> tags, there you will find the row's data as it was originally read. The row being modified or inserted is marked with attributes. During the edit session a maximum two versions of the data will be kept--the original as it was read and the actual as it is after the last changes in the row.

This is the XML as it will be sent back to the Web Service and as it will reach the Update method of the DataAdapter which now has a lot of information to commit all modifications, additions and deletions to the database. To get an idea how this is done I am going to explore a multi-user scenario.

DiffGram's and Multiple Users

The client application is working offline with the data. It could take quite some time before the client will return the modified data to the Web Service. In the meantime another client can read the same data and start working with it well. What will happen when the two, possibly different, updates return to the service?

If you inspect the update SQL statement of the DataAdapter it looks quite complex

What this query does is check the original value of every field against the actual values. In the data returned in the XML diffgram every row can be identified by the idField. And the original values of all fields are in the diffgram as well. The DataAdapter can identify the row and can check if the row was changed since it was read. If it was changed the Update method throws an exception and will not update the record. The Web Service will pass the exception message back to the client and that is quite informative.

Using this mechanism reliable multi-user database applications can be built, even if the connection between data on the client and data on the server is lost for quite some time.

Conclusion

Thanks to many tools built in VS.NET I have had the time to dive deeper into complex data traveling between a Web Service and its consumer. To use the full possibilities of the XML diffgram I have created  a consumer with .NET. The diffgram is a Microsoft format but thanks to the standard way in which XML data is structured a diffgram can be created by any XML aware programming tool. Personally I have had some experience with a Delphi consumer, Delphi does understand XML very well but to return an XML diffgram takes some programming effort.