With the emergence of XML, it is now feasible to construct intelligent Internet agents that can interact reliably with multiple, diverse hosts. XML acts like the ASCII of the Internet, allowing interaction between hosts regardless of their operating systems, database managers, or data formats. Add ASP and standard data formatting options such as Cascading Style Sheets (CSS) or Extensible Stylesheet Language (XSL) to the mix and impressive network applications can be written without even a passing nod to the Java language, C++, sockets, or other complex tools.    In this article, I'll walk through a multitier application for Computer Finder, a fictitious company that offers users a Web page for specifying desired computer features, such as RAM, processor, and price. When the form is submitted, Computer Finder steps through its list of vendors, sending a query to each one based on the user-specified parameters. It then assembles the results and formats them for display to the user.    Along the way, I'll demonstrate a number of important technologies, including: Multitier application configuration ASP employed as a business services tier application (most multitier applications use COM components in this middle tier) Real-time query and retrieval from multiple sources, rather than maintaining a centralized, possibly out-of-date database Database retrieval using ADO Use of XML to standardize data from diverse sources Manual conversion of XML to HTML XML Document Object Model (DOM) manipulation with JScript® Formatting XML using XSL at the client and server The Architecture    Computer Finder's system architecture is illustrated in Figure 1 . The basic flow is as follows. First, the user accesses Computer Finder's search form using a Web browser. Figure 2 shows an example of the HTML form presented to the user. The user specifies the features she wants for a computer system, such as RAM and disk size, as well as a price limit. Then the user clicks on the Submit button, sending the form to Computer Finder, which plays the role of an intelligent agent.

Figure 2 Computer Finder's Search Form

Computer Finder steps through its list of participating vendors, sending each one a request based on the user's parameters. Each vendor's site uses the specified parameters to query their database, formatting results as XML for return to Computer Finder. Computer Finder accumulates the results from all the vendors and sends formatted results of the search back to the user. Figure 3 displays an example of the results that would be presented to the user.

Figure 3 Search Results

The user services tier is based on HTML, with some CSS for formatting. The Web page doesn't use any script, but script could be added to do some local work such as verifying that numbers are entered into numeric fields.    While I've seen many examples of multitier applications that utilize XML, they all used the Java language or COM on the business services tier. However, as I'll demonstrate here, fairly complex business logic can be realized using ASP for the business services tier. In the Computer Finder app the business services tier is written with JScript and includes XML DOM manipulation. The data services tier also employs ASP in JScript that uses ADO to access the vendor's ODBC database.    Most ASP applications use VBScript since it is processed at the Web server and does not care which Web browser is used. However, I prefer to use JScript because it allows me to construct utility functions that can be included in both client and server pages. In my opinion, JScript is more powerful than VBScript in many aspects, although I do miss the simplicity of VBScript functions like Trim. Of course, all of the popular server components used with VBScript, such as ADO, are available to JScript as well.    The computer Document Type Definition (DTD) for describing computer features is illustrated in Figure 4. While a real-life DTD for computers would have a lot more information, this one includes enough complexity to create powerful searches. Along with the expected elements with attributes, the DTD also has a text node for the description element, attribute value lists for several elements such as operatingsystem, required and implied values for attributes, and a multiple element level under inputoutput. These features will aid in demonstrating in-depth DOM manipulation. Figure 5 shows the code for the complete computer DTD. The User Services Tier    The Computer Finder search form (compuser.htm) is a standard HTML form that uses HTML tables for layout and some CSS for display formatting. For instance, a style class for labels is defined so I can experiment with and quickly change the appearance of form field labels.    The most important part of compuser.htm is the HTML form's target, compbusi.asp:

<form name="ComputerOptionsForm" action="compbusi.asp"> <input type="hidden" name="HTMLCaller" value="compuser.htm"><table> <tr> <td><table border="0"> <tr> <td class="LabelStyle">Price</td> <td><input type="text" name="Price" size="10" value="2000"><span class="UnitsStyle"> (maximum)</span></td> </tr> . . . </table> </form>

The compbusi.asp target is the business service tier application, which is the heart of the intelligent agent processing. A complete listing of the user services HTML page is shown in Figure 6.    In Figure 2, the Display selection at the bottom of the form lets the user select a method for displaying the result. This is clearly out-of-place; you would usually let the ASP target page check the user's browser capabilities and use the most appropriate method. However, in this application I will be demonstrating a number of different formatting techniques. I'll delve into these format options later. The Data Services Tier    The data services tier application (compdata.asp) will necessarily vary for each participating vendor. This ASP page's job is to take parameters from Computer Finder's agent and query the vendor database, then format the result as XML according to the computer DTD. The process is straightforward: Retrieve the search parameters from the calling page's query string (BuildSQL). Step through the search parameters, creating a SQL statement that matches the vendor's product database structure (BuildSQL). Execute the SQL statement using ADO (BuildXML). Step through the result rows of the database query, creating the XML (BuildXML).    One interesting twist is that the BuildSQL function allows for some variation in the exact values for arguments. For instance, if the OperatingSystem argument's upper-cased value contains at least SERVER or NTS, then Windows NT Server is assumed. The argument names themselves must match exactly, however.    After constructing the SQL statement, ADO (via JScript) is used to query the database and create the XML response. First, the XML header and <computers> top-level tag are written to the response object. Then, the vendor's product database is opened:

objConn = Server.CreateObject("ADODB.Connection"); objConn.Open("XYZComputers");

The SQL statement is executed, and the application steps through the returned rows:

objResult = objConn.Execute(strSQL); if (!objResult.EOF) { objResult.MoveFirst(); while (!objResult.EOF) { . . . //write the corresponding XML to the Response object . . . objResult.MoveNext(); } } objConn.Close();

The columns from each row are then written out according to the DTD:

Response.Write(" <computer model=\"" + objResult("Model").Value + "\" price=\"" + objResult("Price").Value + "\">\n");

An interesting deviation you should be aware of is that the vendor's Web site, stashed in the WEB_SITE global variable, will be added to the returned data as the website attribute of the vendor element:

Response.Write("<vendor name=\"XYZ Computers\" " + "website=\"" + WEB_SITE + "\"/>\n");

A complete listing of the data services ASP page is shown in Figure 7.    Computer Finder would likely offer vendors a skeleton, or at least a bundle of common functions, to make construction of this page easier. Regardless, it is not very difficult if the vendor can access their product database with ADO.    While constructing the data services tier, I thought it would be neat if a tool existed that allowed you to specify a mapping between XML and an ODBC database. The tool would accept this mapping along with the XML and database schema, then spit out conforming XML. The Business Services Tier    The business services tier application (compbusi.asp) is the most interesting—and most complex—part of the example. This ASP program's job is to receive a query request from the user, send the request to all of its known vendors, retrieve and combine results from the vendors, and format the result for the user.    The main body of compbusi.asp starts off by determining the URL of the page containing the form so a Return To link can be added on the results page:

strReturnURL = Request.ServerVariables("HTTP_REFERER");

After determining the requested output format by looking at the DisplayType form field value (more on this later), compbusi.asp gets down to business by instantiating an XML parser object:

objXMLDocument = new ActiveXObject("microsoft.xmldom");

It uses the FileSystem server component to read the list of vendor URLs from a file on the server, sending the user query to each one and accumulating the XML query results:

objVendorFileSystem = new ActiveXObject("Scripting.FileSystemObject"); objVendorFile = objVendorFileSystem.OpenTextFile(VENDOR_FILE); while (objVendorFile.AtEndOfStream == 0) { strURL = objVendorFile.ReadLine(); if (strURL.length > 0) { objXMLDocument.async = false; strURL = BuildURL(strURL); objXMLDocument.load(strURL); if ((objXMLDocument.parseError.errorCode == 0)) { AddXMLBody( objXMLDocument.documentElement); } . . . } }

The vendor URL that was read in specifies the vendor's domain and data services tier ASP page, such as http://www.xyzcomputers.com/compdata.asp. The BuildURL function is used to construct the full URL for the vendor by concatenating the query arguments (which came in via compuser.htm) to the base URL. This complete URL is used in the load method of the parser object, creating a parse tree from the XML results sent back by the data services tier ASP program. Finally, AddXMLBody walks the parse tree and appends the XML text to the strXML global variable.    Why bother parsing the returned XML, and then just turn around and recreate the XML text? There are several good reasons. First, it verifies that the XML returned by the vendor is well formed. Second, it assures that there are no unexpected elements or attributes in the vendor XML. Third, it allows the vendor XML to be easily manipulated, if necessary.    A lesser reason is that it serves to illustrate recursively walking the parse tree, as shown here in pseudocode:

AddXMLBody(node) loop through child nodes of node if child node is not an element node, then skip it get attributes of child node add the element opening tag to strXML loop through child nodes of this child node if child node is a text node, save its value and break end loop call AddXMLBody with child node (recursion) add any text node value to strXML add the closing element tag to strXML end loop

After closing the vendor URL file, the final task is to send the results to the user. As mentioned earlier, a field on the HTML form allows the user to specify the format option. While such an option would never really be presented to a user, I included it here so different formatting alternatives can be examined and tested. In real life, compbusi.asp would check the user's browser capabilities and determine the best method of formatting the result.    The complete source code for the business services ASP page is shown in Figure 8. Formatting the Results    The business services tier application can format the vendor query results for presentation to the user via several methods. First, it can use plain text to display the XML as a text string. This is useful for debugging. The business services tier can also turn the XML into HTML for browsers that cannot handle XML. In this case the HTML specifies a CSS file in a <LINK> tag for displaying the results. It can also send the XML file back to the user. While the result will appear similar to using the plain text option, this method is significantly different in that it relies on the user's browser to format XML files as it sees fit.    Another option is using CSS to format the XML. This is not as powerful as using XSL, but is worth examining. The XML specifies the stylesheet processing command

<?xml-stylesheet type="text/css" href="compuser.csx"?>

so the XML and CSS files are sent to the user's computer for transformation. Unfortunately, a bug in either Microsoft Internet Explorer or ASP prevents this from working correctly. Using a static XML file that references a CSS file will work, but if the XML is created dynamically with ASP, the CSS file will not be sent back with the XML file.    You can use XSL to format the XML at the user's computer. The XML includes the stylesheet processing command

<?xml-stylesheet type="text/xsl" href="compuser.xsl"?>

so the XML file and the XSL file are sent to the user's computer for transformation. Alternately, you can use XSL to format the XML at the server with the transformNode function. The result is that HTML is sent to the user. XML and XSL can therefore be used regardless of the browser's capabilities.    All of these formatting options are illustrated in Figure 9.

Figure 9 Result Formatting Options

Displaying the XML as HTML plain text requires the following steps: Add the XML header to strXML. Add the XML data string to strXML. Add the XML footer to strXML. Add the HTML header to strHTML. Add strXML enclosed in <plaintext> tags to strHTML. Add the HTML footer to strHTML. Send the complete strHTML to the user.    Transforming the XML to HTML involves these steps: Add the XML header to strXML. Add the XML data string to strXML. Add the XML footer to strXML. Create a parse tree from the complete strXML result. Add the HTML header with a CSS file reference to strHTML. Use AddHTMLBody to convert the XML to HTML. Add the HTML footer to strHTML. Send the complete strHTML to the user.    The AddHTMLBody function walks the parse tree, creating an HTML table structure from the data. Unlike the AddXMLBody function shown earlier, AddHTMLBody does not recurse, but simply loops through the second-level elements of the tree. It's safe to do this since I created this XML from vendor-supplied XML.    Sending raw XML back to the client is simple: Add the XML header to strXML. Add the XML data string to strXML. Add the XML footer to strXML. Send the complete strXML to the user.    To return XML that is formatted with CSS: Add the XML header to strXML. Add the XML data string to strXML. Add the XML footer to strXML. Create a parse tree for the complete strXML result. Use MassageXML to prepare the XML, modifying the parse tree. Clear the existing strXML. Add the XML header that references the CSS file to strXML. Use AddXMLBody to create XML from the modified parse tree, adding it to strXML. Add the XML footer to strXML. Send the complete strXML to the user.    The MassageXML function serves to modify the XML data so CSS can operate on it. Unlike XSL, CSS cannot create tables, hyperlinks, and so on. This function turns attributes into text nodes enclosed in HTML table tags and adds an attribute for making the vendor Web site into a hyperlink. As with the AddHTMLBody function, I just loop through the second-level elements in the parse tree rather than using recursion.    This example illustrates that making XML get along with CSS may actually be more work than either translating the XML to HTML or using XSL. However, it does serve as good sample code for XML DOM manipulation, showing how to use methods such as appendChild and createTextNode in MassageXML.    These steps transform the XML with XSL at the client: Add the XML header with an XSL file reference to strXML. Add the XML data string to strXML. Add the XML footer. Send the XML to the user.    To transform the XML with XSL at the server and return HTML, do the following: Create a parse tree from the XSL file. Add the XML header to strXML. Add the XML data string to strXML. Add the XML footer to strXML. Create a parse tree from the XML file. Transform the XML parse tree using the XSL parse tree. Send the resulting HTML to the user. The transformNode method of the parser object takes care of the transformation from XML and XSL to HTML.    The last aspects of formatting are the stylesheets themselves. The compuser.css file (see Figure 10) is the CSS file used to format HTML in the method that directly converts XML to HTML. Also, the methods that transform XML with XSL add a reference to it in the resulting HTML. Finally, it is used in any HTML error pages returned to the user. In essence, it is the base file for colors, fonts, and other display basics.    The compuser.csx file (see Figure 11) is the CSS file used for the method that formats XML with CSS. I used a nonstandard file extension .csx to signify that it was CSS for XML.    The compuser.xsl file (see Figure 12) is the XSL file used for the methods that transform XML with XSL. The XSL file is fairly simple. It builds a table with three rows for each computer element: vendor name and URL link, computer description, and list of computer features. However, there are a couple of specifications in the XSL file that deserve further explanation.    First, I want the output data sorted by price, from lowest to highest. In the <xsl:template match="/"> section, I could add:

<xsl:apply-templates select="computers/computer" order-by="+@price"/>

However, this would sort the prices alphabetically, so $500 would actually follow $1000. I need to sort numerically, but there's a dilemma: the DTD does not allow specification of data types. (One of the main features of the newer XML Schema approach is that elements can be typed.) XSL does offer a workaround via data type casting, so I cast price as a number:

<xsl:apply-templates select="computers/computer" order-by="+number(@price)"/>

The second point of interest in compuser.xsl is how a hyperlink to the computer vendor's Web site is added to the HTML produced by the XSL. To do this, simply create the <A> tag with an HREF pointing to the desired URL:

<xsl:template match="vendor"> <xsl:value-of select="@name"/>: <A> <xsl:attribute name="HREF"> <xsl:value-of select="@website"/></xsl:attribute> <xsl:value-of select="@website"/> </A> </xsl:template>

This XSL code results in this HTML:

<A HREF=http://www.xyzcomputers.com>XYZ Computers</A>

Setting Up the Application    To set up and run the application described in this article, follow these steps using the code that can be found at the link at the top of this article: Copy the xyzcomp.mdb database to the PC serving as the vendor's server. Set up a System DSN with the name XYZComputers that points to xyzcomp.mdb. Create a new Web site on the vendor's server, and copy the file compdata.asp to it. Create a new Web site on the agent's Web server, and copy the compbusi.asp, vendurl.txt, compuser.htm, glass.gif, finder.gif, computer.dtd, compuser.css, compuser.csx, and compuser.xsl files to it. Set the URL in vendurl.txt to point to the URL for the data services ASP page (compdata.asp) on the vendor's Web site. At the user's PC, use Internet Explorer to open the compuser.htm file on the agent's Web server.    To make things easier, the vendor and agent Web sites can be on the same PC. To really simplify things, you can install all elements on the same server and run Internet Explorer there as well.    To simulate multiple vendors, just replicate the vendor URL in vendurl.txt one or more times so that the database will be queried multiple times. Conclusion    The flexibility of XML will certainly drive creation of agent applications like the one described here. One missing ingredient, however, is the creation and publication of standard XML schemas so businesses can openly interact with each other without going through the tedious process of setting up schemas for each relationship. The BizTalk™ initiative, spearheaded by Microsoft, is dedicated to publishing XML schemas for coordinating business-to-business electronic communications. See the sidebar "The Buzz about BizTalk" for more information.    The solution presented here shows how ASP and XML can be combined to create powerful multitier applications. Even better, perhaps, is the fact that server-side scripting with ASP further simplifies such solutions, since no additional language (such as the Java language) is needed at any point in the chain.    One caveat of this architecture is the use of XMLDOM.load, which uses WinInet. This is not necessarily the best fit for server execution since it is limited by default to only two concurrent connections to a given client (see Knowledge Base article 237906 for more information). To scale to the high load and concurrency requirements of the server, this should be rearchitected to use either a third-party or custom HTTP COM object using Winsock.    As the example stands, the performance of dependent Web servers would be tied to the performance of the vendor's server. Thus, the developer's app will block until the XMLDOM.load call returns. An even better suggestion would be to use a custom HTTP object in conjunction with MSMQ.

For related information see: XML Developer Center at: http://msdn.microsoft.com/xml/default.asp. Also check http://msdn.microsoft.com for daily updates on developer programs, resources and events.

From the February 2000 issue of Microsoft Systems Journal. Get it at your local newsstand, or better yet, subscribe.