Serve clients’ specific protocol requirements with Brazil, Part 2 | InfoWorld
How to support XML applications with the Brazil project
In Part 1 of this series, I introduced the concept of a developer-friendly experimental Web server technology called Brazil. The Brazil technology supports additional functionality via handlers, the simple Brazil Scripting Language (BSL), and a global dictionary of system and user properties. (For more information on BSL, properties, and handlers, refer to Part 1 of this series.)
The technology also supports languages that facilitate the rapid creation of XML content. In this month’s installment of this series, I will explain how to represent different types of data in XML while still supporting those applications that lack XML ability. I will apply XML to three example applications’ data: detailed realtime weather data, stock data, and data from other applications or devices. In addition, I will demonstrate how to use other languages, such as Python and Tcl scripting, with the Brazil Web server. This article will focus on XML generation, with future articles covering XML consumption.
Read the whole series on Brazil technology:
- Part 1: Learn how to build an application server that can deliver data to clients requiring different protocols
- Part 2: How to support XML applications with the Brazil project
- Part 3: Economically sustain PQA, UP.SDK, and J2ME with the Brazil project
- Part 4: Build multicast-aware apps with JRMS
- Part 5: Manage users and content with Brazil
- Part 6: Plug Jini, BeanShell, and JAXM into Brazil
Benefits of XML
Numerous Webpages combine formatting and data. One of the primary benefits of XML is that it facilitates the separation of data from the formatting of that data. Good Web applications will support XML while providing support for those that don’t.
Let’s begin with a small example that demonstrates the coupling of data with formatting. Let’s say we’re interested in the collection and display of weather data. To make the temperature bold in an HTML table, we could write the following code:
<td>Temperature <bold>76,5 </bold> </td>
To generate the page, eventually the data and code must be combined. For example, the above code segment could be realized in BSL by the following:
<td>Temperature <bold> <property temperature> </bold> </td>
Listing 1. Combine data and formatting in HTML via the typical approach
If you need to place the temperature on a page, the above approach is much easier than creating HTML each time by generating the content with a procedural programming language that requires recompilation for simple changes. You also don’t have to search for a location for the temperature; simply put the <property ws.sample.Temperature> tag anywhere on the HTML page where you would like realtime weather data, and the Brazil technology will rewrite the page so that it will display the data when a browser displays it.
In prior articles, I configured the Brazil Web server to interface with a realtime weather station and provide Web data in the typical manner — content plus data together — as shown in Listing 1. For display to users, this approach is suitable, but more and more new Web applications consist of data from multiple Websites. Requiring developers that work on these new types of applications to parse HTML can lead to complications. To create a new service, developers would have to parse the data to obtain the weather information from site A to combine with site C’s data. A process that separates the formatting from the content would solve these complications.
Click on GetSample.html at our weather station ( ) to use BSL to display weather station data. It will produce the following output:
weather.sampleRate 5000
weather.sample.Time 23:26:32 PM
weather.sample.Temperature 73.34
weather.sample.Time.UTC 970025192844
weather.sample.WindDirection NOT CALIBRATED
weather.sample.WindSpeed 0.00
Listing 2. Use BSL to display weather station data
(Note: All of the examples in this article are available at https://www.brazilhandlers.com:9090/Applications/WeatherStation.)
The HTML page that utilizes BSL with the Brazil technology to generate the data in Listing 2 follows:
<TABLE>
<foreach name=a match="(weather.sample.*)" >
<tr>
<td> <property a.name.1> </td>
<td> <property a.value> </td>
</tr>
</foreach>
</TABLE>
Listing 3. Generate the HTML from Listing 2
By substituting property values with real values in realtime, BSL improves the separation of formatting information and data. The BSL handler processes the simple lines above, and any properties matching weather.sample.* are assigned to the variable a and are available for page placement as shown by <td> <property a.name.1> </td>. The 1 represents the matched string as specified in most regex implementations.
The above technique is useful for generating your own pages, but quickly becomes tedious when you have to supply data to numerous users that all have different requirements. How do you accomplish this objective without staying up all night? You need to provide XML representations of the data and get out of the UI business — in other words, you need to separate your data from content with XML. You must be pragmatic here, however. Your boss will not think too highly of your intelligence if your application supports XML yet doesn’t meet the needs of those users lacking XML support. One of the strengths of the Brazil technology is its ability to live in both worlds, supporting both new and traditional technologies through task-specific handlers.
A formal XML primer would be beyond the scope of this article. Many books providing detailed examples are available to guide you (see Resources for help with XML). To generate XML tags, you need to examine your data elements; usually the nouns will be elements. In our example, temperature, wind speed, wind direction, precipitation, and barometric pressure are all nouns. For our weather station example, I am producing tags to simplify the XML. The sidebar at the Brazil Website features a real-world XML definition of weather data that defines a common language for the myriad of weather reports coming from ships, buoys, balloons, and manned and unmanned stations in different formats (see Resources for direct link). Before defining your own XML tags, search the Internet for preexisting definitions. Basically, you will search for a Document Type Definition (DTD).
Realtime weather data: A simple XML example
As explained in Part 1 of this series, our weather station can deliver data in a number of formats. Included among the supported formats is HTML, which features layout information, and a simple property format; applets consume data stored in the latter format to use in other applications. Data can also be exchanged via XML, which, as shown below, can be created by BSL.
Listing 4 defines a DTD for simple weather data. If you want to define your own XML tags, choose a DTD that defines the tags’ values and specifies whether the elements are strings or tokens, and whether a specific element is mandatory or optional. Consult the XML specification for more details.
Several experts suggested the style featured in Listing 4, though some believe that XML DTDs should define tags that make it easier for computers to parse the tagged text. Specifically, developers often disagree whether to use elements and/or attributes. An element is the fundamental logical unit of an XML document. All content falls between elements. An attribute contains additional information about an element. Some designers prefer to place all the weather readings in one element with many attributes so that Temperature is an attribute of weather sample. More traditional programmers would use Temperature as an element. Some users find the single element with many attributes easier to read and easier for a computer to parse. The supporting Website in Resources contains examples of both, as BSL and Brazil support both versions. Here, I feature Temperature as an element.
<! ELEMENT MeteorologicalSample (Lat, Lon, Elev, Time, Temperature,
WindSpeed, WindDiection)>
<!ELEMENT Lat (#PCDATA)>
<!ELEMENT Lon (#PCDATA)>
<!ELEMENT Elev (#PCDATA)>
<!ELEMENT Time (#PCDATA)>
<!ELEMENT Temperature (#PCDATA)>
<!ELEMENT WindSpeed (#PCDATA)>
<!ELEMENT WindDirection (#PCDATA)>
Listing 4. A DTD for simple weather data
Below is an example of how BSL can be used to generate XML:
<TAG>?xml version=<property weather.xml.version>?</TAG>
<TAG>DOCTYPE <property weather.xml.simple.title> SYSTEM <property
weather.xml.simple.dtd></TAG>
<TAG><property weather.xml.simple.title></TAG>
<Lat> <property weather.location.lat> </Lat>
<Lon> <property weather.location.lon> </Lon>
<Elev> <property weather.location.elev> </Elev>
<Time> <property weather.sample.Time.UTC> </Time>
<Temperature> <property weather.sample.Temperature>
</Temperature>
<WindSpeed> <property weather.sample.WindSpeed>
</WindSpeed>
<WindDirection> <property weather.sample.WindDirection>
</WindDirection>
<TAG>/<property weather.xml.simple.title></TAG>
Listing 5. The .xml file, populated with real data via BSL
The following code segment is the result of using BSL to populate an XML template with real values. Note that each value — including Temperature — is an element, not an attribute.
<?xml version=1.0?>
<DOCTYPE SimpleMeteorologicalSample SYSTEM
<SimpleMeteorologicalSample>
<Lat> 41.222093 </Lat>
<Lon> -73.317894 </Lon>
<Elev> 100 </Elev>
<Time> 970025759193 </Time>
<Temperature> 72.93 </Temperature>
<WindSpeed> 0.00 </WindSpeed>
<WindDirection> NOT CALIBRATED </WindDirection>
</SimpleMeteorologicalSample>
Listing 6. The result of applying BSL to Listing 5
Realtime weather data: A real-world XML example
As I am no weather expert, I refer to a comprehensive weather data DTD that I found on the Web. More details on this DTD can be found in Resources.
Listings 7 and 8, which are located on the supporting Website https://www.brazilhandlers.com:9090/Applications/WeatherStation/JavaWorld, demonstrate how to use Brazil technology to comply with existing DTDs quickly.
Stock data
An overabundance of Websites sputter a constant stream of stock data. Take a peek underneath the browsers at the streams flowing into your machine. In most browsers, you can do this by selecting View Page Source. What you find is mind boggling: a massive amount of free-form scripting that combines user interface code and data. It works, but the data is not easily reusable. All Web content features this general problem; it is difficult to separate the content from the formatting in a deterministic way. A deterministic technique always provides you with a correct answer, whereas a heuristic technique does not. For example, a deterministic technique would never fail in finding a stock price on a Webpage, even if the table layout changed. A heuristic technique would fail since it, by definition, is only a best guess. Often that is enough, but don’t bet your life on it.
Moving content from one Website to another is difficult to do if there are no published definitions, as there are for XML-tagged data. What if you want to combine stock prices from source A with news data from source B in some new way? To accomplish this task with today’s data producers, you would need to parse the HTML pages and hope that the layout of those pages doesn’t change. This is an ad hoc solution best left to experts that can worry about the capricious nature of Websites. An alternative is to work with Websites that provide data independent of data formats. In this example, we would take data from an established Website, parse it in realtime with either Tcl or Python, and generate XML data by issuing the following URL:
The Brazil Web server implements a Tcl script to query a remote Website that assumes it is delivering data to a browser. Instead, the Tcl program takes the data, parses it, and places the results into the request property, which makes the data available to the BSL scripting facilities.
Using a Brazil server configured as described in Listing 10, you would receive the results illustrated in Listing 9 for data that is normally displayed on a Webpage. This format is simple to parse and used in Java applications or applets. In fact, the Property class includes a method that loads this data directly. Some developers feel that the Property class does not satisfy their needs. They forget that you can provide your own Property class that can abstract several different concrete implementations. If you have a million properties, you might want to store them in a database, or some type of linked list or tree-data structure, and possibly provide a regular expression interface to find the keys or values you are looking for.
The stock data below originates from a Web server that assumes it is serving an HTML user.
TERN.current=34.0625
TERN.date=9/26/2000
TERN.time=4:00PM
TERN.change=0
TERN.open=0
TERN.high=0
TERN.low=0
TERN.volume=0
TERN.bid=34.0625
TERN.ask=34.125
Listing 9. Output of a Tcl script that parses data originally targeted to a Webpage
Parsing a realtime data stream and generating XML is a much simpler approach. And, if the stream is already XML, then you’ve got it made. However, in many real-world applications, some interesting piece of data always exists on some other Website. (Please note that there are many legal restrictions on taking content and redistributing it.) Emerging satellite multicast technology can multicast audited, authentic financial data in realtime to the entire world in an XML format, driving the cost of realtime financial data to pennies.
The following paragraphs and examples demonstrate how to retrieve data from an existing Website and reformat it. The output of the Tcl script below produces the data in Listing 9.
Use Tcl as the scripting language
Many programmers use the Tcl programming language for scripting applications (see Resources for more information on Tcl). The Brazil project supports Tcl scripting directly in a Webpage. The procedure for using Tcl is very simple:
- Enable Tcl in your
configfile - Create a Webpage with the
<server language="tcl">tag
Below, you will find a typical configuration file for constructing a Brazil Web server that supports Tcl. This example assumes that other handlers are creating properties.
handler=main log= port=8081
main.class=sunlabs.brazil.template.TemplateHandler main.prefix=/
main.suffix=.html
main.templateClass=sunlabs.brazil.tcl.TclServerTemplate
main.cookie=tclserver
Listing 10. Configure the Brazil technology to support Tcl server-side scripting
The following code searches for a property called symbol. It then takes the values of symbol and iterates over them looking for "symbol.bid", "symbol.ask", and "symbol.price", producing an HTML table.
<HEAD><META HTTP-EQUIV="Refresh"
CONTENT="5"></HEAD>
<server language="tcl"> package require java set props [java::field
$request
props]
puts "<table class="legacyTable" border=2>" puts
"<tr><th>Symbol<th>Price</th><th>Bid</th&g
t;<th>Ask</th></tr>"
foreach stock [$props getProperty "symbols"] { puts "<tr>" puts
"<td>
$stock </td>" foreach attr "price bid ask" { puts
"<td>[$props
getProperty $stock.$attr]</td>" } puts "</tr>" } puts
"</table>"
</server>
Listing 11. Tcl code to extract and parse the source of the stock data
The above Tcl script obtains a Webpage that has stock data mixed in with formatting data. (In the future, we hope that this will be an XML page or an RMI service.) The Tcl script gives each value a fully qualified unique name and puts this data in the request object. The code below combines BSL and HTML to easily generate HTML for a user to read.
<table class="legacyTable" border>
<tr><th>STOCK</th>
<foreach name=th property=rta.sma.trackAttributes >
<th> <property th> </th>
</foreach>
</tr>
<foreach name=stock property=trackStocks sort>
<tr>
<td><property stock>
<if name=rta.sma.trackAttributes value="">
<foreach name=x list="name price pe">
<td><property [stock].[x]>
</foreach>
<else>
<foreach name=x property=rta.sma.trackAttributes>
<td><property [stock].[x]>
</foreach>
</if>
</tr>
</foreach>
</table>
Listing 12. BSL for putting data produced in Listing 11 on a Webpage
Use Python as the scripting language
Python is a new, fashionable language that Brazil also supports. The example below demonstrates how to configure a handler in the Brazil Web server to support Python.
handler=main log=9 port=8085
main.class=sunlabs.brazil.template.TemplateHandler main.prefix=/
main.suffix=.html
main.templates=sunlabs.brazil.tcl.PythonServerTemplate sunlabs.brazil.template.PropsTemplate sunlabs.brazil.template.BSLTemplate
main.cookie=Pythonserver main.debug=on
Listing 13. Python configuration file
Below is an HTML page that uses this handler to rewrite the page in the Python language:
<title>python server
test</title> <h1>test</h1> <server
language=python> print
"<br>Hello from port", server.props.getProperty("port") print
"<br>The
request headers are:<pre>", request.headers print
"done” request.props.put(“PYTHON”, “true”)
Boo Boo
in python interp
The end
Listing 14. Using Python — HTML example
Handling data from devices or from applications
A good Web-based service should provide information to users so that they don’t have to visit numerous Websites. Such services require Web applications that easily integrate with devices and other Web applications. In order to realize this goal, Web applications need to implement better mechanisms to exchange data. One such technique is XML, parsing existing content heuristically when you have no other choice. This type of content aggregation — taking content from different sources and synthesizing new content — is a unique ability of Brazil technology. It completes content aggregation via extensible handlers that allow content to pass through n levels of filtering until it reaches its desired form. This approach could suit many Web applications.
For example, consider the typical Internet search engine. Many search engines do not return their results in a format that is easy to use by other programs. In an ideal world, they would produce XML output directly. To overcome this short-term deficiency, we can parse a Website’s data and tag it with some XML — accepting the possibility that the parsing could break at any moment, either intentionally or due to a new revision — by using Brazil’s handlers to create Property objects. Application developers can then use that data to generate XML-tagged data for use in other applications. If all search engine companies tagged their data this way, one interface program could aggregate the results of a search across multiple engines and create a unified view of the results.
I am often frustrated with the lack of inter-Web application abilities. For example, my brokerage, bank, insurance, and credit card companies do not communicate with each other over the Web. When you add bill-paying services to the mix, it becomes more confusing. Many of us pay companies to simply open our envelopes and scan our bills in so we can read them as PDF files. In a more intelligent reality, companies could provide secure XML representations of my data and allow me to develop my own applications so that I would not have to use a portal that tries to act as a one-stop source. Perhaps personal portals provide the solution; they would combine and aggregate content on your behalf. The same idea applies to devices in your home that do not interact with each other. To have single sources of information and control, we need to improve the ways in which we define data when using applications that can’t combine data and content.
Conclusion
In this article, we discussed ways to use XML with the Brazil Web server to exchange content between applications. I see tremendous value in the XML approach and in the Java object approach easily supported by JINI and RMI technologies. “The Java Electronic Commerce Framework and Modern Protocol Designs” in Resources discusses this subject in greater detail.
In Part 3 of this series, I will build on the concepts introduced here — devices, scripting, XML — to introduce yet another scripting language and technique for supporting wireless applications. Part 4 will demonstrate how to support JINI infrastructure and RMI technology.
