Pete Lacey’s Weblog

My intent with this post was to discuss a transport protocol for Moxie, but it’s really not possible to discuss the transport without a notion of the system as a whole. Herewith, a high-level view of how I currently envision the Moxie system.

Moxie Overview

Moxie is (will be) an open, asynchronous, XML messaging system and nothing more. Applications will publish messages to a Moxie server and other applications will retrieve and process those messages. These receiving applications may publish a response back to that same Moxie server or to another Moxie server. The original application — or some other application — will retrieve these “responses” and process them in turn. Nothing terribly new here, except that many of you are probably thinking of a queue-based message oriented middleware (MOM) system, ala MQ-Series — I mean, WebSphere MQ, but I’m not. Instead, Moxie will implement a MOM system using the Tuple Spaces architecture. Even this is not new. Apparently RogueWave tried to do this a few years back with their Ruple product, but for some reason abandoned the effort. I have zero familiarity with Ruple, but it sounds a lot like what I have in mind. I hope its failure is not a reflection on the concept.

Back to Transports

Since the design of Moxie is fluid, I can’t lay it out for you as a fait accomplis. We’ll have to tackle it bit by bit, till a formal specification comes out at the end. For this post we’ll cover the transport. That is, moving messages from point A to point B.

The SOAP specification is transport neutral, and that’s a good thing. In other words, a SOAP system can use any transport to get a SOAP message from here to there. But let’s not kid ourselves, the world has translated “transport neutral” as “HTTP.” Sure, it works just as well over, say, Tibco Rendezvous, and there may even be some deployments that don’t use HTTP, but the technology is called web services for a reason.

I’m siding with the REST camp on this one. HTTP is not a generic transport, it’s an application protocol. And if you don’t abide by the semantics of that protocol, well, then you’re just tunneling. Of course, there was a good reason for using HTTP in the first place; it’s ubiquitous. But web services, by layering on all of the complexity of the WS-* stack, has rendered that point moot, and the REST architecture really owns this argument.

If not HTTP, then what? Well, it seems to me that Moxie needs an entirely new transport. Now calm down. I know the world doesn’t need a new transport, and its not very likely that the security guys are going to let it through the firewall, but bear with me, I think I have that covered.

It’s been said that if the Web was invented today it would fail because the security admins wouldn’t let HTTP through the firewall. I’m not sure I believe that. I think there’s a reason HTTP is allowed through (the first layer, anyway); it’s safe. HTTP is connectionless and stateless (though state has been grafted on), it has a few simple methods: GET, POST, etc., that are well understood, largely idempotent, and sent as clear text. In other words, barring bugs in the httpd implementation and poorly written web applications, there’s not much an attacker can do at the protocol level.

The Moxie transport protocol (the acronym MTP is already taken, so I’ll call it MXTP from here on) will emulate the connectionless and stateless nature of HTTP, but will take the protocol one step further. Recall from the Loose Fish post how it was noted that the WS specifications are simply moving existing opaque and/or proprietary transport protocols off the wire and into the message. Well, that’s what we’ll be doing here. The transport itself will be nothing more than a transient socket connection to the server. If the connection is a write, the server will accept the XML message (if it wants to), place an XML formatted response code on the wire, and disconnect. As Moxie is asynchronus, the server will not return any “meaningful” response. For reads, the server will accept an XPath/XQuery expression, and return a status code and all matching messages. I do worry about being pinged to death with reads, and there’s a possibility of having reads use a slightly different, persistent transport, but that has headaches too so it probably won’t happen. The message itself (to be discussed in future posts) will contain all the necessary header and body information. But response codes have to be thought about now.

MXTP Response Codes

Just as HTTP has methods: GET, POST, PUT, and so on, Tuple Spaces have operations. In its purest form these are: out (write), rd (read), in (take), and eval (start a process). I’m pretty sure I won’t be needing an “eval” operation, but am also pretty sure I’ll want something like a “notify.” Given this, lets see what JavaSpaces has to offer in terms of operations: write, read, take, notify. That looks good. I’m sure there’s a lot to be learned from looking at the HTTP status codes as well.

While it’s not possible at this stage to enumerate all possible response codes, let’s imagine a few so that we can get the XML format right. At the very least there should be an error number and an error message. Like HTTP we’ll glom related messages together in groups of 100.

Success messages first. I think it’s safe to say that we’ll need an “OK” response, lets take a crack at it.

<response>
  <code>200</code>
  <message lang="en">Message received</message>
</response>

That looks good. For reads and takes we’ll need a “no match” response — which is not an error.

<response>
  <code>201</code>
  <message lang="en">No messages matched your request</message>
</response>

We may or may not need redirection responses, depending on how URIs are exposed, but we’ll definitely need some error messages though.

<response>
  <code>400</code>
  <message lang="en">Message not understood</message>
</response>

<response>
  <code>403</code>
  <message lang="en">Client not authorized</message>
</response>

There’ll be a lot more status codes to come, especially error codes, which I’ll track on Moxie’s living design document. Right now, I just wanted to set some guidelines and try out an XML response format.

Also, from here on in, I’ll be building a reference implementation along with the design. This might take a while as I also plan to use it as a means of learning Ruby. I did order the Ruby book, however, and expect it to arrive tomorrow or the next day. I also need to get Subversion installed and figure out if Ruby has anything comparable to JUnit, etc., etc.

More to come

Updated for formatting

In my previous post on this subject I explained that the complexity of web services stems from 4 things:

• Some of the web service standards are simply flawed
• Some of the standards are flawed simply because they’re built on top of other flawed standards
• Some of the standards aren’t standards. There’s too much duplication of functionality amongst competing specifications. Aka, vendor wars
• Web services are complex because technology is complex

The last item was explained in that post. It’s the prior three that I’d like to start addressing now. This won’t be done by simply explaining things away, or by accepting our fate, or even by planting a flag and screaming. Instead, what I’d like to do is deconstruct web services the way they stand today, and reconstruct them from scratch. I want to do this while minimizing the impact on existing web service implementations, but I don’t think that’s entirely possible. At the very least we’ll keep the bath water even as we toss out the baby.

I do not have any of this work in some pre-existing form. I will be creating it here off the top of my head (though it’s been gestating for some time). I also anticipate doing some completely bone-headed things. But this is a living, iterative design, and I will be going back and correcting things as bad ideas become evident. I sincerely hope that others will discover this blog and provide their input into the design. I will be tracking the “new web services” on this page here: http://wanderingbarque.com/ws-reloaded.

In this post I’ll take the first baby step in reconstructing web services; doing something about the name. As a name “web services” is next to meaningless. Or, rather, it has too much meaning; it’s overloaded. Since I envision a message oriented XML-based Internet — ermm — exchange, the new system will be called “Moxie.” And, honestly, I just thought that up while I was typing.

Up next: Thinking about transports.

Tim Bray styles himself as the Loyal Opposition. Opposed, that is, to the rush towards web services and their attendent complexity. In a recent post he provided a balanced set of links illuminating the schism between the proponents and detractors of web services. Go read that to get up to speed on the debate and to sample not a small amount of vitriol.

If you are not inclined to read through all of those postings, I’ll summarize it here for you. Everyone agrees that sending standardized (XML) messages over the Internet is a good thing. Web service detractors, principally those in favor of the REST model, believe that they have the superior mechanism for doing this. While the web service proponents believe the same. In short, the RESTafarians think web services are too complex, and the WS crowd think REST is too simple.

And you know what? They’re both right!

Web services are complex, indeed, overly so. But a lot of that complexity is simply a reflection of the underlying technical challenges that the various WS specifications are attempting to address. As said elsewhere, web services are targeted at the business archictect/developer to whom reliability, security, asynchrony and the like are very, very important. Challenges simply not addressed in the REST model.

The REST model on the other hand, reflects the Web. Well, in fact, it is the Web, with all of the web’s simplicity, ubiquity, and power — and a dollop of XML. The RESTafarians prize simplicity above all, and are justly happy with the way their world works (though I have a few minor reservations).

So while REST perfectly solves the Internet messaging needs of the blogosphere, Technorati, and Yahoo, it is not sufficient for Citigroup, Ford, and AIG.

Now, on to that complexity. The seeming complexity of web services stems from a number of roots:

• Some of the standards are simply flawed
• Some of the standards are flawed simply because they’re built on top of other flawed standards
• Some of the standards aren’t standards. There’s too much duplication of functionality amongst competing specifications. Aka, vendor wars
• Web services are complex because technology is complex

In future posts I plan to address all of these items. Right now, though, I’d like to focus on the last one. As already mentioned, a lot of WS complexity is simply a reflection of the underlying complexity of the technology now being modeled in XML. Here’s the takeaway: The WS specifications are simply moving existing opaque and/or proprietary transport and wire protocols off the wire and into the message., and in doing so are opening up opportunities not previously available. While there’s not always a one-to-one mapping, maybe this table will help clarify things.

Once again, the various WS specs are mostly moving existing opaque technologies into XML, facilitating interoperability and intermediation. Furthermore, some of the WS specs are attempting to address technology challenges that the existing systems simply ignore. For instance, now that everyone speaks the same language, it would be nice if there was a standard means of discovering (at development time or runtime) what the configuration of a particular service is. Is it secure, if so how? Will it participate in a transaction? Where’s the interface contract (WSDL) it implements? Etc? From this we get WS-Policy and related domain specific policy languages (e.g., WS-SecurityPolicy), WS-Discovery, WS-MetadataExchange, and so on. Similar in many ways to protocols and APIs such as LDAP, JNDI, or DHCP.

So while all of this seems to be happening in a mad rush, and vendors are loosing imperfect and competing specifications on the public (the above are from Microsoft and IBM), one cannot say that WS specifications are complex without also saying that the sum of CORBA, RPCs, DCOM, RMI, PKI, SSL, Kerberos, MIME and S/MIME, PGP, LDAP/AD, Message Oriented Middleware, TP Monitors/XA, HTTP, HTML, CSS, SMTP/POP/IMAP, SNMP, object modeling, relational databases, application servers, and so on, and so on are also comple; which, admittedly, they are. The only real difference is that the WS versions of these protocols (not the implementations) are visible to the user.

Furthermore, while existing systems are inherently complex, one reason you don’t hear too much complaining about them is that the complexity is often hidden from the developer. It’s built into their tools and libraries and design practices. I should stress that the same is and will hold true for web services. The typical WS developer does have to know their craft, but they don’t have to know how each and every spec is implemented. For instance, there’s no good reason why WS-ReliableMessaging can’t be built behind JMS, and thus present no more complexity than MQ-Series does to the Java programmer. Similarly, there’s no reason for a developer to employ each and every one of these specifications; if reliability is not important, don’t use it. And remember, these specifications allow for business partners to participate. So, continuing our example, unlike MQ-Series, WS-ReliableMessaging allows, say, Visteon (if not the Internet at large) to reliably deliver messages into Ford’s systems, regardless of whether they use the same language or platform. That’s pretty powerful!

Well, I hope that helped a little. It’s not the end of the story, though. Yes, some complexity is inherent, but not a small amount of it is unnecessary. In future postings I hope to deconstruct the WS stack and build up a simpler, more sensible versions. In doing so, I want to keep compatibility with existing systems, but already fear that that won’t be fully possible. Not that it matters really as nobody knows me, and this blog is too new to get noticed. Besides, it’s not like IBM and Microsoft are going to build what I describe. It should be fun, though, and maybe educational.

Oh, as for the title of this entry. If Tim is the Loyal Opposition, then I, as someone who sees merit in both REST and web services, am, though an American, a loose fish.

I work for the Burton Group, a technology research and advisory company. I’m a consultant in Burton’s Application Platform Strategies service. One of the things I hope to do with this blog is provide insight into how our (to remain anonymous) customers are deploying technology. In particular, how they are approaching SOA, Web Services, XML, open source software, databases, J2EE, .NET, etc. Maybe I’ll have something to say on this tomorrow after I visit a with large university near Toronto, Ontario, Canada.

For now, though, let me tell you how Canada took me for $150CAD.

I flew this afternoon from Boston to Toronto. When I get up to passport control the following (summarized) exchange takes place between me and the passport agent.

“How long will you be in Canada?”
“2 days.”
“Business or pleasure.”
“Business.”
“What do you do?”
“I’m a computer consultant.”
“What’s your business in Canada?”
“I’m providing some consulting to [Large University].”
“Are you getting paid for this?”
“Indirectly.”
“Your company’s getting paid?”
“Yes.”
“Please step over to immigration.”

Long story short, it seems that if you come to Canada to do any work that you get paid for, you need to get a “work permit” at a cost of only $150 Canadian. When I asked the immigration agent what I could have said to avoid being fleeced (his word, by the way, not mine), he answered that I should claim to be in Canada for a sales call.

Next time I head North, I’m a sales guy. Call me, we’ll do lunch.

But this time it’s MarsEdit. Assuming this posts I think we already have a winner.

This is only a test — of the ecto blog editor and publisher.

Back in August the family moved into a new house, and that meant getting Comcast to run some new cable and Verizon to add two new phone lines. Long story short; Comcast did exactly what they were suppose to do - no muss, no fuss. Verizon, however, drop the ball completely. Shortly thereafter my Vonage equipment arrived.

I had Vonage installed and working in minutes. Some number of days later, though, my Internet service started degrading and then gave up the ghost entirely. I rebooted all of the networking equipment, which brought it all back to life, but a few days later the same thing happened. Then it happened again. At that time a whole bunch of things had changed in my setup. I’d received a new cable modem, I’d replaced my Linksys wifi router with Apple’s Airport Extreme, and I got the Vonage box. What was the culprit? I didn’t nail it down right away, but it was the Vonage box.

You see, the simplistic Vonage installation instructions really encourage you to install the device directly behind your cable modem/dsl line, which I did. Problem is, as I discovered after inordinate amounts of Googling, is that the router built into the Motorola VT1000 VoIP equipment is for the birds, and after a certain amount of usage the internal routing tables get all out of whack. This may be true for other devices that Vonage resells, but I have no way of knowing.

The solution is straight forward enough; arrange to have your Vonage box behind your router. You do have a router don’t you? Even if you only have one computer? Even if you’re not using wifi? If you don’t, click over to Amazon and get yourself one.

Putting your Vonage box behind your router is a simple matter of plugging it into the router as explained here. What those instructions don’t mention is that you will almost certainly have to set up port forwarding, though Vonage does explain how to do that here. And what those instructions don’t tell you is how to give your Vonage box a static IP address. You’ll need to do that too. Instructions can be found here. Of course, the address you use isn’t random, but must be part of the IP range in use on your internal network. Probably something like 192.168.1.xxx. Also note that the final, xxx, octet should not be in range of the addresses handed out by your DHCP server.

The long and the short of it is; if you are using Vonage, then do not let your VoIP equipment (at least the VT1000) also be your router/NAT/firewall/DHCP box. Instead use something better suited to the job like a Linksys, Netgear, or Airport router.

The Good Bit

However, if you are using an Airport Extreme (and maybe others), did you know that you can hook a device up to it, but put it in front of the firewall, thus skipping all the port forwarding stuff? That is, you can put a single device into a mini DMZ. To do this, you’ll have to give the Vonage box a static IP address. Then go to the Airport Admin Utility, select the main Airport tab, click “Base Station Options,” select the “WAN Ethernet Port” tab, and check the “Enable Default Host at” box. The first 3 octets of your internal network will be there, just fill in the final octet with whatever you set for your Vonage box. Now you have the Airport doing all that good NAT/router/DHCP stuff for you, and the Vonage box with a direct connection to the Internet, without port forwarding. You’ll lose any of the packet shaping (call quality) abilities of the VT1000, but we’ve been running this way for months and haven’t noticed anything.

I understand it’s possible to do this the other way round too. That is, to have the VoIP box downstream from the cable modem, and place your router in it’s DMZ. This will resurrect the call quality features of the VT1000. But my configuration seems more natural to me, and my call quality is fine.

If you’re a complete IP neophyte, then most of that didn’t make much sense. Leave a comment and I’ll help you if I can. Otherwise, find yourself a guru and have him/her do it for you.

If you’re reading this and there are few other posts, then you probably got here after stumbling on my mail server HOWTO and backing up a few levels in the URL. Well, welcome. Nice to meet you. Fact is, though, I just got WordPress installed, and it’ll be a little bit before I get into the flow of this whole blogging thing. When I do get my act together, however, the plan here is to talk about technology. It’s what I do. I have a professional career that at the moment has me circling Web Services, application deployment platforms, XML Schema, and the like. At home I’m an OS X and Linux guy. That said, I’m no luminary. I won’t spot the next del.icio.us or Skype before you do. What I can do is explain things, and, presumably, that’s mostly what you’ll find here; explanations. On occasion my wife, son, dog, town, or a particularly clueless service person may make an appearance.

Hope to see you soon