So I looked it up on Wikipedia and found this little gem, out of the blue:
Systems that follow Fielding's REST principles are often referred to as RESTful; REST's most zealous advocates call themselves RESTafarian.
Rereading Wikipedia's summary the ideas in REST begin to make sense: build systems with URLs and send-command/return-contents protocols that are stateless and cacheable, instead of the usual remote procedure call/method invocation paradigm.
An interesting aspect of XML-RPC and SOAP is that you take a flexible format like XML that can easily support forwards and backwards compatibility and force a strict RPC interpretation down on top of it (if I understand things correctly, admittedly I'm no expert on web services). It may be easier than parsing the XML itself, but it is a poor man's version of a protocol. And why use XML when you're not really exploiting it anyway? Certainly not of performance reasons...
I have a feeling the requirement of a stateless protocol is a problem in many circumstances, though, having spent some time working on distributed file systems. Sometimes it's just better if the server can send a notification back to a client instead of the client having to poll. I believe NFS has left its much touted stateless approach with version 4.