Just last month, Microsoft announced that their Windows Azure Virtual Machines were no longer considered a pre-release service. In other words, that was the official notification from Microsoft that they feel their Virtual Machines offering is ready for enterprise class deployments. In fact they even offer uptime guarantees if you employ certain round-robin and/or load balancing deployments that help mitigate the downtime in your cloud environment.
Essentially the Virtual Machines offering on Windows Azure equates to a virtual dedicated server that you would employ from most hosting companies. The only different with the Windows Azure platform, like most cloud-based offerings, is that you need to serve as your own system admin. This is not web hosting for business owners but for tech geeks. In other words, it works perfect for guys like me.
Or so I thought.
Different Shades of White
As I learned tonight, there are differences between the various cloud offerings that are not easy to tease out of the hundreds of pages of online documentation touting how awesome a service provider’s cloud services are. Sure, there are the metrics. You can compare instance sizes in terms of disk space, CPU, and bandwidth. You can comparing pricing and the relative costs of operating your server on each of the cloud platforms. You can even get the background information on the company providing the virtualized environment, getting some clue (though never a clear picture) of where the servers are physically located, how many servers they have, how secure the environment is, and more.
At the end of the day they all look very similar. Sure there are discrete elements you can point to on each comparison spreadsheet you throw together, but in the end the differences are relatively minor. They pricing is similar. The network and server room build-outs are similar. The support offerings look similar. When all is said-and-done you end up making a choice based on price, the reputation of the company, the quality of the online documentation, and the overall user interface experience (UX) that is presented during your research.
After a lot of research, and with quite a bit of experience with Amazon Web Services, all the cloud based offerings were very similar. Different shades of white. In the end I decided to try the Microsoft Windows Azure offering. Microsoft has a good reputation in the tech world, they are not going anywhere, and as a Microsoft Bizspark member I also have preview access and discount services.
My decision to go against the recommendations I’ve been making to my clients for years, “Amazon was one of the first, constantly innovates, and is the leader in the space”, was flawed. Yes, I tested and evaluated the options for months before making the move. But it takes an unusual event to truly test the mettle of any service provider.
Breaking A Server
After following the advice of a Microsoft employee that was presented in a Windows Azure forum about Linux servers, I managed to reset the Windows Azure Linux Agent (or WALinuxAgent) application. No, I did not do this on a whim. I needed to install a GUI application on the server and followed the instructions presented. It turns out that Microsoft has deployed a custom application that allows their Azure management interface to “talk” to the Linux server. That same application DISABLES the basic NetworkManager package on CentOS. To install any kind of GUI applications or interface you must disable WALinuxAgent, enable NetworkManager, install, disable NetworkManager, then re-enable WALinuxAgent. The only problem with the instructions that are published in several places is they omit a very important step. While connected with elevated privileges (sudo or su) you must DISABLE the WALinuxAgent (waagent) provisioning so that it does not employ the Windows Azure proprietary security model on top of your installation. If you do not do this and you log out of that elevated privs session y ou will NEVER have access to an elevated privs account again.
Needless to say, you cannot keep an enterprise level server running in this state. Eventually you need to install updates and patches for security or other reasons.
As I would learn, there is ZERO support on recovering from this situation.
Support versus support
In the years of working with Amazon Web Services and hosting a number of cloud deployments on their platform, I had come accustomed to being able to gain access to support personnel that actually TRY to help you out. They often go above-and-beyond what is required by contract and try to either get you back on track through their own efforts of at least provide you with enough research and information that you can recover from any issues you have with limited effort. Amazon support services can be pricey, but having access to not just the level one but also higher level techs is an invaluable resource.
The bottom line is that Microsoft offers NO support services for their Linux images, even those they provide as “sanctioned images”, beyond making sure the ORIGINAL image is stable and that the virtual machine did not crash. Not only do they not have any apparent means to elevate support tickets, as it turns out there is NO SUPPORT if you are running a Linux image.
Clearly Microsoft does not put this “front and center” on ANY of their Windows Azure literature. In fact, just the opposite. Microsoft has made an extended effort in all their “before the purchase” propaganda to try and make it sound like they EMBRACE Linux. They go out of their way to make you feel like Linux is a welcome member of their family and that they work closely with multiple vendors to ensure a top-quality experience.
Until you have a problem. At which point they wash their hands, as is evident in this support response along with a link to the Knowledgebase article saying “Linux. Not our problem.”:
Hello Lance, I understand your concerns and frustration, but Microsoft does not offer technical support for CentOS or any other Linux OS at this time.
Please, review guidelines for the Linux support on Windows Azure Virtual Machines: http://support.microsoft.com/kb/2805216
While the lack of support and the inability to regain privileged user access to my server is the primary concern that has me on the path of choosing a new hosting provider, there have been other issues as well.
A few times in the past several months the WordPress application has put Apache in a tailspin. This consumes the memory on the server. While that is not necessarily an issue with Windows Azure, the fact that the “restart virtual image” process DOES NOT WORK at least 50% of the time IS a big issue. Windows Azure is apparently overly-reliant on that dreaded WALinuxAgent on the server. If it does not response, because memory is over-allocated for example, the server will not reboot. The only thing you can do is press the restart button, wait 15 minutes to see if it happened to get enough memory to catch the restart command, and try again. Ouch.
The Azure interface is also not as nice as I first thought. While better than the original UX at Amazon Web Services, it is overly simplistic in some places and downright confusing in others. Try looking at your bill. Or your subscription status. You end up jumping between seemingly dis-jointed sites. Forget about online support forums. Somehow you end up in the MSDN network, far removed from your cloud portal. I often find myself with a dozen windows open so I can keep track of where I was or what I need to reference, lest I lose my original navigation path and have to start over. Not too mention the number of times that this site-to-site hand-off fails and your login is suddenly deemed “invalid” mid-session.
So once again, I find myself looking for a new hosting provider. Luckily I recently made the move to Windows Azure and not only have VaultPress available to make it easy to relocate the WordPress site but also Crash Plan Pro to get all the “auxiliary” installation “cruft” moved along with it.
Where will I go?
In my mind there are only two choices for an expandable cloud deployment running Linux boxes. Amazon Web Services or Rackspace. I’ll likely end up with Amazon again, but who knows… maybe it is time to try the legendary support at Rackspace once again. We’ll see. Stay tuned.