Ran into a unique situation while updating my VVV box after a weekend of WordPress Core and plugin development at WordCamp US this past weekend. Today, after the formal release of WordPress 4.4 I needed to update the code in my WordPress trunk directory on the VVV box. Since I have other things in progress I didn’t want to take the time to reprovision the entire box. Though, as it turn out, that would have been faster.
The issue was when I tried to do the svn up command to update the /srv/www/wordpress-trunk directory and make sure I was on the latest code. The command failed, insisting that a previous operation was incomplete. Not surprising since the connectivity at the conference was less-than-consistent. svn kindly suggest I run svn cleanup. Which I did. And was promptly met with an “invalid device cross-link” error when it tried to restore hello.php to the plugin directory.
The problem is that I develop plugins for a living. As such I have followed the typical VVV setup and have linked my local plugin source code directory to the vvv plugin directory for each of the different source directories on that box. I created the suggested Customfile on my host system and mapped the different directory paths. On the guest box, however, the system sees this mapping as a separate drive. Which it is. And, quite honestly I’m glad they have some security in place to protect this. Otherwise a rogue app brought in via the Vagrant guest could start writing stuff to your host drive. I can think of more than one way to do really bad things if that was left wide-open as a two-way read-write channel.
Comment out the mapping in Customfile on the host server. Go to your vvv directory and find that Customfile. Throw a hashtag (or pound sign for us old guys) in front of the directory paths you are trying to update with svn. In my case wordpress-trunk.
Run the vagrant reload command so you don’t pull down and provision a whole new box, but DO break the linkage to the host directory and guest directory.
Go run your svn cleanup and update on the host to fetch the latest WP code.
Go back to the host, kill the hashtag, and reload.
Hope that saves you an extra 20 minutes surfing Google, or your favorite search service, for the answer.
This morning I spent over an hour trying to publish a new update of Store Locator Plus to the public WordPress extensions directory. It failed, multiple times. I assumed it was something wrong with our repository so I decided to move on to something else until the remote server was fixed.
My next task was to get the WordPress language translation tools into my dev kit so we can start providing better international support. I decided to fetch the latest language dev kit with subversion via the standard checkout:
svn co http://svn.automattic.com/wordpress-i18n/tools/trunk/
Here are a couple of the many failure messages I received back after about 10 minutes:
svn: PROPFIND of '/wordpress-i18n/tools/trunk': could not connect to server (http://svn.automattic.com)
svn: PROPFIND of '/wordpress-i18n/!svn/vcc/default': could not connect to server (http://svn.automattic.com)
My first assumption was that my virtual machine was having network issues. I reset the network, then shut down & restarted the virtual machine. No luck. I then tried directly from the host. Again no luck. I decided to go to the office and try it from there. Maybe my modem or router at the house was causing issues.
I get to the office and try again. Same problems. Odd, more Googling was in my future. After reading a lot of articles about proxy servers with svn (I don’t use a proxy) and doing all the “svn tricks” I know and that I could dig up online, I stumbled across an interesting post at Stack Overflow. This is what caught my attention:
I had a co-worker test this out on his home connection — he uses Comcast as well. He got the same error as I did. So it appears to be some Comcast-related issue specific to the WordPress svn repository. I was able to checkout other public repositories via http (e.g. from Google Code) just fine.
Huh, that’s interesting. I too could use SVN with a variety of other services. I also was using Comcast at the home office and on one-half of the network at the corporate office. So I decided to try a couple of things.
First, shut off the Comcast connection at the office and force my system to connect via the T1. Guess, what? It worked. The repo was cloned immediately.
Interesting. Second test, log in to our server in Michigan on multiple backbones, NONE of which are on Comcast. Hey, look at that… it worked immediately as well.
Back to the office services. Turn off the T1, turn on just Comcast. Instant fail. Well not instant, it waits for about 5 minutes then fails.
In addition to the failure to pull subversion content directly from the WordPress IP addresses, we have also found several other interesting things about Comcast Business Class Internet. Comcast is billing us for 50Mbps/10Mbps speeds at both my home office and our corporate office locations. We have NEVER been able to get anything close to that at either location. Our download speeds always seem to max out around 20Mbps/4Mbps at home and 26Mbps/6Mbps at the office.
Today, in an effort to understand what may be going on, we ran ShaperProbe from GA Tech. It turns out that at my home office we are dropping so many upload packets that many tools, including ShaperProbe fail. We also learned the Comcast is THROTTLING the incoming bandwidth at the home office to 17Mbps, less than HALF the 50Mbps promised and paid for. This is one method used to ensure all users have some bandwidth when they oversell a neighborhood. Ouch.
Comcast Speed Test Results
After the “network improvement” work that Comcast did this week our Corporate line is now crawling at 1/10th the advertised download rate. We are able to receive our packets from WordPress, but now we can’t get more than a handful of simple transfers going at the same time.
I am still doing research on this issue and will post updates here. However it is very obvious from the initial tests that Comcast is doing some sort of traffic shaping or other network manipulation on their business class services and it is “breaking the Internet”.
I’ll try contacting them, but I am 99.999% certain that whomever I get ahold of will be clueless. They usually are. In fact I bet the first thing they ask me to do is reboot my computer, then turn off the modem. Then they’ll bounce the modem remotely.
We have called Comcast Business services and we are quite shocked to have reached Terry at Business Class Services. She actually emailed us and is escalating the problem to a higher level tech and is going to chase this down for us today, on a Saturday of all days. Wow. That was surprising! Now lets all pray for some results as doing this proxy thing is a pain!
Comcast has found a routing issue and is working on it. We’ll see what happens.
12/17/2011 04:15 EST
Comcast claims the routing is fine. The problem, they claim, is on the WordPress servers. I made it clear via email this is not the case. The service works 100% fine when I switch all routing to/from the wordpress.org or automattic.com domains to go over the Windstream T1 v. Comcast business service. Looks like they couldn’t find a quick/easy answer and are back to their lame excuses and passing-the-buck.
12/18/2011 01:05AM EST
Comcast reps never pinged me back before they left for the day (10PM) as promised. Not surprised about that. “Dave”, the level 2 tech, said it is not a Comcast problem and that was that. Bah. Time to ratchet it up a notch.
12/21/2011 10:03AM EST Nobody ever called back or emailed us about this issue. We did receive two automated calls the past 2 days in a row that our service would be offline from midnight until 5AM for “network improvement
work. This morning our WordPress packets are now arriving intact. Slow as can be though. Our 50M/10M line is now clocking in at 5M/5M as noted by Comcast Speed Test.