Blog of Hamlesh

Researchers at Bell Labs have managed to transfer optical data at the incredible rate of 16.4 Tbps over a 1,584 miles distance. That is 2.05 terabyte (2,050 gigabyte) per second, which is leaps and bounds ahead of what normal network equipment can currently handle. What 16.4 Tbps transfer speeds are capable of To give you some perspective of how blazingly fast such a connection is, here are some examples of how long it would take to transfer some common storage media over it:

• One DVD (4.7 gigabyte) – 2.3 milliseconds
• One Blu-ray Disk (50 gigabyte) – 24.4 milliseconds
• One 500 gigabyte hard drive – 244 milliseconds

Once the internet is capable of these kinds of transfer rates, almost instant backups and synchronization over the internet will be possible. Google could stop using FedEx Sometimes it’s faster to carry data on disks from one location to another (often called sneakernet) than transferring it over the internet. For example, Google uses FedEx to transfer massive amounts of Hubble space telescope data. It’s actually faster for them send the 120 terabyte of space telescope data with overnight delivery than transferring it over the internet.

[Google] sends actual physical disk arrays via regular mail, something they have dubbed, for fun, FedExNet. This allows them to get the data within 24 hours. To transfer the same amount over the internet in 24 hours, Google would have to be able to achieve transfer rates of more than 11 gigabit/s running constantly maxed out. On a regular 100 megabit connection, transferring 120 terabyte of data would take almost four months (111 days).

However, with the transfer rates that Bell Labs achieved, it would only take one minute to transfer those 120 terabyte of data. Google could probably live with that… ). Now all we need to do is get disks/arrays that can read and write data that quickly. The "home computer" will be dead at this stage - centralised/utility computing for the win baby!

This is a list of 14 of the largest social networks in the world, and how much downtime they have had so far in 2008.

Social Network	Home page (monitored)	Downtime in 2008 (until Feb 25)
Bebo	www.bebo.com	12h 28m
Windows Live Spaces	spaces.live.com	7h 25m
Friendster	www.friendster.com	6h 0m
hi5	www.hi5.com	5h 5m
Reunion.com	www.reunion.com	2h 55m
LinkedIn	www.linkedin.com	4h 0m
Classmates.com	www.classmates.com	2h 5m
Facebook	www.facebook.com	1h 35m
Orkut	www.orkut.com	1h 10m
Last.fm	www.last.fm	1h 10m
Xanga	www.xanga.com	45m
MySpace	www.myspace.com	25m
LiveJournal	www.livejournal.com	10m
Yahoo! 360	360.yahoo.com	5m

As we mentioned in a recent report, Bebo’s downtime has increased significantly lately and has had by far the most downtime of the 14 social networks we monitored for this survey. More than 12 hours of downtime in less than two months is a lot, and it could possibly be caused by the new open application platform that Bebo launched in December, allowing third-party developers access to its platform, Facebook style. It could be putting more strain on Bebo’s systems than they anticipated. The two giants in the field, MySpace (with 25 minutes of downtime) and Facebook (with one hour and 35 minutes of downtime), can both be considered to be within acceptable limits, especially MySpace. Social networks, just like any other websites on the internet, will occasionally suffer from downtime, either planned or unplanned. However, social networks have a different type of usage than most websites, with frequent visits from the same user and many page views per visit. Therefore downtime can often be even more noticeable and frustrating to social network users. A note about the monitoring: All monitoring was done using Pingdom’s uptime monitoring service. If a web page is not reachable, returns an error, or takes longer than 30 seconds to load, it is considered as down. Downtime is always confirmed from two geographically separate locations.

The social network site Bebo has for the last three months been experiencing a trend of increasing downtime. The popular social networking service had only five minutes of downtime in November and 30 minutes in December, but in January that increased to five hours and 20 minutes. Looking at Bebo’s downtime so far in February, the trend seems to be continuing. With less than half of February gone by, they have already had 3 hours and 14 minutes of downtime.

 

Image: Note that the February downtime only includes data until February 14.

 

On December 12, Bebo launched their Open Application Platform, based on the one from Facebook, allowing third-party developers access to their service. It is interesting that Bebo’s downtime has increased after this. It could be that the access from third-party applications is putting additional strain on Bebo’s platform, resulting in more downtime for the service.

 

Bebo is one of the largest social networks in the world, with 40 million users. According to Wikipedia it is the second most popular social network in the UK, and the third most popular in the US, second only to MySpace and Facebook. Hopefully the problems they are having will soon be resolved.

 

Note: The monitoring of Bebo’s website was performed by Pingdom’s uptime monitoring service.

There have been voices raised that we are running out of IPv4 addresses for some time now. So who has taken them? After discussing this at a coffee break here at Pingdom, we were curious. Are there are any “big spenders” who have allocated a huge share of the IP space for themselves? Yes there are.

When looking at the assigned IP blocks at IANA (Internet Assigned Numbers Authority), what becomes immediately clear is that the US Department of Defense has a significant number of large IP blocks. We only looked at /8 IP blocks, the largest blocks you can get, and there are 12 of them assigned to the US DoD and related organizations. Each /8 block holds 16,777,214 IP addresses, so the DoD have in effect allocated more than 200 million IP addresses. That should hold them for a while.

The closest any other corporation comes to this are Level 3 Communications and Hewlett-Packard, with two /8 blocks each. The DoD-owned IP blocks together with the 26 corporations and universities who have their own /8 blocks hold more than 671 million IP addresses.

These were all early land grabs, most of them made between 1991 and 1995.

Some notables among the companies with one /8 IP block are Apple (but no Microsoft in sight), IBM, Halliburton and the Ford Motor Company.

Owner	Blocks	~IP addresses
US Military (Department of Defense etc.)	12	201 million
Level 3 Communications, Inc.	2	33 million
Hewlett-Packard	2	33 million
AT&T Bell Laboratories (Alcatel-Lucent)	1	16 million
AT&T Global Network Services	1	16 million
Bell-Northern Research (Nortel Networks)	1	16 million
Amateur Radio Digital Communications	1	16 million
Apple Computer Inc.	1	16 million
Cap Debis CCS (Mercedes-Benz)	1	16 million
Computer Sciences Corporation	1	16 million
Deparment of Social Security of UK	1	16 million
E.I. duPont de Nemours and Co., Inc.	1	16 million
Eli Lily and Company	1	16 million
Ford Motor Company	1	16 million
General Electric Company	1	16 million
Halliburton Company	1	16 million
IBM	1	16 million
Interop Show Network	1	16 million
Merck and Co., Inc.	1	16 million
MERIT Computer Network	1	16 million
Massachusetts Institute of Technology	1	16 million
Performance Systems International (Cogent)	1	16 million
Prudential Equity Group, LLC	1	16 million
Société Internationale De Telecommunications Aeronautiques	1	16 million
U.S. Postal Service	1	16 million
UK Ministry of Defence	1	16 million
Xerox Corporation	1	16 million
	40	671 million

The table doesn’t include the blocks allocated for RIRs and other special purposes.

One thing that strikes you is the low number of ISPs on this list. This is most likely due to their late appearance on the internet when it was harder to reserve big IP blocks, forcing them to settle for smaller blocks.

That the US Department of Defense has such a huge amount of the IPv4 address space is partly due to historical reasons: They developed the internet in the first place (ARPANET) and have a strong vested interest in the evolution of the internet.

Of course, IPv6 is on its way which will end this discussion for good (or at least for a looong time), but with a bit of rearranging IPv4 could probably last us for quite a while yet.

A small note:
We only looked at IANA’s own numbers. We didn’t look at the Regional Internet Registries such as ARIN or RIPE, partly to limit the scope of the research and partly because we were only interested in the huge /8 IP blocks. There are only 256 /8 blocks, and many of these are reserved for special purposes.

In November we posted a list in our blog of old Apple Macs that are being used as web servers. Being an uptime monitoring service, we figured we would monitor their uptime and see how they were performing. The post actually ended up on the front page of Digg, which threw an unexpected amount of traffic in the direction of these old machines, even if it was secondary traffic via our blog. Most of them (not surprisingly) temporarily faltered under the extra pressure from Digg visitors, but we were really impressed by Paul’s Macintosh IIci, which hasn’t had a second of downtime, not even while the post was on Digg. This is quite an achievement for a server based on hardware that’s almost 20 years old. BSD + Apple = true love? The old Macintosh IIci won’t run the BSD-based Mac OS X, but Paul has managed to install NetBSD on it. Ok, it’s not Mac OS X, but it’s BSD. It seems it took him some effort to get it up and running, though. On his site Paul says:

It took a long time and was fairly complicated to install NetBSD, but finally I got it working, installed Apache, and hooked it up to the vast tubes of the interweb for your enjoyment.

So not only is Paul’s old Mac doing a great job as a web server, it runs an OS that is a close relative to the modern-day Mac OS X. Even though he has enhanced the basic hardware a bit, we here at Pingdom think that’s pretty impressive. You can kind of say that he’s given his Mac a modern twist. Paul’s Apple Macintosh IIci specification Introduced: September 20, 1989 CPU: Motorola 68030, 25MHz Ram: Upgraded to 128 MB OS: NetBSD Web server: Apache Upgrades: Cache card and 1 GB SCSI hard drive No downtime As we mentioned, during the time we have monitored it (since November 28), Paul’s Mac server has had no downtime at all. This is better than many web hosting companies. Yes, that’s comparing Apples (pun intended) and oranges, but it’s nevertheless an interesting observation. Maybe all hosting companies should host on Mac servers?

Digg has a problem. That problem is that the more users Digg gets, an increasing amount of the stories that reach the Digg front page are bound to be unavailable, brought down by the storm of visitors from Digg.

If Digg keeps growing it will automatically kill almost any site that reaches the front page.

Digg can’t keep using the same model they are now. It simply doesn’t scale. It will reach a point where it has so many users that the amount of traffic that comes from being on the front page of Digg is more than any normal site can handle. (You could argue that this is already happening.) As it grows, Digg will be knocking out larger and larger sites.

As you can see we divided sites into three main categories:

1. Sites on shared hosting accounts. Can handle a normal amount of traffic. The majority of sites on the internet are hosted this way, and a lot of these will have serious problems (i.e. crash) if they end up on the Digg front page.
2. Sites on a dedicated server. Can handle a lot of traffic, but can, and often do, run into trouble during the initial peak of traffic from the Digg front page.
3. Sites on multi-server setups. Handle anything you throw at them, but are basically only used by media portals, large news sites (think CNN.com) and large enterprises. Digg isn’t a problem here.

Sites with multi-server setups only make up a tiny part of the content available on the internet. Isn’t one of the main points of Digg to show niche content and find news before the big media finds it? This content is often driven by people who earn no or very little money and host their sites on modest setups.

Mirroring content isn’t the answer (legally speaking)

The so-called Digg effect is so common that it has given rise to a separate mirror service (Duggmirror) that makes copies of downed sites. The legality of this practice is highly dubious from a copyright standpoint, and isn’t run by Digg. There is even a statement by Digg-founder Kevin Rose from 2006 regarding the caching of content, which is quite interesting in this context:

“Digg should never take away traffic or cache content, should always push to site creator.”

Summary

The conclusion here has to be that if Digg wants to keep growing, and we assume they want to considering all the attention they have received from VCs, they need to somehow change their model. If they just keep going the following will happen:

1. Most of the websites on the front page of Digg will be unavailable, which means that…
2. The quality of Digg will suffer and it will be much less useful.
3. Some sites may not even WANT to be on Digg. It may simply be more trouble than its worth.

Perhaps the Digg crew is working on some solution to this problem. We don’t know. What do you think?

A small note on Digg’s users and visitor numbers

According to an article by John Graham-Cumming, Digg currently has about 2.7 million registered users, which is almost three times as much compared to a year ago. But of course, the majority of visitors to Digg will not be registered users, but simply people there to find interesting news. According to data from comScore, the Digg website had about 12 million unique visitors worldwide in December. As an example, last time we had a post on the front page of Digg, the dedicated server for this blog couldn’t keep up with the initial burst of visitors from Digg. This is kind of ironic considering we’re an uptime monitoring company…

We have posted pics of some truly messy data center cabling in the past, but this time we figured it was time to do the opposite: Show how some people have managed to organize cables into something close to art. Courtesy of Digital:Slurp. — Courtesy of ChrisDag. — Courtesy of mbm3290. Swedish colors! How could we NOT include it? — Courtesy of mmayo. — Courtesy of Jef Newsom. — Courtesy of Precision Fiber Optics. — Courtesy of tim d. — Courtesy of Network1 Cabling. (We’re not sure this is practical or not, but it LOOKS impressive…) — This post is dedicated to Daniel Norman, cable guy extraordinaire at Loopia.se.

This is a look at the state of Linux through the eyes of Google Trends, Google’s highly useful search trend analyzer.

The Linux distributions compared

Everyone says Ubuntu is the most popular Linux distribution, and boy does it become obvious when looking at it through Google’s eyes. With all the buzz, it shouldn’t come as a surprise that Ubuntu is king, and while the other distributions are experiencing a downward trend, Ubuntu keeps growing. It’s also interesting to see that Red Hat is firmly at the bottom when comparing these five distributions. (Google Trends only allows you to compare up to five different search terms, if you wonder why we didn’t include more.)

KDE or Gnome?

How about Linux desktop environments? KDE or Gnome, which one is more popular? Once again, let’s see what Google have to say. They’re pretty much equal in popularity, but it seems like KDE shows up a lot more in the news (the small graph at the bottom).

Linux beats Windows Vista

According to this comparison, Linux is actually beating Windows Vista. However, it appears that “Linux” as a search term is clearly on the decline. And just to tie in with the initial trend graphs for Linux distributions, how does the popular Ubuntu fare against Windows Vista? An interesting observation from the last two graphs, if you combine them, is that “Ubuntu” may soon be a more common search term than “Linux”. That would be quite a feat…

Conclusion

Though looking at search statistics can never give a complete picture, this gives an interesting perspective to how things are going for Linux, especially when viewed through the eyes of such a dominant and pervasive search engine as Google. Did we miss something blatantly obvious and should be shot on the spot? Tell us in the comments (but please don’t actually shoot us).

Apple is famous for its well-designed hardware and software. Much of their success is due to innovative products combined with smart marketing. They have always had their own way of doing things. If you’re an Apple fan you have probably seen that famous yellow post-it note. We’re of course talking about the “maintenance” page that Apple puts up at Apple Store every time they are about to release something new. And this is where the magic happens. It’s like one of those Pavlovian conditioned reflexes: When Apple Store is taken down, the web start to buzz. Will they release new products? What will they release? When will Apple Store be back up? They have managed to take something most people consider a big negative, i.e. downtime, and transformed it into something positive. Apple Store downtime is a delimiter, an indication of something new to come. It has become an automatic buzz creator. If Apple releases a new product, it will of course be talked about anyway, but with this additional marketing trick they have added another layer of anticipation, another thing to talk about and set off speculation. This has been going on for years. Apple has managed to create a massive PR machine at the cost of closing their online sales for a couple of hours, and they have done it so consistently that they have trained Apple users and journalists alike to react this way. They could basically just take down Apple Store and the word would spread automatically that something is about to happen. Who else would get this huge, positive reaction by taking down their online store for a few hours? The last example was last Tuesday, January15, when Apple released Macbook Air. During this launch Apple Store was down (for “maintenance”) between 15:20-19:40 CET and then again 20:05-20:20 CET, a total of 4 hours and 35 minutes according to Pingdom monitoring.

Any doubts that this is creating buzz?

• A search for ”apple store down” on Google gives 21,000 results.
• A search for ”apple store down” on Technorati in blogs with a lot of authority gives 43,238 results.

It seems pretty clear that this is an intentional, and quite ingenious, marketing stunt from Apple. No other online store takes down their site on purpose to update with new products. Even the most simple e-commerce system doesn’t need that. You really have to love them for pulling this off. Downtime is hated everywhere, but somehow everyone cheers when Apple Store goes down. Now that’s an accomplishment.

Pingdom Tools is a free set of web-based tools for webmasters and other web-curious people. We launched it this summer and it has proven extremely popular. So far, more than two million tests have been performed with its graphical website load time, ping and traceroute tests. We have just added a major improvement to this tool: The ability to save tests, which can then be accessed by anybody through a direct link.

Why is this so cool?

1. Because you can see exactly how a site was behaving at a specific point in time.
2. It becomes extremely easy to share the results of a test so others can examine it.

Since the AJAX-based graphical interface of Pingdom Tools makes it easy to pinpoint why a site is slow (or down, for that matter), a lot of people post screenshots of tests in forums and blogs. Web hosts can also use Pingdom Tools when communicating with their users, or vice versa. We wanted to give even more power to those users. With the new save and link features sharing this information will be easier than ever. Just save your test and you will have a permanent link you can send to anyone as proof. They will be able to see exactly the result you got and when the test was performed. A saved test is much better than just a screenshot, because you can properly examine, sort and filter the saved result. And if a site is down, you have the Ping and Traceroute tools to help you find potential connectivity problems.

See all previous test results

You will now be able to see a list of all previous (saved) tests of a website when you perform a test in Pingdom Tools. In March, you will be able to go back and look at tests that you or anyone else performed in January. If a website is tested regularly, you will have a really handy list of historical tests to look at. This gives you lots of opportunities to compare “before-and-after” results, to name just one possibility. With this feature Pingdom Tools has become a lot more powerful, and the more people use it, the more powerful it will become.

Share, share and share some more

We are very happy about all the positive feedback we have received about Pingdom Tools and that so many people are using it. We hope that these new additions will help people share test results among themselves to an even greater degree and that Pingdom Tools will keep acting as a great resource to the webmaster and web developer community (and everyone else as well!). Note: Currently all Pingdom Tools tests are performed from Dallas, TX, USA. More locations will be added in the future. Read more about how to use Pingdom Tools to analyze websites.