Cloudfreeday's Blog

If you’ve read some of my other posts you’d know that I’m using a program called NetLimiter to track the network activity of Carbonite online backup.

On a Vista installation, NetLimiter stores its logs in the following location: C:\Users\All Users\Locktime\NetLimiter\2\Stats\nlstats-*.dat

This data is important to me as I want to track network usage over an extended period – not only for Carbonite but also for other applications. Hence, I thought it a good idea that these files were backed up – just in case of crash of my hard drive or other disaster.

“The status of each of these files according to Carbonite is ‘File is pending backup when not in use by another program’.”

Despite this being user data, in a user folder, and not in one of Carbonite’s special exclusion zones (see my post “What should be included in your backup?” for more information about Carbonite’s exclusion list), the files were not being automatically backed up.

Over 48 hours ago I manually selected these files for backup. With the exception of a few reboots, my computer has been on and connected to the internet the whole time.

Guess what? Carbonite hasn’t backed up any of these files! For the record, there are only 164 files, totalling a tiny 1.18 MB. The status of each of these files according to Carbonite is “File is pending backup when not in use by another program”. This message doesn’t inspire much confidence in me about Carbonite’s ability to backup these files.

Now, isn’t Carbonite supposed to be able to backup open and locked files? I’m pretty sure the answer is “yes”, but it would seem that the correct answer is “sometimes”.

This is a copy of the script error caused by the Carbonite backup application.

Regarding open or locked files, Carbonite has previously said to me “Carbonite does not directly access files, but uses a Windows Service called virtual Shadow which makes a copy of the file then Carbonite backs up the copy.” I’ve checked Event Viewer and am not seeing any errors relating to Shadow Copy (also called Volume Snapshot Service or VSS).

I’ll contact Carbonite to see what it has to say about this lack of backup!

Oh, one more thing… the Carbonite application also shows the following script error when attempting to backup these files.

I couldn’t quite think of a good title for this post – “update” isn’t quite accurate as there’s no real change to the upload speeds from my computer to the Carbonite backup servers.

Carbonite bandwidth usage as at 16 October 2009 21:04

Check this out… for days on end (with the exception of the occasional spike of a few seconds duration which is an aberration more than anything) the upload speeds have consistently been just under 32.25 KB/s. My connection is capable of a lot more, I assure you.

Also, if I was umm, “maxing out my connection”, you would expect that the speeds to Carbonite would change over time as other activities on my computer took some of the available upload bandwidth. The fact is that I have ample bandwidth for Carbonite to run at full speed and for other regular computer activities. Carbonite uploads have been limited – but not by me or by my connection.

Yesterday Carbonite managed to backup a measly average of 25 MB per hour. For the 19 hours thus far (today) the average is 29 MB per hour. As stated in one of my earlier posts, this is barely enough backup capacity to cope with the daily changes to my Outlook PST files.

I would love to hear about the experiences of other Carbonite customers – especially the following types of users:

• You already have stored with Carbonite greater than 100 GB, and or
• You have in the past 2 weeks uploaded to Carbonite a significant amount of data, say, greater than 50 GB.

It really would be beneficial if you have some before and after statistics so you can point to a change in behaviour (if any) of the Carbonite application.

Carbonite has publicly stated that it manages bandwidth, and I guess with a service claiming to offer “unlimited” data storage, it must manage bandwidth availability and costs, but I guess that the question is whether this “bandwidth management” (or throttling) is fair and equitable, and still consistent with that which is advertised.

Update on Saturday, 17 October 2009 at 00:52

For some, a still picture isn’t quite enough, and I guess that’s fair enough.

In this 1 minute video you’ll see not much action – time moves on, but the slow uploads remain. I guess what it does show you, much better than a still image, is the consistency of the upload speed that I’ve mentioned in a number of my posts.

Now, if during that 1 minute period my internet connection was under strain from the other activities on my computer, you would expect to see some fluctuation in the Carbonite uploads as the available bandwidth is redistributed to the applications that need it.

The fact is, I have plenty of available upload capacity – I just wish that Carbonite would use it.

Update on Saturday, 17 October 2009 at 17:44

Carbonite bandwidth usage as at 17 October 2009 17:42

During the backup of my Outlook PST files I thought that I’d take a peek at NetLimiter to see how “fast” Carbonite was sending off my data. Boy, did I receive a surprise!

From the already slow speeds of around 30 KB/s discussed in this post, the speed – all of a sudden – dropped to about 12 KB/s.

Just so you know – no nothing else changed on my computer on or around that time. This is clearly a result of bandwidth management by Carbonite.

Had this issue been caused by my connection or other local factors the change in speed would not have been so dramatic nor would it have stepped down as evenly as it did.

Carbonite bandwidth usage as at 17 October 2009 17:55

This new low upload speed remained in effect for some 13 minutes. Whilst writing this post, the speed went back to the regular slow speed – as you can see.

I haven’t heard from Carbonite in a few days – I just hope that they’re not “parking” my correspondence as they did in the first half of this year when I also questioned Carbonite upload speeds.

At this rate, I would be lucky to be able to backup 30 MB per hour. What a sorry state of affairs!

I’ve previously talked about Carbonite failing with large files (here Carbonite seems to fail with large files) and that I’ve written to Carbonite seeking its assistance. I wanted to give you more information about this whole scenario.

Today I spent some time to repeat the Carbonite-large-file-failure problem. Along the way I took various screen shots to assist with my explanation of what goes on. This is what I did.

Carbonite was nearing the end of backing some recently changed files. All was running normally on my computer.

I selected an AVI video file, 6.3 GB in size, for backup by way of right-clicking in File Explorer, then from the Carbonite menu choosing “Back this up”.

Carbonite menu in Windows Vista File Explorer

What follows is an enormous amount of sustained hard disk drive activity. Presumably this is Carbonite encrypting and encoding the first portion of the file for transmission and, I guess, to be expected.

Then, all uploads from Carbonite will cease. Yes, completely! Carbonite remains in this no-upload state until you next reboot – I’ve previously tested this overnight and yes, there are hours and hours of zero upload from Carbonite. You can see the upload ceasing see this from the screen print.

Here you can see Carbonite uploads going to zero... that's right, no data is being backed up!

Worst of all, not only does Carbonite cease to upload anything, the application takes CPU usage to an incredible 50%. This fluctuates up and down by 1 or 2 percentage points, but aside from that it remains at 50%. As with the zero uploads, this high CPU usage remains until you deselect the large file from your backup and reboot. Simply deselecting the file from the Carbonite backup menu will not resolve the high CPU usage (or the associated zero-upload issue, for that matter). 

Throughout this whole process the Carbonite application shows that it is uploading the file. The reality is far from the truth  – and NetLimiter data proves this. Aside from making your hard disk drive look busy and from taking your CPU usage into high levels, Carbonite is not doing much at all – certainly it is not backing up your file.

Carbonite application status window

In response to my detailed email to Carbonite about this issue, customer service replied with a stock standard response – I know it’s a standard response as I’ve seen the same (or very similar) text quoted on other websites. Carbonite did not properly review my correspondence and it did not address my issue. This is what they said:

Now, I’m only at 102 GB – nowhere near the 200 GB mark – the point at which Carbonite starts to throttle uploads “allocates less” bandwidth.

I have sought more information from Carbonite – let’s see what happens second time around.

So, is this large file backup issue more widespread? Is it a deliberate attempt to discourage people from backing up large files? Something doesn’t smell right about this.

"Carbonite... has an algorithm"

I found this undated comment from Mr David Friend, Chief Executive Officer of Carbonite on http://www.onlinebackupsites.com/Carbonite.php

Given the apparent upload Carbonite throttling issues I’ve previously discussed in my blog, Mr Friend’s comments simply don’t ring true. I’ll explain why.

Based on what Mr Friend stated in his post it could be inferred that upload speeds will change over time depending on “users who happen to be online at that time”. Over any 24 hour period you would expect the number of online Carbonite users to go up and down significantly – especially as the day dawns across the USA (where I expect the great majority of Carbonite’s customers are located).

Now, let’s look at my upload statistics for the last 19 hours or so – over that time you would expect the number of online Carbonite users to change significantly from hour to hour – and for Carbonite’s algorithm to distribute that “available data center bandwidth equitably across all users who happen to be online at that time”. That is, you would expect Carbonite upload speeds to change from minute-to-minute from hour-to hour.

Carbonite upload statistics for 11 October 2009 (generated by NetLimiter)

Looking at the statistics maintained by NetLimiter, you will see that the Carbonite upload from my computer has been fairly constant over that whole time. If you’re not convinced, the statistics for all of yesterday tell a similar story!

Pretty paltry when you consider that as recently as a few weeks ago I was uploading over 4 GB per day!

Did the number of online Carbonite users remain constant over that whole time? That seems improbable. People go to sleep, they switch off computers, they reboot computers and internet connections go up and down. So why then was my upload to Carbonite so constant?

It seems to me that Carbonite limits the bandwidth for individual users – that would be the logical explanation for why I’m not seeing any change in upload speeds over the period discussed.

I’ll challenge you – see if you can get hold of 50 GB worth of files – different files – maybe you have some movies or something – and set them to backup. Make sure you have some way you can monitor the upload speed and amount from your PC to the internet. As I said, I use NetLimiter, however you can also use statistics that your internet service provider might make available. Just remember that your provider’s data will include all data uploaded from your connection – not just that uploaded by Carbonite.

I would say that as you near the end of your 50 GB upload you will see dramatic drops in your upload speeds. When this happens, write to Carbonite and ask them why your upload speed was dropped – let’s see what sort of response you receive.

Carbonite should come clean on what it actually does to control uploads – is the algorithm simply broken, does it even exist, or does it have other means for dealing with “connection hogs”?

After all, Carbonite can’t be “unlimited” if it is limiting you.

These days there are a lot of cloud services being touted by all sorts of folk – some we know little about yet we trust them with some of our most personal data – more about security another time.

In this post I’d like to talk about bandwidth – that is, the amount of your internet connection these services consume.

If you’re going to use cloud-based products and services then you’re going to need some way of getting your data to the cloud and back. Whether it’s online backup, photo storage or even email, you have to have an internet connection.

Many countries still don’t have unlimited internet connections readily available to residential users – or they are prohibitively expensive. Worse still, some internet service providers count uploads as well as downloads in any calculation they make about your usage. Or, once you have exceeded your limit, rather than limiting your connection to a slower speed, some providers have excess usage charges – upload more than your plan allows and it will cost you real money on your next bill!

Skype 4.1 usage as shown in NetLimiter

Bandwidth limitations pose a real issue for those who are unable to access an unlimited internet connection. It means keeping a close eye on what you do online, keeping a close eye on others who may be sharing your connections (e.g. family or friends) and it means forking out cash when you exceed your rate plan’s limit.

So how do you monitor the data use of particular applications? I mentioned in an earlier post that I would discuss the nifty application I’m using to monitor Carbonite (and other applications on my computer).

I have researched a number of applications which can monitor your bandwidth usage and found that the best one which could monitor usage down to an application level was NetLimiter. It also retains detailed history for comparison of usage over time.

You can find information about NetLimiter at its website www.netlimiter.com. There’s even a 28‑day free trial. I liked it so much that I purchased a license.

Out of interest, here are some interesting statistics from today about the bandwidth usage of Skype. I was always worried about Skype usage, but as you can see, my fears were unfounded.

Skype & MSN logos

In fact, Skype used less bandwidth than Windows Live Messenger. Skype used 8.1392 MB (data up and down) over the period shown, whereas Messenger used 9.2326 MB over the same period. During this period both applications were not being used actively, that is, they were idling.

Services like online backup require that you have a speedy connection (to get your data up there initially and to get it back if and when your computer or hard disk fails) and unless you have an unlimited connection, you better make sure that you’re not up for hefty fees if you exceed your upload or download limits.

If you’re going to be using cloud-based services, you better make sure that your internet connection (and perhaps your wallet) can support the use of such services.