Bandwidth monitoring

[server dummy]

Is there a way to set a limit for users' bandwidth usage but not have their sites automatically shut down if they exceed it? I'd like to charge my customers for bandwidth overages, but not have their sites shut down...that seems like it would drive customers away.

Also, is there a huge difference between the bandwidth reported by cpanel and the actual bandwidth used by a user? I'm worried about bandwidth cost...

[Atjeu]

There is definitely a difference in actual bandwidth used and what cpanel reports - in whm it actuall tells you that on the bandwidth page. Its usually small like 5% less then actual but depending on what the user is doing it could be more.

Cpanel is either coming out with or has already come out with a way to "oversell" bandwidth with cpanel - you can oversell disk space already.

[Zoltan Orc]

It seems to me it's not just 5% but WAY more... I just checked last month's bandwidth stats in the new billing system and compared it to the WHM bandwidth stats... WHM says some 28 gigs and the billing system says over 62 gigs - what's up? :(

[Atjeu]

its highly unlikely your whm stats reset on the same day and time our billing system does - our billing sys revolves around your billing date. The server you are probably talking about has quite a few large and regular peaks - remember per gb billing on bandwidth is in + out = total - whm shows outgoing only. Even more dramatic is our cpanel virtual server which whm says used 19.46 gb for the month of january - actual stats 82.41gb! clients send a lot of email outgoing from the server? whm stats dont include smtp, or dns for that matter... Thats why there is a big red warning on the whm stats page in cpanel... if you want accurate stats of total usage install hotsanic or mrtg and you will get exactly the same results as our billing sys does...

[Zoltan Orc]

The difference between billing month and calendar month is but 3 days. OTOH the fact that WHM doesn't count incoming traffic did elude me. BTW what everything can be incoming traffic? Incoming mails and FTP uploads are easy to guess, but are there maybe other significant categories? I wonder e.g. how much incoming traffic is generated by web-based forums, compared to the outgoing traffic generated by them.

Back to WHM, I conducted a test with an account that doesn't have any HTML traffic to speak of (maybe 10 KB/day). I uploaded some 30 megs or so by FTP and then downloaded it and another 10-20 megs. WHM still shows absolutely nothing of it as of now, 2 days later - although I use ProFTP exactly because WHM claims it can count that but not PureFTP.

I have HotSaNIC but it doesn't tell me which client used how much. Can mrtg do that or does it give only full server stats, too? How can I calculate the real bandwidth used by a customer?

Just as a feedback, it'd be nice if you'd give a small note to new cpanel server owners that they shouldn't believe in WHM numbers and count with at least double the bandwidth displayed there, i.e. if they sell a client 5 gigs in a month, they actually sell more like 10 gigs, and prices should be calculated accordingly to prevent money loss.

[Noldar]

All Cpanel servers come with bandmin installed. Just go to http://www.anydomainonserver.com/bandwidth/

Bandmin tracks all traffic in and out of your network card, so it should give you a fairly accurate picture of the amount of bandwidth that you are using. It can only break it down to the IP level, so unless you give each of your users their own IP you won't be able to track individual sites.

You'll probably want to password protect the bandmin htdocs directory so that not just anyone can view the page. The directory is located at /usr/local/bandmin/htdocs.

Richard

[Atjeu]

mrtg can be setup to do it by ip as can hotsanic - hotsanic can do it by domain but then it breaks down and is not as accurate... hotsanic will measure traffic on the nic regardless of the type of traffic so its much more accurate - bandmin for our own cpanel virtual only measure 44gb for jan but its actual usage was 149gb. Why the diff on ours? because we have some ftp sites on the server that are setup as unix users and are not part of cpanel and so bandmin is not accounting for all of that traffic - whm is even less - the only true counters are switch port or nic counters. They dont care who or what but they increment everytime an octet passes... That can be done on your server the same as on the switch. We have meaured simultaneously on both for tests and found them to be identical (of course they would have to be) - unfortunately that does not help in trying to track the who and how much.

Incidentally, we identified a bug - sort of - when we reboot our switches they counters reset to zero which causes the billing program to increment the number of rolls by 1 - this leads to an addition of bandwidth of the amount of 0 to 4 gb each time the switch is rebooted - usually we dont reboot the switches but a few days ago we updated the os in all the switches (caused a reboot) and then rebooted them later that day so we have adjusted everyones bandwidth accordingly by just subtracting the max amount that it could have been overed. Now when we know we are going to reboot a swtich we can make the adjustment before rebooting - for some people whos counters were close to rolling there was little to no extra - only for those whose counter just rolled was there any dramatic amount - the counters roll at 2 to the 32 power or 4gb.

[server dummy]

How much total transfer does the average customer use? In other words, how many users could be housed on a server with 100Gigs of monthly xfer?

[Noldar]

Actually bandmin is totally seperate from CPanel/WHM and does not even look at any of it's configuration files. It is a seperate package and you can install it on any Linux server with iptables. It tracks all traffic in and out of the server through the nic card and will pick up any bandwidth usage whether it is a virtual user set up through CPanel or a unix user. It adds an accounting rule to your iptables, so you must make sure it is the first rule. If there are any drop rules before the accounting rule then it will miss some traffic. I'm going to install some additional bandwidth monitoring software to try and validate bandmin's accuracy and I'll let you know how it goes.

Richard

[Atjeu]

Everyones different - we have servers with 1 or 2 websites using a terrabye and servers with 500 users using less then 100gb -

Yes we know bandmin and the whm stats are separate things so in reading our last post when we mention whm we are talking about the whm stats and not bandmin...

Hotsanic is a great choice to measure and graph traffic at the nic... hotsanic.sourceforge.net

[Zoltan Orc]

cPanel 6 Stable and Release versions are out since last night (this far only Edge was out and I'm not updating to that), and as of 6.0 cPanel/WHM has a new, supposedly more accurate bandwidth measuring system, among other things outgoing SMTP traffic is now supposed to be counted for every user.

[Zoltan Orc]

Just updated to 6.0 stable... I'm not sure whether it was so already or only since the update, but that account I told about earlier on (see further above in this topic) now shows 263 MB for this month, which is just as strange. Hope DarkOrb is going on tweaking it...

[Atjeu]

There are posts in the cpanel forum about the bandwidth reporting issues with 6.

[Digitalhost]

I am waiting patiently to get Cpanel V6 , but everyday more problems are arising for people who bolted at the gate.

One feature I am looking forward to is restricting or even stopping people sending emails out as "nobody" from the server, at last people will have to use their own bandwidth instead of ours.

We are in the same situation as you people one of our servers constantly show around 98 gigs for the full month in cpanel and bandmin shows 119 gig but billing came in around 150+

I don't know who or how people are using the extra 30 gigs, even our email stats come in around 4 gigs for the month so still 25 gigs going somewhere, could it be ftp ?

[Zoltan Orc]

FTP bandwidth measuring seems to be completely broken in cpanel 5 (see my test described further above). I also have now a new account from Monday I set up for FTP only. There were a couple of FTP transfers this far but WHM still doesn't show any traffic for the account.

Also, something rather strange, I've got a domain running on a Webmin server and a couple of subdomains of it on a cPanel server. All DNS entries are handled by the Webmin server which runs the main domain and nearly all subdomains, rerouting only the files and mail subdomains to the cPanel server. For months, only these subdomains showed any traffic... for this month, WHM now shows 1.2 gigs for the base domain, which should be impossible as there are no files in there, you can't connect to the cpanel server through the main domain name, etc. etc.

Well I've upgraded to cpanel 6 once the first Stable release was out, so now there is no way back - my only option is hoping DarkOrb will hurry up fixing all those probs.

[Digitalhost]

I read on the www.cpanel.net (http://www.cpanel.net) forums that reverting back to V5 and then upgrading to V6 again solved some issues.

In Particluar the bandwidth measurement problem.

worth a try?