For one of my for-university projects/products, I need to provide a
web server (which will have other daemons running as well such as
FTPD, SSHD, crond, etc.) which will serve the entire university (but
obviously not everyone at the same time). It is primarily a web and
file server via HTTP. I'm guessing that at peak, it will need to be
able to serve 1000 or more people downloading one or more files at a
time. This number could be wayyyy more or wayyy less, I've never done
anything with servers before.
For this purpose (non intensive php pages, posgresql, and intensive
file downloading server), how would I go about
building/buying/commissioning such a server?
- I don't mind assembling it myself if I have to.
- Cost (to the university) should be reasonable (nothing SUPER expensive)
- Should be able to serve simultaneously (pontially wayyyy) more or
less people as described above.
- I'm thinking about 8TB worth of disks to start with in RAID 5
configuration (with parity disks and hot standby).
- What form factor should I go for? Full tower? Rackmount?
- How many ethernet ports and stuff?
- Will I need multiple servers to handle this volume? If so, how is
storage handled... leading to:
- Should I go for a storage area network (SAN)?
- How should backup be handled?
Answers? Tips? Comments/suggestions? There's a lot of other related
questions left out that I can't think of at the moment. Any help is
much appreciated
> For one of my for-university projects/products, I need to provide a
> web server (which will have other daemons running as well such as
> FTPD, SSHD, crond, etc.) which will serve the entire university (but
> obviously not everyone at the same time). It is primarily a web and
> file server via HTTP. I'm guessing that at peak, it will need to be
> able to serve 1000 or more people downloading one or more files at a
> time. This number could be wayyyy more or wayyy less, I've never done
> anything with servers before.
>
> For this purpose (non intensive php pages, posgresql, and intensive
> file downloading server), how would I go about
> building/buying/commissioning such a server?
>
> - I don't mind assembling it myself if I have to.
> - Cost (to the university) should be reasonable (nothing SUPER expensive)
> - Should be able to serve simultaneously (pontially wayyyy) more or
> less people as described above.
> - I'm thinking about 8TB worth of disks to start with in RAID 5
> configuration (with parity disks and hot standby).
> - What form factor should I go for? Full tower? Rackmount?
> - How many ethernet ports and stuff?
> - Will I need multiple servers to handle this volume? If so, how is
> storage handled... leading to:
> - Should I go for a storage area network (SAN)?
> - How should backup be handled?
>
> Answers? Tips? Comments/suggestions? There's a lot of other related
> questions left out that I can't think of at the moment. Any help is
> much appreciated.
Update: I'm probably going to go for a prebuilt server like one of these
V G wrote:
> It is primarily a web and
> file server via HTTP. I'm guessing that at peak, it will need to be
> able to serve 1000 or more people downloading one or more files at a
> time. This number could be wayyyy more or wayyy less, I've never done
> anything with servers before.
>
A good PC at the development time, I think. Later you could analyze
the bottlenecks and choose the appropriate hardware configuration.
> For this purpose (non intensive php pages, posgresql, and intensive
> file downloading server), how would I go about
> building/buying/commissioning such a server?
>
Posgresql should be placed on a separate hardware, I believe
On Thu, Dec 29, 2011 at 12:34 AM, c h <.....cognitive.harmonyKILLspam@spam@gmail.com> wrote:
> A good PC at the development time, I think. Later you could analyze
> the bottlenecks and choose the appropriate hardware configuration.
This is by no means an uncommon application, file and webserver. I
think the bottlenecks are already known to server
administrators/whomever.
> Posgresql should be placed on a separate hardware, I believe.
The load on the database daemon will be relatively light. It is not a
database intensive task - but I'll keep this in mind
There are other things to take into account as well.
The two main things that come to mind are backups and reliability.
A single server unless it's a 2U rackmount (or greater) designed to
take lots of disks or a tower server (also made to hold a lot of
disks) will not be able to hold 8TB and even if they can it will not
leave much room for RAID (guessing you will need that).
Reliability is a large area to cover. It would be useful to know how
much down time is acceptable and what your budget is. What OS are you
planning on running and where will your server be stored (do you have
access to a rack).
So a quick run down of questions you should really know the answer to:
1) How important is the data stored on the server?
2) What is your budget?
3) What are the storage options for the server?
4) How reliable does the service being provided need to be?
I agree with C H though. Either set up a VM of the OS you are planning
on using on your comp or install it on a spare comp and get your
application fully set up on there first.
For 8TB and assuming people would be pissed if you lost data I would
suggest at lease RAID 1 which requires a RAID card (supplied on some
server and not on others) and double the amount of hard drive space
you will be using, so in your case that is 16TB.
Sorry if the above seems a bit unorganised, this is just a brain dump.
On Thu, 2011-12-29 at 14:04 -0500, Joshua Shriver wrote:
> Just buy a barebones system for $299 throw Ubuntu linux on it and
> install the appropriate daemons.
> Make sure it has gigabit ethernet, voila!
Unfortunately been there, done that, have the t-shirt.
While it SOUNDS like a good idea, the issue is stability, and
reliability.
First, jsut the logistics: to support 8TB in RAID, that means probably
16TB of disk space. Depending one what kind of speed is needed, that can
be 6-16 disks (or more). That's alot of SATA ports, meaning an expensive
SATA RAID card (please, don't even consider the "software" RAID options
consumer hardware pushes all the time).
That's just the beginning, getting a case and power supply for this is
non trivial.
After you've put together this beast, keeping it up will be a big job
(budget an hour or 2 a day). It will fail, with so many drives in such a
small space, it will probably fail often.
In the end, unless you've got tons of time to waste and your users don't
mind the down time, using a commercial solution will be FAR better.
There are lots of options out there. Considering the amount of drive
space required, chances are you'll need 2 units, the "server", and a
storage tower.
It would be really useful to the op to have someone local familiar in
this sort of stuff guide them, does the University in question not have
an IT department?
On Thu, Dec 29, 2011 at 2:30 PM, Herbert Graf <hkgrafKILLspamgmail.com> wrote:
>
> Unfortunately been there, done that, have the t-shirt.
>
> While it SOUNDS like a good idea, the issue is stability, and
> reliability.
Really? It didn't even come close to sounding like a good idea.
> First, jsut the logistics: to support 8TB in RAID, that means probably
> 16TB of disk space. Depending one what kind of speed is needed, that can
> be 6-16 disks (or more). That's alot of SATA ports, meaning an expensive
> SATA RAID card (please, don't even consider the "software" RAID options
> consumer hardware pushes all the time).
Why do you say software RAID unsuitable for this?
> That's just the beginning, getting a case and power supply for this is
> non trivial.
Yeah, most likely going to buy a Dell or IBM server. I still need to
know what to look for (like RAID cards, storage expansion options,
etc.).
> After you've put together this beast, keeping it up will be a big job
> (budget an hour or 2 a day). It will fail, with so many drives in such a
> small space, it will probably fail often.
The U of T IT department will handle the maintenance.
> In the end, unless you've got tons of time to waste and your users don't
> mind the down time, using a commercial solution will be FAR better.
> There are lots of options out there. Considering the amount of drive
> space required, chances are you'll need 2 units, the "server", and a
> storage tower.
Ah. That makes sense.
> It would be really useful to the op to have someone local familiar in
> this sort of stuff guide them, does the University in question not have
> an IT department?
They do, but their competence is questionable (from experience).
APPROXIMATELY how much do you think the full server setup would cost?
$100? $1000? $5000? $10000? Just the approximate range
> There are other things to take into account as well.
>
> The two main things that come to mind are backups and reliability.
>
> A single server unless it's a 2U rackmount (or greater) designed to
> take lots of disks or a tower server (also made to hold a lot of
> disks) will not be able to hold 8TB and even if they can it will not
> leave much room for RAID (guessing you will need that).
>
> Reliability is a large area to cover. It would be useful to know how
> much down time is acceptable and what your budget is. What OS are you
> planning on running and where will your server be stored (do you have
> access to a rack).
>
> So a quick run down of questions you should really know the answer to:
>
> 1) How important is the data stored on the server?
> 2) What is your budget?
> 3) What are the storage options for the server?
> 4) How reliable does the service being provided need to be?
1) Not mission critical. Not personal user data. No one will die if
the server goes down for a couple of days.
2) I don't know yet. I'm in the thinking out loud phase.
3) I need around 8TB of storage, with room to expand the storage space.
4) Need to have reasonable uptime, but it can be taken down once a
week (or in the night time) for maintenance if needed.
> I agree with C H though. Either set up a VM of the OS you are planning
> on using on your comp or install it on a spare comp and get your
> application fully set up on there first.
The development will be done in phases - the full application will not
yet be ready.
> For 8TB and assuming people would be pissed if you lost data I would
> suggest at lease RAID 1 which requires a RAID card (supplied on some
> server and not on others) and double the amount of hard drive space
> you will be using, so in your case that is 16TB.
On Thu, 2011-12-29 at 16:05 -0500, V G wrote:
> On Thu, Dec 29, 2011 at 2:30 PM, Herbert Graf <EraseMEhkgrafspam_OUTTakeThisOuTgmail.com> wrote:
> >
> > Unfortunately been there, done that, have the t-shirt.
> >
> > While it SOUNDS like a good idea, the issue is stability, and
> > reliability.
>
> Really? It didn't even come close to sounding like a good idea.
Perhaps to you, but to most getting into this stuff it does sound like a
good idea.
> > First, jsut the logistics: to support 8TB in RAID, that means probably
> > 16TB of disk space. Depending one what kind of speed is needed, that can
> > be 6-16 disks (or more). That's alot of SATA ports, meaning an expensive
> > SATA RAID card (please, don't even consider the "software" RAID options
> > consumer hardware pushes all the time).
>
> Why do you say software RAID unsuitable for this?
Because "software RAID" is always unsuitable. It's horrible for
stability, reliability, and even to initially get working. I've gone
down that road. I've worked through the problems. I've got things
working. I've suffered the issues. I eventually got a hardware RAID card
and haven't looked back.
> > After you've put together this beast, keeping it up will be a big job
> > (budget an hour or 2 a day). It will fail, with so many drives in such a
> > small space, it will probably fail often.
>
> The U of T IT department will handle the maintenance.
Curious, if they are handling the maintenance, why aren't the involved
in the procurement? Are they at least available to give advice?
> > It would be really useful to the op to have someone local familiar in
> > this sort of stuff guide them, does the University in question not have
> > an IT department?
>
> They do, but their competence is questionable (from experience).
Ouch, that doesn't sound promising!? :(
> APPROXIMATELY how much do you think the full server setup would cost?
> $100? $1000? $5000? $10000? Just the approximate range.
I'm not familiar enough with what kind of hardware you need to support
what you're doing (I'm not in IT), but based on similar things I've
seen, you're looking at least $10k. The drives and their housing alone
will probably use a good amount of that budget (please, don't consider
consumer drives, they are not suitable for the usage you have planned).
On 29 December 2011 16:17, Herbert Graf <hkgrafspam_OUTgmail.com> wrote:
> I'm not familiar enough with what kind of hardware you need to support
> what you're doing (I'm not in IT), but based on similar things I've
> seen, you're looking at least $10k. The drives and their housing alone
> will probably use a good amount of that budget (please, don't consider
> consumer drives, they are not suitable for the usage you have planned).
You'd be surprised - I've seen rack fulls of "consumer" drives at a
well known storage company.
If you keep them cool they'll be fine. Your RAID should have hot spares though.
On Thu, 2011-12-29 at 17:18 -0500, M.L. wrote:
> On 29 December 2011 16:17, Herbert Graf <@spam@hkgrafKILLspamgmail.com> wrote:
> > I'm not familiar enough with what kind of hardware you need to support
> > what you're doing (I'm not in IT), but based on similar things I've
> > seen, you're looking at least $10k. The drives and their housing alone
> > will probably use a good amount of that budget (please, don't consider
> > consumer drives, they are not suitable for the usage you have planned).
>
> You'd be surprised - I've seen rack fulls of "consumer" drives at a
> well known storage company.
I'm not saying it isn't done, Google has been well know for using
consumer equipment for server duties.
That said, in my experience, if uptime is at all important to you, don't
use consumer drives for stuff like this. Aside from the much shorter
warranty, they are much more unknown for stability. I've had consumer
drives used in very heavy usage scenarios last for many years. I've had
other consumer drives very lightly used die after a year. The trick is
you never know. Especially when dealing with new models with zero track
record, which is going to be the case when having an array that big.
If you can put up with failures like that, then go for it (be aware,
it's VERY common for drives from the same batch to die very close in
time to each other, so a RAID 5 array might be very vulnerable if you
only have one hot spare. We once had a farm of machines with Fujitsu
drives. We had daily failures. Within a few months almost every single
drive in the farm had died).
Just my 2 cents. TTYL
On 29 December 2011 17:42, Herbert Graf <KILLspamhkgrafKILLspamgmail.com> wrote:
> If you can put up with failures like that, then go for it (be aware,
> it's VERY common for drives from the same batch to die very close in
> time to each other, so a RAID 5 array might be very vulnerable if you
> only have one hot spare. We once had a farm of machines with Fujitsu
> drives. We had daily failures. Within a few months almost every single
> drive in the farm had died).
That's why storage is a big money maker and hot spares are an absolute
necessity.
I've bought my last two servers from http://www.abmx.com/ . One has been
running continuously for more than 3 years. I'm just bringing up another
one to have a spare and to migrate from Fedora 11 (on the current server)
to Fedora 16 (on the new server). Once I have everything up and running on
the new one, I'll use g4l to create a drive image, then copy it to the old
server. It'll be ready as a backup at any time.
Anyway, I like the 1RU servers I get from abmx.
Harold
-- FCC Rules Updated Daily at http://www.hallikainen.com - Advertising
opportunities available!
Not sent from an iPhone
As a follow-up on this... Many years ago I set up a system with hardware
RAID on the motherboard. A later OS upgrade left the hardware RAID
controller unsupported. So... I haven't used RAID since.
I currently just use a single 1TB drive that's about half full. I keep a
backup server ready to go online at any time. A monthly g4l image is done
of the online server drive to an external USB drive (a SATA drive in a USB
dock). A daily rsync of content is also done to an external USB drive so
updated content can be copied over to the backup server if needed.
While RAID deals with hard drive failures, I've always thought there are
lots of other things that can go wrong, so I like to just have a backup
server. Another concept I find interesting is the "redundant array of
inexpensive servers." This allows a system to continue running on the
failure of a server, no matter what the cause was. Another method of
handling large arrays of drives that I've always found interesting, but
never used, is ATAoE. I've seen a lot of ads for Coraid and always found
the idea interesting.
Harold
-- FCC Rules Updated Daily at http://www.hallikainen.com - Advertising
opportunities available!
Not sent from an iPhone
> After you've put together this beast, keeping it up will be a big job
> (budget an hour or 2 a day). It will fail, with so many drives in such
> a small space, it will probably fail often.
This becomes a non-trivial task as some of my friends at work will attest - they are looking after large arrays of disk with so many disks that they reckon on a disk failure every week, almost "without fail" as the saying goes. With the number of disks in these arrays the manufacturers MTBF figure about matches the number of failures they see.
These arrays are petabyte arrays for data streams coming from satellites and tier 1 data storage from the Large Hadron Collider. But the size of your array, even if built with 1TB or larger drives will still have a notable impact on the MTBF rate, so you do need to budget in a couple of spare drives, possibly set up as hot spares.
> > It would be really useful to the op to have someone local familiar in
> > this sort of stuff guide them, does the University in question not
> > have an IT department?
>
> They do, but their competence is questionable (from experience).
But earlier in the mail you said they will be handling the maintenance ???
> While RAID deals with hard drive failures, I've always thought there
> are lots of other things that can go wrong, so I like to just have a
> backup server. Another concept I find interesting is the "redundant
> array of inexpensive servers." This allows a system to continue running
> on the failure of a server, no matter what the cause was. Another
> method of handling large arrays of drives that I've always found
> interesting, but never used, is ATAoE. I've seen a lot of ads for
> Coraid and always found the idea interesting.
One disk array system I worked on used n+1 PC style power supplies that had extra circuitry in them so they could be hot swapped without disrupting the 5V and 12V busses in the system.
>> 1) How important is the data stored on the server?
>> 2) What is your budget?
>> 3) What are the storage options for the server?
>> 4) How reliable does the service being provided need to be?
>
> 1) Not mission critical. Not personal user data. No one will die if
> the server goes down for a couple of days.
> 2) I don't know yet. I'm in the thinking out loud phase.
> 3) I need around 8TB of storage, with room to expand the storage space.
> 4) Need to have reasonable uptime, but it can be taken down once a
> week (or in the night time) for maintenance if needed.
1) Wasn't about server down time but about lost data, as in for good.
Can you afford to even lose a bit of the data permanently? Can the
server completely die and all data be lost and not cause you to lose
any sleep (or whoever the stored data effects).
3) Storage was meant to mean where the server(s) will physically be housed.
4) What about unplanned down time? I have had servers running critical
databases go down for 1-5 days due to a predecessors lack of
appreciation for testing setups and considering backup equipment.
> How about RAID 5 with hot standby?
I have only used RAID 1 before but a quick read up makes me think that
read/write times are going to be very slow, especially with the amount
of disks you need.
Refurbished hardware might save you a couple of £1000
I would normally say stay away from 2TB drives for servers but
considering price is a concern then 2TB would make buying a chassis a
lot cheaper (need less bays).
I know, HP is expensive, but anyway.
I'm working at mobile telecom company, and we are using (in our
department) exclusively HP hardware. Servers are running 24/7 with
allowed service downtime 10 minutes a year. Hardware failures never
had stopped our services, those always were software problems.
We had a EVA array with 52 or so hard drives, and had replaced
probably 4 disks in 5 years in that array, and 2 or 3 fans.
That platform (6 racks) was replaced with newer one (2 racks). It
already had one motherboard replaced, but still no disk failures in
two years now, if I remember right.
Systems are running in properly maintained server rooms, about 18
degrees celsius.
So all I can say - HP is probably worth the money.
>> After you've put together this beast, keeping it up will be a big job
>> (budget an hour or 2 a day). It will fail, with so many drives in such
>> a small space, it will probably fail often.
>
> This becomes a non-trivial task as some of my friends at work will attest - they are looking after large arrays of disk with so many disks that they reckon on a disk failure every week, almost "without fail" as the saying goes. With the number of disks in these arrays the manufacturers MTBF figure about matches the number of failures they see.
>
> These arrays are petabyte arrays for data streams coming from satellites and tier 1 data storage from the Large Hadron Collider. But the size of your array, even if built with 1TB or larger drives will still have a notable impact on the MTBF rate, so you do need to budget in a couple of spare drives, possibly set up as hot spares.
>
>
> --
> Scanned by iCritical.
>
On Thu, Dec 29, 2011 at 6:45 PM, <spamBeGonealan.b.pearcespamBeGonestfc.ac.uk> wrote:
>> They do, but their competence is questionable (from experience).
> But earlier in the mail you said they will be handling the maintenance ???
Yes, they will handle simple things. I will handle what they can not
On Thu, Dec 29, 2011 at 7:02 PM, Alex Wood <TakeThisOuTkingofthemodsEraseMEspam_OUTgmail.com> wrote:
> 1) Wasn't about server down time but about lost data, as in for good.
> Can you afford to even lose a bit of the data permanently? Can the
> server completely die and all data be lost and not cause you to lose
> any sleep (or whoever the stored data effects).
Well, of course, I would rather it be very reliable, but losing the
data will not have any super bad negative effects. I want them backed
up anyhow.
> 3) Storage was meant to mean where the server(s) will physically be housed.
In the university server room most likely. Need to discus this with
the dean, etc.
> 4) What about unplanned down time? I have had servers running critical
> databases go down for 1-5 days due to a predecessors lack of
> appreciation for testing setups and considering backup equipment.
That's fine. I'm not too worried about downtime. Only my service will
rely on this server (no other university services will rely on it), so
a few hours a day of downtime even isn't going to kill anyone. But I'm
sure that amount of downtime is not going to happen.
>> How about RAID 5 with hot standby?
>
> I have only used RAID 1 before but a quick read up makes me think that
> read/write times are going to be very slow, especially with the amount
> of disks you need.
Really? RAID 5/6 is slow? Isn't it faster than a large single disks?
I'd imagine it actually is pretty fast
>> I have only used RAID 1 before but a quick read up makes me think that
>> read/write times are going to be very slow, especially with the amount
>> of disks you need.
>
> Really? RAID 5/6 is slow? Isn't it faster than a large single disks?
> I'd imagine it actually is pretty fast.
Just a guess. Like I said I have never used it before. I would suggest
you research it and find out what its actual pitfalls are. But
accessing sveral disks to retive data seems like it would take longer
then a single disk (for small files)
>>> I have only used RAID 1 before but a quick read up makes me think that
>>> read/write times are going to be very slow, especially with the amount
>>> of disks you need.
>>
>> Really? RAID 5/6 is slow? Isn't it faster than a large single disks?
>> I'd imagine it actually is pretty fast.
>
> Just a guess. Like I said I have never used it before. I would suggest
> you research it and find out what its actual pitfalls are. But
> accessing sveral disks to retive data seems like it would take longer
> then a single disk (for small files).
The general rule is that RAID will speed up reads but slow down writes.
FreeBSD, Solaris, OpenSolaris, and Linux via beta 3rd-party driver, support this advanced filesystem. I would trust Solaris or FreeBSD the most..
Every ZFS metadata block contains a checksum of the blocks it references. Every time ZFS reads a block, it verifies the checksum. If it's not right, it reads the copy from the {mirror/raid} and rewrites the original block (thus self-repairing). This is far more advanced than simply checking if the IO channel returned a read error, and then trying it on the other IO channel (RAID-1). Even if the IO channel returns no errors, the data could still be corrupted. In fact, a standard hardware RAID will be happy to copy corruption from one disk to another in the name of "recovery", as long as the disks don't report read errors. ZFS can identify incorrect data coming off the disk, something a hardware RAID controller or a software emulation of a RAID controller cannot.
ZFS supports volumes of many mirrored sets (my preference), or groups of RAID with single, double, or triple parity. (Meaning one, two, or three drives can fail).
ZFS also supports real scrubbing (reading every allocated block and comparing it to its checksum)... Which done regularly, will identify and self-repair disk errors in a very reliable way. It also supports in-place growth, zero-cost snapshots, replication, etc, etc....
On Solaris it supports transparent kernel-level Windows file service
via CIFS/SMB, with ACLs et al.
> On Thu, Dec 29, 2011 at 2:30 PM, Herbert Graf<RemoveMEhkgrafEraseMEEraseMEgmail.com> wrote:
>> Unfortunately been there, done that, have the t-shirt.
>>
>> While it SOUNDS like a good idea, the issue is stability, and
>> reliability.
> Really? It didn't even come close to sounding like a good idea.
>
>> First, jsut the logistics: to support 8TB in RAID, that means probably
>> 16TB of disk space. Depending one what kind of speed is needed, that can
>> be 6-16 disks (or more). That's alot of SATA ports, meaning an expensive
>> SATA RAID card (please, don't even consider the "software" RAID options
>> consumer hardware pushes all the time).
> Why do you say software RAID unsuitable for this?
>
>> That's just the beginning, getting a case and power supply for this is
>> non trivial.
> Yeah, most likely going to buy a Dell or IBM server. I still need to
> know what to look for (like RAID cards, storage expansion options,
> etc.).
>
>> After you've put together this beast, keeping it up will be a big job
>> (budget an hour or 2 a day). It will fail, with so many drives in such a
>> small space, it will probably fail often.
> The U of T IT department will handle the maintenance.
>
>> In the end, unless you've got tons of time to waste and your users don't
>> mind the down time, using a commercial solution will be FAR better.
>> There are lots of options out there. Considering the amount of drive
>> space required, chances are you'll need 2 units, the "server", and a
>> storage tower.
> Ah. That makes sense.
>
>> It would be really useful to the op to have someone local familiar in
>> this sort of stuff guide them, does the University in question not have
>> an IT department?
> They do, but their competence is questionable (from experience).
>
> APPROXIMATELY how much do you think the full server setup would cost?
> $100? $1000? $5000? $10000? Just the approximate range.
> May I suggest a ZFS based filesystem?
>
> FreeBSD, Solaris, OpenSolaris, and Linux via beta 3rd-party driver,
> support this advanced filesystem. I would trust Solaris or FreeBSD the most.
>
> Every ZFS metadata block contains a checksum of the blocks it
> references. Every time ZFS reads a block, it verifies the checksum. If
> it's not right, it reads the copy from the {mirror/raid} and rewrites
> the original block (thus self-repairing). This is far more advanced than
> simply checking if the IO channel returned a read error, and then trying
> it on the other IO channel (RAID-1). Even if the IO channel returns no
> errors, the data could still be corrupted. In fact, a standard hardware
> RAID will be happy to copy corruption from one disk to another in the
> name of "recovery", as long as the disks don't report read errors. ZFS
> can identify incorrect data coming off the disk, something a hardware
> RAID controller or a software emulation of a RAID controller cannot.
>
> ZFS supports volumes of many mirrored sets (my preference), or groups of
> RAID with single, double, or triple parity. (Meaning one, two, or three
> drives can fail).
>
> ZFS also supports real scrubbing (reading every allocated block and
> comparing it to its checksum)... Which done regularly, will identify and
> self-repair disk errors in a very reliable way. It also supports
> in-place growth, zero-cost snapshots, replication, etc, etc....
>
> On Solaris it supports transparent kernel-level Windows file service
> via CIFS/SMB, with ACLs et al.
>
> Joe Koberg, AE5NE
> RemoveMEjoeTakeThisOuTspamosoft.us
>
That sounds good. So you're saying go with a ZFS system on software
RAID rather than hardware RAID?
On Fri, Dec 30, 2011 at 9:47 AM, Bob Ammerman <EraseMEpicramspamspamBeGoneroadrunner.com> wrote:
> The general rule is that RAID will speed up reads but slow down writes.
> > The general rule is that RAID will speed up reads but slow down
> writes.
>
> On what RAID levels? RAID 0 speeds up both.
RAID 0 speeds everything up because there is no protection to compute.
Remember all RAID is software - but the 'normal' meaning of software RAID is that it is the operating system that is doing the RAID work. Hardware RAID means there is a dedicated processor on the drive controller that is doing all the RAID work, making the disk array look like a single drive to the host operating system.
Then again you can have mixed systems where you have two RAID controllers, each running their own set of striped disks in RAID 5, and then the two controllers are run in RAID 1 in the host operating system to give mirrored RAID sets. This can have an advantage for disk read access speed under some circumstances.
Another trick with hardware RAID controllers is for them to have heaps of RAM on them as disk cache. This can make huge differences if the OS is running database applications where there is a lot of R/W access in a limited area of the disk. Having multiple mirrored disks can also make a big difference with this database software, one mainframe system I was involved with would have great arrays of drives mirrored together, often 6 to 8 drives in a single mirror of the key structure of the database if it was getting a hammering.
If the IT department cannot help configure a system, surely there is a computing faculty with some profs who could give guidance, or recommend some bright young things among their students who might give actual help in configuring (and you could probably pick up useful tips from them as well) ...
> Update: I'm probably going to go for a prebuilt server
That seems like an uncommonly sensible idea. With a maintenance contract, perhaps.
Is there a nearby department who already has a similar system? What would "central facilities" want you to have, if they had to take over maintaining it?
As far as I've been able to tell, a "server" is usually a pretty standard PC with better disks and lesser graphics card, compared to a desktop.
You didn't mention the budget, which is usually a significant factor. Homebuilt budget systems are for departments with no cash and a steady throughput of computer-savy students or longtime employees to maintain things. Giving up "system management" to a separate team is usually a significant and traumatic event in the life of a small company.
Or the type of files being served. 8TB (even losing 30% to raid) is an awful lot of storage unless you're serving up video.
RAID is reported to frequently be more trouble than it's worth. For a read-mostly server, a second full backup system synced across the net may be more useful.
Document everything. (or not! do you want to work there the rest of your life? University environments are temptingly pleasant places to work... :-)
On Fri, Dec 30, 2011 at 7:37 PM, William "Chops" Westfield
<RemoveMEwestfwKILLspammac.com> wrote:
>
> On Dec 28, 2011, at 9:30 PM, V G wrote:
>
>> Update: I'm probably going to go for a prebuilt server
>
> That seems like an uncommonly sensible idea. With a maintenance contract, perhaps.
> Is there a nearby department who already has a similar system? What would "central facilities" want you to have, if they had to take over maintaining it?
It's uncommon? I thought buying prebuilt servers IS common - like the
Dell/IBM/HP rackmount servers.
> On Fri, Dec 30, 2011 at 9:47 AM, Bob Ammerman <KILLspampicramspamBeGoneroadrunner.com>
> wrote:
>> The general rule is that RAID will speed up reads but slow down writes.
>
> On what RAID levels? RAID 0 speeds up both.
As I understand it, RAID 0 = STRIPING isn't really RAID at all because there is no redundancy. I guess you could call it AID instead.
> > That seems like an uncommonly sensible idea. With a maintenance
> contract, perhaps.
>
> It's uncommon? I thought buying prebuilt servers IS common - like the
> Dell/IBM/HP rackmount servers.
It's the sensible part that is uncommon - think in terms of dry English humour ...
-- Scanned by iCritical.
V G wrote:
> c h wrote:
>> A good PC at the development time, I think. Later you could analyze
>> the bottlenecks and choose the appropriate hardware configuration.
>
> This is by no means an uncommon application, file and webserver. I
> think the bottlenecks are already known to server
> administrators/whomever.
>
>> Posgresql should be placed on a separate hardware, I believe.
>
> The load on the database daemon will be relatively light. It is not a
> database intensive task - but I'll keep this in mind.
> --
- Start with single PC,
- then separate out the login/register database PC;
- then start adding as many PCs to do web access / file storage job as needed;
- the web access / file storage PC would either serve the request if
the file is available locally,
- or, if the file is not available locally, the PC would ask local
database about which servers keep the file, and then redirect to one
of them.
- the synchronization of the local databases shoud be performed over
resonable time interval;
- you don't need RAID in my opinion, just maintain multiple copies of
files and place PCs at different locations; HDDs are cheap now, 2TB
per server seems to be a reasonable option. With reasonable
motherboard, RAM, and case, the server would be cheap
On Wed, Jan 4, 2012 at 6:11 PM, c h <EraseMEcognitive.harmonyEraseMEgmail.com> wrote:
> - Start with single PC,
> - then separate out the login/register database PC;
> - then start adding as many PCs to do web access / file storage job as needed;
> - the web access / file storage PC would either serve the request if
> the file is available locally,
> - or, if the file is not available locally, the PC would ask local
> database about which servers keep the file, and then redirect to one
> of them.
> - the synchronization of the local databases shoud be performed over
> resonable time interval;
> - you don't need RAID in my opinion, just maintain multiple copies of
> files and place PCs at different locations; HDDs are cheap now, 2TB
> per server seems to be a reasonable option. With reasonable
> motherboard, RAM, and case, the server would be cheap.
That sounds like a good task flow. I am still leaning towards ZFS
RAIDZ2 or something like that.
Next question is: if there are multiple servers set up to serve files,
how will load sharing occur? How do I set up the load sharing?
V G wrote:
> That sounds like a good task flow. I am still leaning towards ZFS
> RAIDZ2 or something like that.
>
> Next question is: if there are multiple servers set up to serve files,
> how will load sharing occur? How do I set up the load sharing?
>
I was actually looking at that board just before you mentioned it! If
the Dell/HP servers prove to be unjustifiably much more expensive than
a build that I do myself, I'll go with this.
How many (gigabit?/10Gb?/100Gb?) Ethernet ports are typically are
on/used per computer in the server world
gmail.com> wrote:
