Drivers 3ware Escalade 6000 Storage Switch

by



Switch

The world of ATA RAID is a small one. When I first became interested there were two players: Promise and Highpoint. Slowly I began to hear of 3ware, a company that produced great products, but at a high price. At that time RAID 0 and 1 were about it, and then Promise released the SuperTrak100 and Adaptec released the AAA-UDMA, both touting RAID 5. At that time 3ware released a BIOS and driver update for their Escalade 6000 series that added RAID 5. Of course there was no onboard RAM cache, so performance was lackluster. Then with the second wave of ATA RAID 5 3ware released the Escalade 7000 series and things looked up. We’re looking at the Escalade 7500-8, an ATA RAID controller capable of handling 8 drives in RAID 0,1,5,10 and JBOD ( just a bunch of disks ) in a Windows 2000 environment.

Escalade® 7500 Series

  • Supports up to 12 drives with a single PCI card enabling up to 3terabytes of storage
  • Ultra ATA/100 interface
  • StorSwitch point-to-point non-blocking architecture
  • PCI 2.2 compliant 64-bit/33MHz bus master1
  • RAID 0, 1, 10, 5 and JBOD support2
  • Highest performance ATA RAID Controller
  • On-board processor to provide true hardware-based RAID
  • Bootable array support for greater fault tolerance
  • BIOS set up utility and 3ware Disk Manager (3DM®) web-based management software
  • Hot-swap and hot-spare capability
  • Windows® and Linux® operating systems support

3Ware 7000-2 - Escalade RAID Controller Manuals & User Guides. User Manuals, Guides and Specifications for your 3Ware 7000-2 - Escalade RAID Controller Computer Hardware, Controller. Database contains 2 3Ware 7000-2 - Escalade RAID Controller Manuals (available for free online viewing or downloading in PDF): Operation & user’s manual, Features.

Install LSI 3ware Escalade 6000 Storage Switch driver for Windows XP x86, or download DriverPack Solution software for automatic driver installation and update. L03B Drivers Installer. 3ware Escalade 6000 Storage Switch Porta giochi per Thrustmaster ACM (prima porta) FOMA N01B OBEX Port (COM4) Contact.

Opening the box revealed: the card, eight 18” ATA 80 wire 40 pin cables, a manual, a driver revision printout, a software envelope containing driver disks and a CD, and four molex “Y” power splitters. I read through the manual to get a feel for the product: it’s a thin photocopy that covers the basics. The manual covers the Escalade 6000 and 7000 series, so the photos in the manual are of 6000 series cards. What I found odd was that the 7.4 driver revision notes were thicker than the manual. Even so, the manual was straight forward and easy going for anyone with RAID experience.

Looking at the actual Escalade 7500-8 adapter, it’s layout is simple and effective. A central 3ware 200-0017-00 chip for RAID calculations and their own StorSwitch technology, two 3ware 200-0033-00 chips to each handle four ATA133 ports. Unfortunately there’s no information regarding individual ASICs on the 3ware website. Two 150 MHz 0.9 MB IDT memory chips for a total of 1.8 MB onboard cache, although I’ve read it online as 2 MB cache available. A few other chips on the board I was unable to identify, plus a four pin LED connector. The board is compact, reasonably short and well laid out.

Installation was straight forward: connected the drives, inserted the card and booted. the Escalade series are 64 bit 33 MHz PCI adapters, allowing for a maximum of 266 MB/sec transfers. Installed the drivers, rebooted and installed 3DM, 3ware Disk Management Utility for Windows. Creating, deleting and repairing RAID arrays is done via the card’s BIOS at boot up, and employs a simple and easy to understand interface. Creating RAID 0, 1 and 10 arrays was immediate: creating RAID 5 arrays took some time as X’s are written to test. This is good feature but takes some time. While it was inconvenient for testing and benchmarking, users should see this as a nice safety feature. As with all RAID products, the Escalade 7500-8 showed as a SCSI controller in Device Manager, and arrays show under disk drives.

Monitoring and reporting is handled via 3DM. This runs as a web server, so access is through your browser and accessed locally or over your network. Again, easy to use and straight forward. Linux users have the CLI, or 3ware Command Line Interface.

In the box I received driver version 7.4, which I installed and began to benchmark with. After a few weeks the board stopped working, so I took advantage of 3ware’s technical support. The system hung at the Escalade 7500-8’s BIOS and wouldn’t go any further. First you register your product online and create a profile, then submit a problem report. Within a day I received an RMA for the board, and since I selected cross shipment by providing my credit card, I received a replacement board within a week. Very nice and smooth technical support.

This time the box contained the 7.53 drivers. I began benchmarking with these, the latest drivers. Performance was ho hum and nothing special. Following Storage Review’s forums on 3ware products revealed users resorting to registry hacks and changing disks from basic to dynamic to get better performance. Then a posting revealed a new driver revision, 7.6, that gave a real performance boost without the “voodoo” tweaking. Redid all benchmarks with the 7.6 driver, firmware and 3DM releases. Doing this revealed the product forces you to update all three items, another nice feature to keep things as 3ware wants it.

Benchmarks don’t lie: the 3ware Escalade 7500-8 with the 7.6 drivers dominated all tests save WinBench99 RAID 5. Sometimes it’s a close call with the FastTrak SX4000, other times a huge margin.

Recommendations are easy: 3ware products give the best ATA RAID performance. If you need to use 8 or 12 drives, they’re the only player in town. The Escalade 7500-4LP is a four drive low profile adapter and fits just about anywhere, another great plus. The only concern is cost: expect to pay 50-100% more for 3ware cards compared to Promise’s FastTrak SX4000. In this case you get what you pay for: the best performance for the highest price.

Editors note: this review was migrated from the old eBabble.net site and the photos updated. Originally published May 30th 2003.

Dave Schuller with three of the four Big Red RAID servers. The fourth hasalready been assigned for duty on the MacCHESS Linux Beowulf clusterSirius .

MacCHESS staff recently (February 2002) completed assembly of 4 two terabyteLinux RAID disk servers in the Big Red RAID project.

6000Drivers 3ware escalade 6000 storage switch -

© Last updated March 2002
Webmeister: David J. Schuller

Design Goals
The point was to build a lot of RAID capacity for not a lot of money. Performance was a secondary consideration. Therefore we went with ATA disksinstead of SCSI, and did not invest in hot swap drives or power supplies.

The Name
The enclosures are beige. Big Red is the unofficial nickname ofCornell University athletics.

Hardware & Construction
Parts, per server:

  • 1x Amtrade 1200RC cube series enclosurewith 26 5.25' half-height open disk bays + room fora motherboard and several power supplies.includes 4x fans, 25 pair mounting rails,mounting hardware, casters with brakes
  • 1x Tyan S2462 ThunderK7 mother board
  • 2x AMD Athlon 1.2 GHz MP CPUs
  • 2x Monarch Computer #130955-2CPU cooling fans
  • 2x 256 MB ECC registered DDR PC2100 RAM Micron part# CT3272Y265
  • 1x IBM DeskStar60GXP 20 GB hard disk (for root disk)
  • 1x CDROM drive, 56X EIDE for software installation & updates
  • 1x NMB SD025A460WSW power supply for Thunder K7
  • 1x 300W ATX power supply
  • 18x Jameco #117217 3.5/5.25' harddrive bracket
  • 7x custom 3.5/5.25' perforated aluminum disk sledsconstructed by Bill Miller.
  • 3x 3WareEscalade 7810 storage switch, 8 ATA100 ports,64 bit 33 MHz PCI, includes 4x power splitter cables and 8x 18' ATA100 cables
  • 24x Maxtor 536DX ATA1005400 RPM 100 GB disks
  • 1x IntelPro/1000T gigabit copper ethernet NIC
  • 12x 3Ware single drive ATA100 cable, 36'40 pin/80 conductor, pack of 2, 3W-CBL-36
  • 2x Jameco #140724 insulationdisplacement wire splice, 18-14 ga
  • 1x Jameco #34323 plastic drivemounting rails, pair (for root disk)
  • 1x Jameco #147379 ATX power connector
  • 2x AdaptecACK-68I-68E-LVD SCSI internal to external cable

Construction
Some of the interesting points:

  • Two of the eight Athlon CPUs were DOA. We burned a third one in testing.
  • The extended ATX motherboards are big. They hung over into the disk bays.Two of the Amtrade enclosures were an inch shorter than the others, whichexacerbated the problem in those units. We were able to use all the baysanyway after Bill Miller constructed some customdisk sleds.
  • The motherboard trays came pre-installed in the left side of the enclosures(as seen from the front). We switched them to the right side to keep all thoseATA cables from blocking airflow to the motherboard.
  • The Tyan ThunderK7 motherboard requires a special 460 W powersupply, the connectors differ from the ATX standard. This supply alone wasunable to handle the startup surge from a completed server, so we added a 300 WATX power supply to handle half of the RAID disks. We spliced into the greenwire of the NMB power supply (thus the wire splices, ATX power connector, andsome unspecified wire) so that both supplies start up at the same time throughthe logic control power switch. I miss the days when computers had real powerswitches. Its not a good feeling when you're into a chassis up to your elbows,and then notice that the fan is still running.
Cabinet views
Front view, doors removed
Drivers 3ware escalade 6000 storage switch replacementThe 7 lowest drives on the right hand side have the customsleds. The root disk is located behind the power switch panel at the topleft.
Front view, with doors

Drivers 3ware Escalade 6000 Storage Switch Plate


Inside view
Look at that mess of cables! We moved the motherboard to the right side of thechassis to make more room for the cables coming off the PCI cards. The special460 W NMB power supply is just above the motherboard, it runs the motherboard,fans, root disk, CD-ROM and right-side RAID disks. The additional 300 W ATXpower supply (bottom foreground) runs the left-side RAID disks.

Software & Configuration

  • For an OS, we used Red Hat Linux 7.1.Aside from an undeniable Open Source bias on our part, it has good SMP support,good reliability, excellent software RAID support. It comes with all thedrivers we needed. It also supports NFS, which is how we'll be sharing spaceto our Unix systems. Oh yeah, and it's free. The LinuxSoftware-RAID HOWTO document was very helpful for RAID configuration.
  • We used a separate root disk and saved ourselves the trouble of trying toboot from a RAID volume.
  • We considered a number of different RAID configurations. Factors to considerincluded the number or disks/controller, the number of controllers, capacity,reliability and speed. The 3Ware cards can be configured to run RAID levels0(stripe), 1(mirror), 10(striped mirror?) or 5. Linux software is similarlyflexible, and can also support multiple tiers of RAID. We tried setting up the3Ware cards with RAID5 and striping across the 3 controllers in software, andstriping across various numbers of disks on the cards with RAID5 in software. Wesettled on the latter after seeing the Benchmark resultsbelow.
  • The 3Ware cards need to be configured during the boot sequence, so wehooked up a keyboard and monitor during configuration. The kernel driver forthe 3Ware controller cards comes standard with Red Hat 7.1. Load the 3Waremodule at boot time by adding this line to file /etc/rc.d/rc.sysinit in lineafter 'swapon'You could alternatively tailor a kernel to include the module. If the 3Waresupport is running properly, the devices you configured on the controllersshould show up as SCSI devices listed in /proc/scsi/scsi. #Ware distributes adaemon, 3dmd, tocheck up on the controllers and report any problems, but the version availabledid not run with recent kernels.
  • construct a /etc/raidtab file for your Linux softwareRAID configuration. Implement the configuration like so:
    Check results by looking in the /proc/mdstat file.
  • Build the filesystem. We wanted a journaling filesystem, because waitingfor a 2 TB volume to fsck is no fun at all. We usedReiserfs because it's already integratedinto the Linux kernel and the tools come with Red Hat Linux.
  • put an entry in /etc/fstab file:(the noauto flag was just for testing)
  • Construct the mount point and mount your new RAID volume:
  • Export the space via NFS, if that's your pleasure. Make sure NFS isinstalled and configured, and put an entry in /etc/exports.

Drivers 3ware Escalade 6000 Storage Switch Replacement

Capacity
Each server contains 24x 100 GB disks for a total raw RAID space of 2.4 TB.After RAID configuration, formatting and assignment of spares, the usable spaceis 1.95 TB. Current Linux kernels have a hard limit of 2 TB per filesystem, sothere was no point in trying to squeeze any extra space out of it.

Benchmarks
I found a Linux RPMversion of Bonnie, hereare the numbers with some other systems as well for comparison:

  • alpha1: 72 GB, 10K RPM SCSI disk on Compaq XP1000 Tru64 system
  • alpha2: 1+ TB turnkey Ultra160 SCSI RAID5 (8x 7200 RPM disks) system onCompaq XP1000 Tru64 system
  • bigrd1: Big Red server with single 10 GB ATA disk
  • bigrd2: Big Red server with RAID5 on each 3Ware card (7 RAID disks, 1 spare) with software RAID0 over the 3 controllers.
  • bigrd3: Big Red server; each 3Ware controller running RAID0 striping over 4 pairs of disks and software RAID5 over the resulting 12 devices (11 RAID5 devices, 1 spare)
  • chunk size optimised to 64 KB, both on the 3Ware card and in software.
  • **** final configuration
  • the Alpha/Tru64 machine had >1 GB of RAM, while the Linux box had only 0.5GB; that's why the memory size for Bonnie testing varies. 2047 is also themaximum size with the Linux version of Bonnie used.

Drivers 3ware Escalade 6000 Storage Switch Wiring

Commentary
Why we did what we did, and what we would do differently:

  • Motherboard: We chose the Tyan ThunderK7 because we like Athlons. At thetime this was the only dual Athlon board available. We wanted dual CPUs forhandling all the I/O requests and doing the RAID5 parity calculations. Thefive 64 bit/33 MHz PCI slots are great for getting the most out the disk andethernet controllers. The on-board video saved us an extra card duringconfiguration. On the down side, the size of the motherboard caused us somegrief by extending into the disk bays. We hadconsidered the possibility of serving the RAID space over one of the on-boardSCSI ports, but have been unable to find out anything about daemons toaccomplish this. Therefore the dual on-board SCSI is going to waste, unless wetack on a tape drive at some point. The on-board dual 10/100 ethernet is alsoa bit of a waste considering we added a gigabit ethernet card anyway.
  • The 3Ware 7810 storage switches have the most ATA disks/PCI slot we haveseen. The design would have had to change considerably without these. The 64bit/33 MHz PCI interface and ATA100 support go nicely with our other parts. The3Ware 'we're discontinuing these, no we're not' act was a bit disconcerting. There was also a recall on the 7810s; we never did hear from our distributorabout that and we hope it doesn't affect our configuration. The RAID5 writeperformance of the 3Ware cards is also disappointing, but we're doing just finewith running our RAID5 in Linux. Drivers for the 3Ware controllers comestandard with Red Hat Linux, but the 3dmd daemon doesn't seem to work with ourkernel version.
  • disks: We bought the highest capacity ATA disks available at the time.Obviously the technology keeps improving; today we could get either a highercapacity or higher spindle speed. The 536DX drives are quiet and cool, whichis good. During configuration of the four servers (96 disks) we encounteredonly one bad disk. No complaints.
  • Enclosure: lots of disks per box. We could have fit the disks in a smallerbox with 3in2 or 5in3 adaptors, but then heat might be an issue. The systemsseem plenty cool with the standard 4 fans plus a fan in each power supply.Also, the thought of fitting all those ATA cables in a smaller space gives methe willies. We moved the motherboard from the left side of the cabinet tothe right in order to make more room for all those ATA cables. Say, whydoesn't anyone sell chassis with lots of 3.5' bays instead of 5.25' bays?
  • Cables: the 3Ware cards come with a full complement of standard 18' ATA100single drive cables, but in a box this size about half the disks need somethinglonger, which is why we ordered the 3Ware 36' single drive cables. All partswere purchased before any assembly took place, so we went conservative andordered too many. Make us an offer! The cabling is a nightmare, can't waitfor serial ATA.
  • Performance: The Bonnie test results above lead to a number of conclusions:
    • The 24 disks in parallel make for some excellent streaming performance.
    • Random seek performance does not match the SCSI RAID competition. Usingdisks faster then 5400 RPM might give some improvement here, but speed was notour primary design goal anyway.
    • Running RAID5 on the 3Ware cards slowed down sequential block write andrewrite performance, but did not greatly affect input performance. RAID5 insoftware was faster in every category and about 4x faster in sequential blockwrite and rewrite.
  • Reliability: Hopefully the RAID5 configuration will work for us. We couldhave gone with a two-tier RAID5 configuration for extra reliability, or evenRAID10 bullet-proof status. Another possibility would be to break the spaceinto several smaller volumes (probably a more experienced and sane person wouldhave done this).

Personnel
Planning and construction were carried out by Dave Schuller and Bill Miller ofMacCHESS.

  • Back to Top
    Back to Schuller staffpage
    MacCHESS home page