.

Hewlett Packard

 


Features - March 1999 - Testing Windows 2000
This month we bring you a world first, a Windows 2000 system performance test based on Windows 2000 Professional Beta 3 (the product formally known as Window NT 5.0 Workstation).
..

It is now clear to most people that the Active Directory component of Windows 2000 (Win2K) Server is a major deal, and not an upgrade to be taken lightly, it will take planning, testing and time. However there is more to Win2K than the directory service alone, Win2K Professional is the replacement for Windows NT 4.0 Workstation and it is this version I will be testing. Win2K Pro should finally bring us to the point where we can remove Windows 9x from the company workstations and the laptops. So if you are thinking along these lines it is important that you have a good understanding of where to spend the money when you are buying machines that could / will one day run Win2K Pro. These tests are designed to highlight the requirements of Win2K and specifically which of two major components, the system CPU and memory, it is most hungry for. If you take this data and apply it to your current, and future, desktop and laptop systems it will give you an idea of what you should be spending your budget on.

"What is the minimum specification we should plan for our Win2K Pro roll outs?"


This is just the type of question I have been asked on many occasions over the past year or so (although we called it NT 5 then)! Up until now I have had to fall back on the white papers covering the PC98 hardware specifications and experience. While I am pretty sure this will suffice I am never quite happy until I can substantiate my recommendations so that is what I have set out to do. The next problem was a big one…how? Well, firstly Windows 2000 is in Beta, which means doing any of the industry standard benchmarks would be all but pointless. This is because the optimisation phase of the code doesn’t usually occur until the final Release Candidate (RC) not only that but the non-disclosure agreement that everyone on the MS Beta program has signed specifically forbids publishing any benchmark findings based on Beta code.

The solution took some time to find but was in fact simple: if you run the same tests on different systems through different memory and CPU configurations you can generate enough test data to create a results matrix that could show the true performance of these systems relative to each other. This is what I have done, and because the tests are based on my own scripted tests emulating "Real World" application usage the results could be published. Several phone calls later I had secured the required systems with the four memory configurations I wanted 32, 64, 128, 256Mb and the required permissions from Microsoft to work with the very latest interim build of Windows 2000 Professional Beta 3.

Let’s start with the hardware that it to be tested.


I have a range of three systems that should provide a good cross section of possible systems. The first system is a Compaq Armada 7800 Notebook this has a Pentium II 266MHz processor and an S3 ViRGE M5 video display adapter. It is my prediction is that one of the main areas we will see Windows 2000 taking off is in the mobile market place as people realise the benefits of features like the Encrypted File System (EFS), it is for this reason I have included a notebook into the tests.

Note
: Due to limitations of the Armada 7800 machine the maximum amount of RAM the system can be configured for is 224MB. So this was the amount used in the tests, however for clarity I have left the graphs referring to 256MB.

The next system is based on a Socket 7 Diamond Micronics C400 motherboard using the AMD K6 400 MHz processor and a Diamond 3D-2000 Pro graphics card. I wanted to see how well the AMD chipset performed with Windows 2000 so this was chosen to represent a corporate mid-range system. At the high end I tested a Dual Pentium II – 400Mhz system based around an Intel motherboard which also provided the embedded Cirrus Logic 5480 graphics card.

Motherboard Processor Video Hard Disk Controller
Skt. 7 Diamond Micronics C400 AMD K6-400 Diamond 3D-2000 Pro IBM-DDRS-39130 SCSI Diamond FirePort40 PCI Ultra SCSI
Compaq 7800 Intel Pentium II- 266MHz S3 ViRGE M5 IBM-DADA-25400 EIDEc Compaq PCI EIDE
Dual Intel Slot 1 2 x Intel Pentium II- 400Mhz Cirrus Logic 5480 Seagate ST31055W SCSI Symbios Logic 2280X PCI Ultra SCSI

The Software

The operating system on test is obviously Windows 2000 Professional but to be a little more precise it is Build 1946 which is the first internal release candidate (RC) for Beta 3. This was installed onto a NTFS 5 partition and where possible using only the device drivers included with the system. A network card was installed but no Active Directory components were used. A NUL Local Printer port was created and the HP LaserJet 4 Print Driver installed. The application package used was Office 97 Professional with Service Release (SR) 1 installed and the Office Assistants were disabled!

The Tests


Next I set about building some tests, for this task I used the excellent AutoMate 4 from Unisyn, (www.unisyn.com) to script the tests. These tests were designed to replicate a set of standard operations on the three major components of Office 97, Word, Excel and PowerPoint. These tasks included:

Application Start-up Shutdown
File Open File Close
File Print Find and Replace
Data Sorts Cut and Paste
Data Merges Autoformat
.
Test 1 - Word and PowerPoint

As the results from this first test show a quite amazing result in the 32MB bracket with the Dual Pentium system slower than the single Pentium notebook. This is entirely due to the extra memory requirement of the systems extra processor which results in a system bottleneck to end all! It is very obvious that the so-called "minimum" memory requirement of 32MB for Win2K Pro is just that, the minimum for the operating system itself. If you are planning on running Office as well you will need to start with 64MB as your new minimum requirement.


The jump in performance between 64MB and 128 MB is also substantial, some 17 seconds on the Dual Pentium system and a massive 35.3 seconds on the Armada. The additional of another 128MB to take us up to 256MB shaved another 5.4 seconds of the dual system results and 4.7 of the Armada.
.
Test 2 – Spreadsheet Autoformat
Test 2 is very much a test of the CPU as the Autoformat process is very much dependant on the power of the processor very little improvement in performance for the larger memory configurations was expected. See figure 2.


What was a surprise through was just how much the systems were affected by the low 32MB. Again this was particularly the case for the Dual Pentium system as the memory overhead of the extra processor showed itself with a test result of 13.3, slower than the 12.73 of the K6 with the same memory. In fact the K6 has proved itself a worthy contender in all areas easily out pacing the PII – 266Mhz. I have yet to test out it’s 3D capabilities but after its performance in these tests I feel it warrant’s further examination. My only reservations with it is the fact it is based on the Socket 7 motherboard which I feel still has question hanging over it’s future. However, by the time you read this AMD should have released their new Slot 1 based processor so watch this space!
 

Test 3 - File System

The results of this test were always going to make the Compaq Armada look weak, as it was the only system based on a EIDE hard drive. The Diamond SCSI card and IBM drive peaked at around 114 seconds from 64Mb upwards but at 32MB it slows to 129.8 as the caching provided by Win2K Pro is squeezed beyond it’s limits. If you compare this to the Compaq at 256MB you get a result of 128.9. An EIDE drive faster than Ultra Wide SCSI, and all because of RAM! See figure 3.

Test 4 – Logon Times

In this final test I have timed the users logon process for a local logon (one into the local SAM) without the Active Directory or any form of roaming profile configured. As stated earlier this process is often how users judge the speed of the system, it is completely wrong for them to do this but that is the way it is so I have recorded the results here to help you see the differences. See figure 4.



I was fascinated to see how slowly the Dual Pentium system performed this logon, and at time of writing this I have no explanation as to why this is! I can only presume I have uncovered a bug in the code when it is running on this system, more investigation is required.

The remaining results are all rather high reflecting the change in the complexity of the user profiles which are now more exposed to allow for the powerful new remote management features in the Zero Administration for Windows initiative or ZAW. "You don’t get something for nothing" in this world and in this case it is the logon times which are bound to increase as these more complex profiles are parsed.

Summary


Before I summarise the results of these tests I must thanks a few people for their help in making these tests possible. Firstly to Nikola Ellis of "Kingston Technology" www.kingston.com for her support and for supplying the various memory configurations required. Secondly thanks to all at Upgrade Options www.pcupgrader.com for providing the test systems and especially to Andy Cordial for meeting impossible deadlines without complaint.

Well, over several weeks of testing I know have some very interesting results. Firstly Beta 3 for Win2K Professional is in good shape when you consider the amount of thrashing I have put it through it has performed outstandingly. Secondly, Win2K Pro is not the memory hog some of us had feared in fact it compares very favourably with NT4 Workstation in most of these tests. The fact that, true to it heritage, Windows 2000 loves RAM will come as no surprise to those of us who have been using and support NT on the desktop for years. However, for those of you who still primary support either Windows 3.x or Windows 9x as your corporate desktops I hope that these test have left you under no illusions. If you are going to be rolling out Windows 2000 you have to be looking at 128MB as your starting point…64MB is the bare minimum and 32MB is a non-starter!

If your budget is tight and you have to choose between the latest xxxMHz processor or extra memory, then consider this: -

The difference in price between the Pentium II 450MHz and the AMD K6 or Celeron (if you want Slot 1) 400Mhz is currently around £250 an extra 128MB of SDRAM is about £108. So you could have the Celeron and 256MB of RAM for less than the PII on it’s own! If you’ll forgive me the misquote…"SHOW ME THE MEMORY!"

RAM is Windows 2000’s friend and if it is happy, your users are happy and that will make your life a lot easier!

I would like to thank a few people for making these tests possible. Firstly to Nikola Ellis of Kingston Technology for her support and for supplying the various memory configurations required. Secondly, thanks to all at Upgrade Options for providing the test systems and especially to Andy Cordial for meeting impossible deadlines without complaint. -
Richard Harrison, March 1999