Of course there are many many ways to configure any workstation. With the TYAN FT77A removing most of the guesswork around motherboard, case and PSU selection, our options our discussion will be brief and focused around CPU, RAM and GPU options.
CPU: There are quite a few E5-2600 series Xeons, ranging anywhere from 4C/9T to 8C/16T and $310~$1900+ a piece. Yes, you will be using two…
Depending on how you envision your workflow with this system, you might configure it to be almost 100% dedicated to GPU computing (in which case a couple of affordable E5-2609 2.4GHz Quad cores will be sufficient), to a fully capable CPU rendering workstation combined with 7-8 fast GPUs, in which case you should most likely opt for the fastest pair of 8-core E5 Xeons you can afford.
The 2.4GHz Octa-core E5-2665 is almost five times the price of the 2609 Quad, but since the 8C E5-2670 @ 2.6GHz is just 5% more, I would probably go for it should I decide to reach deep in my pockets. The best value in s2011 Xeons should be awarded to the E5-2620, the cheapest of the series 6C with Hyperthreading, yet probably people seriously considering a FT77A system for something more than a GPGPU platform, would opt for a 8C/16T.
Cooling: Most likely CPUs will be air-cooled. There is a large number of s2011 air coolers that can comfortably fit in a 4U case. With the FT77A, most likely you could even use a couple of 120mm AIO water coolers to be mounted behind the large intake fans between the HDDs and the motherboard bay. Thick 120mm radiator based units should be fair game, though most likely one of the supplied fans would be left out:
GPU(s): The FT77A was most likely designed with nVidia Tesla’s in mind. There is an onboard GPU chip that allows you to populate all 8 slots with head-less Tesla cards, should you wish to go for a remote GPGPU solution.
Of course those cards come with an impressive (or intimidating?) pricetag, with each K20X being in the $4,600 range, and the K20 around $3,500. With the introduction of the GTX Titan, users interested in the GK110 can have a card with as much ram as a K20X and faster clocks for around $1,000. It is perhaps the only scenario under which you could consider the GTX Titan a “bargain”.
All GTX Titans are based on the reference design and are sourced directly by nVidia.
In the US GTX Titans are importer exclusively by ASUS and EVGA.
Keep in mind that GTX Titans are in great demand and limited supply. The specialized “factory overclock” versions are kind of a gimmick, as all of the are sharing the same boards , VRMs and cooler. You can overclock all of them to similar levels just as well should you wish to. You could also mix and match Asus and EVGA cards without issue, or even mix any high power GTX card to an unlimited number of combinations.
Titans are far from the only viable scenario: 7-8x of any fast GTX card will be able to give impressive results. Any of the cards below would be a great choice, given those fit within our VRam buffer requirements, and budget:
- GTX 780 3GB – 2304 CUDA Cores x 900* MHz ~ 88% of a GTX Titan
- GTX 680 4GB – 1536 CUDA Cores x 1058*MHz ~ 69% of a GTX Titan
- GTX 670 4GB – 1344 CUDA Cores x 970* MHz ~ 55% of a GTX Titan
* Reference speed: most available 680s and 670s with 4GB buffer are factory clocked higher for little or no price difference with the reference models. Unlike the Titan, customized boards and cooling solutions on those do yield better O/C results.
Keep in mind that under most loads, 2x GTX 680s or even 2x GTX 670s will be faster than a single Titan, while cost less. If you don’t have the money to invest into 6+ Titans or GTX 780s, a full rack of 8x 680s will be a faster choice with better value – given the 4GB max buffer is enough for the tasks you usually undertake. For tasks or GPU renderings that are not VRam limited, the new GTX 780 3GB, based on the same GK110 with a couple computation cluster disabled and ofc half the RAM. A GTX 780 with 2304 CUDA cores is within 10% of a GTX Titan’s performance.
Since GPGPU computing is not limited to an absolute max number of GPUs, you could even go for 8x GTX 690s for a total of 16x GK104 GPUs or more than 24,670 Kepler CUDA processing units in a single box! Unfortunately, unlike the label that mentions 4GB or VRam, only 2GB are available to each GPU, which might be too little for some demanding applications, while there is a high probability that 8x 690s under full load might surpass the available capacity the 2x 1200W PSUs can reliably provide for prolonged time.
A FT77A with a full rack of GTX Titan cards attached.
Image source: http://www.soa-world.de/echelon/
Should you choose to use your FT77A as your primary workstation and not just a GPGPU compute server, you could consider your 8th card being a Quadro (or Firepro) workstation card that will give you fluid viewport performance, while 7x “gaming” GPUs will be dedicated to computation tasks.
This baby “Titan” supercomputer is a bit ahead of its time for the average CG Artist, and very few will be lucky to have a GPGPU monster like that to actually play with. Fully decked out, and without going crazy with system RAM, a FT77A with 8x Titans and a couple of E5-2665 Xeons can be above the $17,000 range, while with a more conservative approach using 8x 680s and vanilla E5-2605 Quads you could aim at around $10,000 as a realistic budget – before taxes/vat. For most interested in custom PC builds, this might sound insane, but once you venture into the pre-configured CG Workstation world of BOXX, Dell, HP and Lenovo, those figures are fair game for high-end workstations, far less potent than this one in the GPGPU and perhaps even for direct CPU computing.
Nevertheless, this blog was written mainly to serve as food for thought and help you understand the possibilities once devs for CG products start getting more involved with GPU acceleration, and not a pee-contest on how much money one can throw into a 4U rack.
Hardware and Software advancements have always been playing “chickens and eggs”, with either sides taking turns in making the next big step.