Status Update

[Bitweasil]

*whistles innocently*

Code: Select all

rgraves@yeti:~$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             133G  5.9G  121G   5% /
varrun                5.9G   48K  5.9G   1% /var/run
varlock               5.9G     0  5.9G   0% /var/lock
udev                  5.9G   84K  5.9G   1% /dev
devshm                5.9G     0  5.9G   0% /dev/shm
/dev/md0              6.9T  5.5M  6.9T   1% /mnt/array1


[Bitweasil]

Importing a few wordlists for use now. I should end up with around 100M words/phrases/etc, lowercase, to mutate. Maybe more, I'm not entirely sure how many unique there are.

[blazer]

aww if the board doesn't support support Double width cards, doesn't that make it a piece of junk, do nvidia make single width cards? or is it only ATi who do that?

[Bitweasil]

aww if the board doesn't support support Double width cards, doesn't that make it a piece of junk, do nvidia make single width cards? or is it only ATi who do that?

nVidia and ATI both make single width cards. The problem is that they're not terribly powerful. For instance, the 8500/8600 series PCI-E cards are both single slot - but they're 16/32 stream processor cards.

All the "good" cards (8800, 9800, GTX*) are double slot. I suppose one could make them single slot with the addition of a good water cooler. That's a lot of complexity and power, though - running 5-6 GTX series cards in a single system is on the "Challenging to borderline impossible" side of things. And you don't do it with the double-PCB cards at all (9800GX2, GTX295), as they have two PCBs, not just a cooler doubling the height.

[mrb]

Does this answer your question?
It's an Asus P6T6 Revolution board.

For some reason I thought you were going to buy the K9A2. But thanks, I am also interested by the data for the P6T6. I see that 3 slots have a PowerLimit value of 75, 25, and 25W. That must be the 3 blue slots one is supposed to use for the dual-slot cards. The remaining slots have the value set to 10W, so I guess a high-end 200+W card would refuse to even POST when detecting that. For that reason I think that even if flexible PCI-E extenders were used, it would not be possible to use that board with 6 high-end video cards.

[Bitweasil]

How much power does a GPU actually pull from the PCI-E slot, though? All of the high end ones have their own (massive) PCI-E power connectors - the high end ones have a 6 pin & an 8 pin, the less-high-end ones have two 6 pins. They're going to be pulling most of their power through that, instead of through the board.

I could try moving the 8800GTX around into a few different slots and see if it matters - that card draws a good bit of power, so if it's happy in a 10W slot, it's likely that the card isn't pulling anything significant from the slot itsself.

[mrb]

The PCI-E spec allows cards to pull theoretically up to 75W from the slot, 75W from a 6-pin power connector, 150W from a 8-pin connector, and the total must not exceed 300W (this prohibits, for example, cards with two 8-pin connectors). But because the slot power limit is software-controlled and doesn't always allow the full 75W (as evidenced from the PowerLimit values on the P6T6), cards usually try to pull most of the power from the extra power connectors.

As of today the 2 most power-hungry cards are the GTX 295 and the HD 4870 X2: 290W each. Which means they could pull up to 290-75-150 = 65W from the slot, and 225W from the power connectors. In my experience though, with my GPGPU apps and a rough approximation with a kill-a-watt, I can never get a card pull more than about 80% of its rated power usage, which would mean (290*.80)-75-150 = 7W from the slot.

I am personally going to buy a PCI-E extender so I can isolate the 12V rails, and buy a clamp-on ammeter to get more precise numbers and satisfy my curiosity

[Bitweasil]

I'll be interested in seeing what you end up with.

My wordlist has 64M entries in it now. Most of the stuff floating around the internet, surprise, was redundant.

8.5GB of InnoDB tables, fully indexed. All lowercase.

Now to mutate & precompute some MD5/NTLM/etc tables for it...

[mrb]

Today, I bought a Craftsman clamp-on meter and measured the power consumption of 2 of my video cards, at idle and 100% ALU load (with one of my GPGPU app - an MD5 bruteforcer):

Reference design Radeon HD 4850 (114W max TDP):

• PCI-E slot 3.3V rail idle: 0.8A (2.6W), load: 0.8A (2.6W)
• PCI-E slot 12V rail idle: 1.0A (12W), load: 3.3A (40W)
• PCI-E 6-pin 12V power connector idle: 2.4A (29W), load: 4.6A (55W)
• Overall idle: 43W (38% of max TDP), load: 97W (85% of max TDP)

Sapphire Radeon HD 4850 X2 (230W max TDP):

• PCI-E slot 3.3V rail idle: 1.2A (4.0W), load: 1.2A (4.0W)
• PCI-E slot 12V rail idle: 2.4A (29W), load: 2.4A (29W)
• PCI-E 6-pin & 8-pin 12V power connectors idle: 4.7A (56W), load: 14.2A (170W)
• Overall idle: 89W (39% of max TDP), load: 203W (88% of max TDP)

(I actually still haven't received my flexible PCI-E extender. So the way I obtained measurements for the PCI-E slot power rails was by comparing the draw on all the PSU output rails (-12V, -5V, +3.3V, +5V, +5VSB, +12V) with and without a video card. Also according to my clamp meter doc, it is only 5% accurate. So take these numbers with a grain of NaCl.)

First remark: for both cards, power consumption on the 3.3V rail is negligible (< 4W) and never seems to vary whether at idle or 100% load, which I expected since they draw most of their power from the 12V rails.

Also, notice how the 4850 was carefully designed to ramp up the load equally on the two 12V rails (slot & power connectors) as the load increases to 100%: 3.3A on the slot 12V rail corresponds to 75% of the 4.4A allowed by the PCI-E 1.0 spec, and 4.6A on the 6-pin connector corresponds to 74% of the 6.25A allowed by the spec.

Conversely the 4850 X2 was designed to limit the current at 2.4A on the slot 12V rail whether idle or under load. Which means if one was to design a system with 4 GPUs, it would be much easier to find a mobo capable of supplying power to 2 4850 X2 (2*2.4A = 4.8A to the slot 12V rails) instead of 4 4850 (4*3.3A = 13.2A to the slot 12V rails) !

Also I cannot seem to make them consume more than 85% or 88% of their max TDP. The rough estimation I made earlier with my kill-a-watt ("not able to reach more than about 80% of the TDP") was pretty accurate

The definitive answer as to how much current is drawn from the slot (3.3V and 12V rails) at max load is 42.6W for the 4850 and 33W for the 4850 X2. Which raises the question: would they even work in a slot advertising a 25W PowerLimit such as 2 of the blue slots on the P6T6 ? From experience, I would say yes. I don't have access to the latest PCI-E specs (they are not free). But maybe an explanation is that the PowerLimit value only applies to the 12V rail. Maybe there is a high +/-10% tolerance. Or maybe vendors don't fully respect the specs and implement hacks/workarounds for power-hungry video cards.

[Bitweasil]

Impressive...