The boards used here weren't really designed for hash breaking. However, this application is so simple in terms of chip interconnect that it can be fitted into virtually any FPGA-based hardware. Thanks to a JTAG reverse-engineering tool written for this project, enough connectivity could be resolved to do that and 2 boards were good enough (after small repairs) to run NSA@home.

The boards contain 15 Virtex-II Pro (XC2VP20) FPGAs in 3 identical sets of 5 (here called "channels"). Each channel also owns a Spartan-II (XC2S50) FPGA that was originally used as a control chip, and a DSP (ADSP21160M) which probably calculated transform parameters. There is also a shared XC2S50 chip, which is not used in this application, just like the DSPs. The clock distribution tree unfortunately contains 2 domains, which means the 39MHz channel clock had to be distributed from chip to chip, using internal Virtex-II DCMs to clean it up.

To control the boards, an USB boardlet has been built and the Spartan-II chips programmed to share the USB interface using a ring protocol. Spartan-II chips translate USB-carried command packets into bus transactions (16-bit address, 18-bit data) and the bus read data into USB packets again. They also service SHA1 breaker IRQs to minimize the number of results that had to be merged into one - there is a buffer for 1024 18-bit reads in each Spartan-II.

The Spartan-IIs also drive a serially-controlled 16-character LED matrix display (2x Siemens/Infineon SCD5583A) that is used to show the current board state.

The boards were notably inconvenient to use without an enclosure. Since the original case for them was a custom-built card cage - dimensions are nonstandard - the boards were housed into hacked-up computer 1u cases. As each board dissipates about 120W while operating, six ducted fans are required to provide cooling. The internal arrangement of the 1u case can be seen below.

After some debugging, which involved issues like signal integrity inside and outside the FPGAs, overheating, power supply issues but no digital design bugs, the boards successfully booted. A closed case with a board displaying "Hello World" looks like this: