Building a zk/ml rig with two nvidia 4090s

The last time I built a custom PC was back in college. After saving up during the summer, I went all out, purchasing the latest consumer hardware available on the market. At that time, it consisted of an Intel quad-core processor, 4GB of DDR3 RAM, an external water cooler, and several hard drives totaling 1TB of space. In front of all this was a giant 42-inch HDTV, and a recliner chair where I could spend as much time as I wanted. 

Since then, I've primarily ventured down software rabbit holes and opted for purchasing prepackaged Apple products.

This new setup was put over the xmas break, but I had to wait ~3 months for some of the orders on this list. 

The result is pretty awesome! 

The specs

Processor: intel i9-13900
Memory: gskill 4 x 32GB ram
Disk: 2x2 TB NVMe
Motherboard: rog asus z790
GPU: 2 x nvidia 4090
Case: cosmos 700m case

Challenges

It was quite challenging to find a case that would accommodate two 4090s and arrive promptly. Each comes with an integrated 3-fan radiator, taking up considerable space. The motherboard features 3 PCIe slots. Initially, I hoped to fit one card at the top and another at the bottom, leaving the middle slot free for some extensions. However, this plan was not feasible. Placing the graphics card in the top slot completely covers the middle slot. When positioning the card in the middle or bottom slot, it obstructs the smaller motherboard connections (fan/light sensors), causing them to bend. Consequently, I had to use a PCIe extension to the side and cannot use the 3rd covered slot.

The hardware that comes with the Cosmos 700M for additional modifications appears comprehensive at first glance. Yet, I found nothing that would securely mount the card on its side. Thus, I had to fabricate a custom attachment. It's also important to monitor airflow to ensure the fans are blowing sideways (since there is another set of 3-fans in the case itself for air intake, directing airflow in the opposite direction would cause conflicting head-on airflows).

The cabling setup was quite extensive, hiding most of the wires behind the motherboard.

It took several hours for everything to boot after connecting. Initially, the screen remained dark, and the motherboard displayed various memory-related error codes. These issues were resolved by unplugging and then replugging each memory stick one by one. It seems one might not have been inserted correctly. I still encounter some controller errors during the boot sequence but haven't found any helpful information about them online.

I installed Ubuntu 22.04. Most things worked seamlessly, except for the graphics cards. After installing all necessary drivers, one issue persists: I cannot resume from sleep mode. This only occurs when the GPUs are utilized. For example, when the monitor is connected to the integrated motherboard video and the GPUs are not used for any other tasks, I can resume normally. However, when the GPUs are engaged for the monitor or other activities, resuming is not possible. Despite trying various solutions from online searches, I haven’t found a fix yet.

Overall, assembling this setup was a thrilling experience, GPUs are top notch, Ubuntu is awesome, and I'm looking forward to testing some systems on it!