Locked HGST drives: Difference between revisions

From MTU LUG Wiki
Jump to navigation Jump to search
(added problems 2 and 3 and fixed up a lot of stuff)
(fixed formatting stuffs)
Line 20: Line 20:
Of note, formatting with the ten-byte variant of <code>MODE SENSE</code>+<code>MODE SELECT</code> (the default) versus the six-byte variant (toggled with <code>--six</code>) will depend on what brand of disk you're using.
Of note, formatting with the ten-byte variant of <code>MODE SENSE</code>+<code>MODE SELECT</code> (the default) versus the six-byte variant (toggled with <code>--six</code>) will depend on what brand of disk you're using.


Netapp needs ten-byte, while our drives should need six-byte (based on a comment from someone in a ServeTheHome thread (find the link for this)
* Netapp needs ten-byte, while our drives should need six-byte (based on a comment from someone in a ServeTheHome thread, find the link for this)
* We probably won't know for sure until we unlock the firmware ourselves though.

We probably won't know for sure until we unlock the firmware ourselves though.


== Problem 2: (Hitachi VSP firmware) ==
== Problem 2: (Hitachi VSP firmware) ==
The primary issue with these drives is the firmware that only allows them to work with Hitachi VSP systems.
The primary issue with these drives is the firmware only allows them to work with Hitachi VSP systems.


They will experience I/O errors when used on a regular system, even after having been reformatted to 512-byte sectors.
* They will experience I/O errors when used on a regular system, even when reformatted to 512-byte sectors.
* Apparently it is related to the firmware only accepting <code>SCSI_WRITE_WITH_VERIFY</code> commands [https://old.reddit.com/r/DataHoarder/comments/7raoz8/firmware_for_hgst_10k_sas_huc109090css600/idxmolz/]


These exact physical drives are also sold as OEM variants that work in generic systems without issue.
Apparently it is related to the firmware only accepting <code>SCSI_WRITE_WITH_VERIFY</code> commands [https://old.reddit.com/r/DataHoarder/comments/7raoz8/firmware_for_hgst_10k_sas_huc109090css600/idxmolz/]


These exact physical drives are also sold as general consumer and OEM variants that work in generic systems without issue. The only difference between our drives and those drives is the differing firmware.
* The only difference between our drives and those drives is the firmware.
* It should be possible to put OEM firmware onto the disks to allow them to work in our servers. (this leads into problem 3)

It should be possible to flash the firmware from the OEM-branded variants onto our drives to enable them to work.


== Problem 3: (Firmware lock) ==
== Problem 3: (Firmware lock) ==
In addition to the previous problem, the drives also have a lock that prevents rewriting the firmware.
In addition to the previous problem, the drives also have a lock that prevents rewriting the firmware.


The disks require a specific command be sent before they will allow overwriting the firmware.
* They require a specific command be sent before they will allow overwriting the firmware.


=== Russian dude (WD Niagara) ===
There is a Russian dude who, for the price of ~$10/drive, will RDP into a Windows environment and flash the drives for you. [https://forum.hddguru.com/viewtopic.php?f=22&t=39927&sid=121a6e12d2d1fdaf816fe00eedac1745]
There is a Russian dude who, for the price of ~$10/drive, will RDP into a Windows environment and flash the drives for you. [https://forum.hddguru.com/viewtopic.php?f=22&t=39927&sid=121a6e12d2d1fdaf816fe00eedac1745]


From what I've gathered, he runs a custom script he made that uses WD Niagara (an internal Western Digital tool intended OEMs for advanced HDD operations) to unlock the drives, then uses Niagara's built-in "firmware download" to flash the firmware.
He runs a custom script he made that uses WD Niagara (an internal Western Digital tool intended OEMs for advanced HDD operations) to unlock the drives, then uses Niagara's built-in "firmware download" to flash the firmware.

Should we have the money, this is the easiest route to solve the problem. However, by that point, it makes more sense to just buy new drives.


He has uploaded a video of the process to YouTube [https://www.youtube.com/watch?v=mAhS_sk3wKE]
He has uploaded a video of the process to YouTube [https://www.youtube.com/watch?v=mAhS_sk3wKE]


If we had the money, this would be the easiest route to solve the problem. However, it'd make more sense to just buy new drives.
According to a Chinese dude who paid for Hydata SCSITools in another ServeTheHome thread, this program can also send the command to unlock the drives and flash the firmware.


=== Hydata SCSITools ===
SCSITools is $2800 for a license, so same as above, at that price it makes more sense to buy different drives.
According to a Chinese dude who paid for Hydata SCSITools in another ServeTheHome thread, this program can also send the command to unlock the drives and flash the firmware.


A SCSITools license is $2800. Same as with the Russian dude, at that price it makes more sense to buy different drives.
SartenX recommended asking them for a free license as students, I doubt they'd hand out a free license though.


* SartenX recommended asking them for a free license as students, I seriously doubt they'd give us one though.
Preliminarily poking at the Windows program, the check for a license key seems really basic.


Poking at the Windows program, the check for a license key seems really basic.
It just pulls in <code>hasp_windows_102966.dll</code> which is the Sentinel HASP (Hardware Against Software Piracy) all-in-one check that looks for a valid USB hardware key plugged into the computer.


* It just pulls in <code>hasp_windows_102966.dll</code> which is the Sentinel HASP (Hardware Against Software Piracy) all-in-one check that looks for a valid USB hardware key plugged into the computer.
I doubt fiddling with Sentinel HASP would be easy, it has checks for VMs, emulated dongles, and even WINE for some reason, but the program itself could probably be modded to ignore the check.
* I doubt fiddling with Sentinel HASP would be easy, it has checks for VMs, emulated dongles, and even WINE for some reason
* However, the SCSITools binary itself could probably be modded to ignore the HASP check


=== Hardware ===
We also considered flashing the drives via the hardware, with a Pomona SOIP8 clip (SOIP8 is the same form-factor as SOP8)
We also considered flashing the drives via the hardware, with a Pomona SOIP8 clip (SOIP8 is the same form-factor as SOP8)


Line 66: Line 68:
We identified the following chips on the drive's mainboard:
We identified the following chips on the drive's mainboard:


* Smooth L7228: Drive Spindle controller?
# Smooth L7228
## Drive Spindle controller?
* [https://www.ariat-tech.com/parts/samsung-semiconductor/K4T51163QJ-BCE7 Samsung K4T51163QJ-BCF8]: DDR2 memory
# [https://www.ariat-tech.com/parts/samsung-semiconductor/K4T51163QJ-BCE7 Samsung K4T51163QJ-BCF8]
* LSI TNNKU873 BJR12034: microcontroller
## DDR2 memory
* MAXIC 25U40325 M1-126: the SOP8 ROM (this is the big one)
# LSI TNNKU873 BJR12034
## Microcontroller
# MAXIC 25U40325 M1-126
## SOP8 ROM (this is the big one)


Apparently only disk-specific calibration information is stored on the ROM and not the actual firmware.
Apparently only disk-specific calibration information is stored on the ROM and not the actual firmware.


^ (Josh found a post from someone trying to do exactly what we planned, and came to this conclusion. Post forum link here)
^ (Josh found a forum post from someone trying what we planned, and came to this conclusion. Josh plz post link here)


So the hardware route is a dead-end :(
So the hardware route is a dead-end :(

Revision as of 21:10, 28 November 2024

Some time ago, we purchased ~25x 1.2TB HGST HUC101212CSS600 drives off eBay for use in our servers.

These disks have three primary issues:

  1. They are formatted with 520-byte sectors. This prevents them from working with Dell PERC and HP Smart Array controllers, even when the controllers are set to "HBA-mode". They are only usable with a true HBA (or a controller flashed to the 3rd-party "IT mode" firmware).
  2. They have Hitachi Vantara firmware [1] that locks them to Hitachi VSP systems and doesn't let them work with normal servers.
  3. They have a lock preventing them from being reflashed with the Dell [2] firmware that would allow them to be used in generic systems.

Problem 1: (520-byte sectors)

This is a very common problem with decommissioned enterprise SAS drives and also typically very easy to fix.

All you need is a "true" HBA in the server that is reformatting the disks. Dell PERC cards and HP Smart Array controllers cannot communicate with 520-byte sector disks whatsoever, even when put into "HBA mode".

The cards inside the Kurisu and Okabe servers have been flashed with the "IT mode" firmware, as their PERC cards were old enough to be flashed using a script).

The sector size of a disk can be reformatted to 512 bytes using sg3utils with the command below:

sg_format -v --format --size=512 --six /dev/sgX

Of note, formatting with the ten-byte variant of MODE SENSE+MODE SELECT (the default) versus the six-byte variant (toggled with --six) will depend on what brand of disk you're using.

  • Netapp needs ten-byte, while our drives should need six-byte (based on a comment from someone in a ServeTheHome thread, find the link for this)
  • We probably won't know for sure until we unlock the firmware ourselves though.

Problem 2: (Hitachi VSP firmware)

The primary issue with these drives is the firmware only allows them to work with Hitachi VSP systems.

  • They will experience I/O errors when used on a regular system, even when reformatted to 512-byte sectors.
  • Apparently it is related to the firmware only accepting SCSI_WRITE_WITH_VERIFY commands [3]

These exact physical drives are also sold as OEM variants that work in generic systems without issue.

  • The only difference between our drives and those drives is the firmware.
  • It should be possible to put OEM firmware onto the disks to allow them to work in our servers. (this leads into problem 3)

Problem 3: (Firmware lock)

In addition to the previous problem, the drives also have a lock that prevents rewriting the firmware.

  • They require a specific command be sent before they will allow overwriting the firmware.

Russian dude (WD Niagara)

There is a Russian dude who, for the price of ~$10/drive, will RDP into a Windows environment and flash the drives for you. [4]

He runs a custom script he made that uses WD Niagara (an internal Western Digital tool intended OEMs for advanced HDD operations) to unlock the drives, then uses Niagara's built-in "firmware download" to flash the firmware.

He has uploaded a video of the process to YouTube [5]

If we had the money, this would be the easiest route to solve the problem. However, it'd make more sense to just buy new drives.

Hydata SCSITools

According to a Chinese dude who paid for Hydata SCSITools in another ServeTheHome thread, this program can also send the command to unlock the drives and flash the firmware.

A SCSITools license is $2800. Same as with the Russian dude, at that price it makes more sense to buy different drives.

  • SartenX recommended asking them for a free license as students, I seriously doubt they'd give us one though.

Poking at the Windows program, the check for a license key seems really basic.

  • It just pulls in hasp_windows_102966.dll which is the Sentinel HASP (Hardware Against Software Piracy) all-in-one check that looks for a valid USB hardware key plugged into the computer.
  • I doubt fiddling with Sentinel HASP would be easy, it has checks for VMs, emulated dongles, and even WINE for some reason
  • However, the SCSITools binary itself could probably be modded to ignore the HASP check

Hardware

We also considered flashing the drives via the hardware, with a Pomona SOIP8 clip (SOIP8 is the same form-factor as SOP8)

There are pictures of the mainboard in this HDDGuru thread [6]

We identified the following chips on the drive's mainboard:

  1. Smooth L7228
    1. Drive Spindle controller?
  2. Samsung K4T51163QJ-BCF8
    1. DDR2 memory
  3. LSI TNNKU873 BJR12034
    1. Microcontroller
  4. MAXIC 25U40325 M1-126
    1. SOP8 ROM (this is the big one)

Apparently only disk-specific calibration information is stored on the ROM and not the actual firmware.

^ (Josh found a forum post from someone trying what we planned, and came to this conclusion. Josh plz post link here)

So the hardware route is a dead-end :(