Locked HGST drives - Revision history

D2wn: 520-byte sector changes

2024-11-29T02:18:03Z

520-byte sector changes

← Older revision		Revision as of 02:18, 29 November 2024
Line 10:		Line 10:
	This is a very common problem with decommissioned enterprise SAS drives and also typically very easy to fix.		This is a very common problem with decommissioned enterprise SAS drives and also typically very easy to fix.

	All you need is a "true" HBA in the server that is reformatting the disks~~. Dell PERC cards and HP Smart Array controllers cannot communicate with 520-byte sector disks whatsoever, even when put into "HBA mode"~~.		All you need is a "true" HBA in the server that is reformatting the disks.

			Dell PERC cards and HP Smart Array controllers cannot communicate with 520-byte sector disks whatsoever, even when put into "HBA mode".
⚫	The cards inside the [[Servers\|Kurisu and Okabe]] servers have been flashed with the "IT mode" firmware, as their PERC cards were old enough to be flashed using [https://fohdeesha.com/docs/perc.html a script]).

		⚫	The cards inside the [[Servers\|Kurisu and Okabe]] servers have been flashed with the "IT mode" firmware, as their PERC cards were old enough to be flashed using [https://fohdeesha.com/docs/perc.html a script].

	The sector size of a disk can be reformatted to 512 bytes using <code>sg3utils</code> with the command below:		The sector size of a disk can be reformatted to 512 bytes using <code>sg3utils</code> with the command below:
Line 18:		Line 20:
	<code>sg_format -v --format --size=512 --six /dev/sgX</code>		<code>sg_format -v --format --size=512 --six /dev/sgX</code>

	Of note, ~~formatting~~ with the ten-byte variant of <code>MODE SENSE</code>+<code>MODE SELECT</code> (the default) versus the six-byte variant (toggled with <code>--six</code>) will depend on what brand of disk you're using.		Of note, needing to format with the ten-byte variant of <code>MODE SENSE</code>+<code>MODE SELECT</code> (the default) versus the six-byte variant (toggled with <code>--six</code>) will depend on what brand of disk you're using.

			* Netapp needs ten-byte
	* ~~Netapp~~ ~~needs ten-byte, while our~~ drives should need six-byte (based on a comment from someone in a ServeTheHome thread, find the link for this)		* Our HGST drives should need six-byte (based on a comment from someone in a ServeTheHome thread, find the link for this)
	* We probably won't know for sure until we unlock the firmware ourselves though.

	== Problem 2: (Hitachi VSP firmware) ==		== Problem 2: (Hitachi VSP firmware) ==

D2wn: Mediawiki settings

2024-11-29T02:13:13Z

Mediawiki settings

← Older revision		Revision as of 02:13, 29 November 2024
Line 82:		Line 82:

	So the hardware route is a dead-end :(		So the hardware route is a dead-end :(

			__FORCETOC__
			__NOEDITSECTION__

D2wn at 02:11, 29 November 2024

2024-11-29T02:11:32Z

← Older revision		Revision as of 02:11, 29 November 2024
Line 42:		Line 42:
	There is a Russian dude who, for the price of ~$10/drive, will RDP into a Windows environment and flash the drives for you. [https://forum.hddguru.com/viewtopic.php?f=22&t=39927&sid=121a6e12d2d1fdaf816fe00eedac1745]		There is a Russian dude who, for the price of ~$10/drive, will RDP into a Windows environment and flash the drives for you. [https://forum.hddguru.com/viewtopic.php?f=22&t=39927&sid=121a6e12d2d1fdaf816fe00eedac1745]

	He runs a custom script he made that uses WD Niagara (an internal Western Digital tool intended ~~OEMs~~ for ~~advanced~~ ~~HDD~~ ~~operations~~) to unlock the drives, then uses Niagara's built-in "firmware download" to flash the firmware.		He runs a custom script he made that uses WD Niagara (an internal Western Digital tool intended for OEMs like Dell/HP) to unlock the drives, then uses Niagara's built-in "firmware download" to flash the firmware.

	He has uploaded a video of the process to YouTube [https://www.youtube.com/watch?v=mAhS_sk3wKE]		He has uploaded a video of the process to YouTube [https://www.youtube.com/watch?v=mAhS_sk3wKE]

D2wn: fixed formatting stuffs

2024-11-29T02:10:46Z

fixed formatting stuffs

← Older revision		Revision as of 02:10, 29 November 2024
Line 20:		Line 20:
	Of note, formatting with the ten-byte variant of <code>MODE SENSE</code>+<code>MODE SELECT</code> (the default) versus the six-byte variant (toggled with <code>--six</code>) will depend on what brand of disk you're using.		Of note, formatting with the ten-byte variant of <code>MODE SENSE</code>+<code>MODE SELECT</code> (the default) versus the six-byte variant (toggled with <code>--six</code>) will depend on what brand of disk you're using.

	Netapp needs ten-byte, while our drives should need six-byte (based on a comment from someone in a ServeTheHome thread (find the link for this)		* Netapp needs ten-byte, while our drives should need six-byte (based on a comment from someone in a ServeTheHome thread, find the link for this)
		⚫	* We probably won't know for sure until we unlock the firmware ourselves though.

⚫	We probably won't know for sure until we unlock the firmware ourselves though.

	== Problem 2: (Hitachi VSP firmware) ==		== Problem 2: (Hitachi VSP firmware) ==
	The primary issue with these drives is the firmware ~~that~~ only allows them to work with Hitachi VSP systems.		The primary issue with these drives is the firmware only allows them to work with Hitachi VSP systems.

	They will experience I/O errors when used on a regular system, even ~~after having been~~ reformatted to 512-byte sectors.		* They will experience I/O errors when used on a regular system, even when reformatted to 512-byte sectors.
		⚫	* Apparently it is related to the firmware only accepting <code>SCSI_WRITE_WITH_VERIFY</code> commands [https://old.reddit.com/r/DataHoarder/comments/7raoz8/firmware_for_hgst_10k_sas_huc109090css600/idxmolz/]

			These exact physical drives are also sold as OEM variants that work in generic systems without issue.
⚫	Apparently it is related to the firmware only accepting <code>SCSI_WRITE_WITH_VERIFY</code> commands [https://old.reddit.com/r/DataHoarder/comments/7raoz8/firmware_for_hgst_10k_sas_huc109090css600/idxmolz/]

	~~These exact physical drives are also sold as general consumer and OEM variants that work in generic systems without issue.~~ The only difference between our drives and those drives is the ~~differing~~ firmware.		* The only difference between our drives and those drives is the firmware.
			* It should be possible to put OEM firmware onto the disks to allow them to work in our servers. (this leads into problem 3)

	It should be possible to flash the firmware from the OEM-branded variants onto our drives to enable them to work.

	== Problem 3: (Firmware lock) ==		== Problem 3: (Firmware lock) ==
	In addition to the previous problem, the drives also have a lock that prevents rewriting the firmware.		In addition to the previous problem, the drives also have a lock that prevents rewriting the firmware.

	~~The~~ ~~disks~~ require a specific command be sent before they will allow overwriting the firmware.		* They require a specific command be sent before they will allow overwriting the firmware.

			=== Russian dude (WD Niagara) ===
	There is a Russian dude who, for the price of ~$10/drive, will RDP into a Windows environment and flash the drives for you. [https://forum.hddguru.com/viewtopic.php?f=22&t=39927&sid=121a6e12d2d1fdaf816fe00eedac1745]		There is a Russian dude who, for the price of ~$10/drive, will RDP into a Windows environment and flash the drives for you. [https://forum.hddguru.com/viewtopic.php?f=22&t=39927&sid=121a6e12d2d1fdaf816fe00eedac1745]

	~~From what I've gathered, he~~ runs a custom script he made that uses WD Niagara (an internal Western Digital tool intended OEMs for advanced HDD operations) to unlock the drives, then uses Niagara's built-in "firmware download" to flash the firmware.		He runs a custom script he made that uses WD Niagara (an internal Western Digital tool intended OEMs for advanced HDD operations) to unlock the drives, then uses Niagara's built-in "firmware download" to flash the firmware.

⚫	~~Should~~ we ~~have~~ the money, this is the easiest route to solve the problem. However~~, by that point~~, it ~~makes~~ more sense to just buy new drives.

	He has uploaded a video of the process to YouTube [https://www.youtube.com/watch?v=mAhS_sk3wKE]		He has uploaded a video of the process to YouTube [https://www.youtube.com/watch?v=mAhS_sk3wKE]

		⚫	If we had the money, this would be the easiest route to solve the problem. However, it'd make more sense to just buy new drives.
⚫	According to a Chinese dude who paid for Hydata SCSITools in another ServeTheHome thread, this program can also send the command to unlock the drives and flash the firmware.

			=== Hydata SCSITools ===
⚫	SCSITools is $2800 ~~for~~ a ~~license,~~ so ~~same as~~ ~~above~~, at that price it makes more sense to buy different drives.
		⚫	According to a Chinese dude who paid for Hydata SCSITools in another ServeTheHome thread, this program can also send the command to unlock the drives and flash the firmware.

		⚫	A SCSITools license is $2800. Same as with the Russian dude, at that price it makes more sense to buy different drives.
⚫	SartenX recommended asking them for a free license as students, I doubt they'd ~~hand~~ ~~out~~ ~~a free license~~ though.

		⚫	* SartenX recommended asking them for a free license as students, I seriously doubt they'd give us one though.
⚫	~~Preliminarily poking~~ at the Windows program, the check for a license key seems really basic.

		⚫	Poking at the Windows program, the check for a license key seems really basic.
⚫	It just pulls in <code>hasp_windows_102966.dll</code> which is the Sentinel HASP (Hardware Against Software Piracy) all-in-one check that looks for a valid USB hardware key plugged into the computer.

		⚫	* It just pulls in <code>hasp_windows_102966.dll</code> which is the Sentinel HASP (Hardware Against Software Piracy) all-in-one check that looks for a valid USB hardware key plugged into the computer.
	I doubt fiddling with Sentinel HASP would be easy, it has checks for VMs, emulated dongles, and even WINE for some reason~~, but the program itself could probably be modded to ignore the check.~~		* I doubt fiddling with Sentinel HASP would be easy, it has checks for VMs, emulated dongles, and even WINE for some reason
			* However, the SCSITools binary itself could probably be modded to ignore the HASP check

			=== Hardware ===
	We also considered flashing the drives via the hardware, with a Pomona SOIP8 clip (SOIP8 is the same form-factor as SOP8)		We also considered flashing the drives via the hardware, with a Pomona SOIP8 clip (SOIP8 is the same form-factor as SOP8)

Line 66:		Line 68:
	We identified the following chips on the drive's mainboard:		We identified the following chips on the drive's mainboard:

	* Smooth L7228~~: Drive Spindle controller?~~		# Smooth L7228
			## Drive Spindle controller?
	* [https://www.ariat-tech.com/parts/samsung-semiconductor/K4T51163QJ-BCE7 Samsung K4T51163QJ-BCF8]~~: DDR2 memory~~		# [https://www.ariat-tech.com/parts/samsung-semiconductor/K4T51163QJ-BCE7 Samsung K4T51163QJ-BCF8]
⚫	* LSI TNNKU873 BJR12034~~: microcontroller~~
			## DDR2 memory
⚫	* MAXIC 25U40325 M1-126: the SOP8 ROM (this is the big one)
		⚫	# LSI TNNKU873 BJR12034
			## Microcontroller
			# MAXIC 25U40325 M1-126
		⚫	## SOP8 ROM (this is the big one)

	Apparently only disk-specific calibration information is stored on the ROM and not the actual firmware.		Apparently only disk-specific calibration information is stored on the ROM and not the actual firmware.

	^ (Josh found a post from someone trying ~~to do exactly~~ what we planned, and came to this conclusion. ~~Post~~ ~~forum~~ link here)		^ (Josh found a forum post from someone trying what we planned, and came to this conclusion. Josh plz post link here)

	So the hardware route is a dead-end :(		So the hardware route is a dead-end :(

D2wn: added problems 2 and 3 and fixed up a lot of stuff

2024-11-29T01:55:31Z

added problems 2 and 3 and fixed up a lot of stuff

@@ Line 1: / Line 1: @@
-On XXXX we purchased ##x 1.2TB HGST HUC101212CSS600 drives off ebay for use in our servers.
+Some time ago, we purchased ~25x 1.2TB HGST HUC101212CSS600 drives off eBay for use in our servers.
 These disks have three primary issues:
@@ Line 6: / Line 6: @@
 # They have Hitachi Vantara firmware [https://www.reddit.com/r/homelabsales/comments/qazb88/pccan_huc101212css600_12tb_10k_sas_6g_25_drives/hh6frb5/] that locks them to Hitachi VSP systems and doesn't let them work with normal servers.
 # They have a lock preventing them from being reflashed with the Dell [https://www.dell.com/support/home/en-au/drivers/driversdetails?driverId=PJ9CD] firmware that would allow them to be used in generic systems.
 == Problem 1: (520-byte sectors) ==
@@ Line 13: / Line 12: @@
 All you need is a "true" HBA in the server that is reformatting the disks. Dell PERC cards and HP Smart Array controllers cannot communicate with 520-byte sector disks whatsoever, even when put into "HBA mode".
-The cards inside the [[Servers|Kurisu and Okabe]] servers have been flashed to "IT mode", as their PERC cards were old enough to be flashed using [https://fohdeesha.com/docs/perc.html a script]).
+The cards inside the [[Servers|Kurisu and Okabe]] servers have been flashed with the "IT mode" firmware, as their PERC cards were old enough to be flashed using [https://fohdeesha.com/docs/perc.html a script]).
 The sector size of a disk can be reformatted to 512 bytes using <code>sg3utils</code> with the command below:
@@ Line 19: / Line 18: @@
 <code>sg_format -v --format --size=512 --six /dev/sgX</code>
-Of note, <code>--six</code> and <code>--ten</code> depend on what brand of disk you are using.
+Of note, formatting with the ten-byte variant of <code>MODE SENSE</code>+<code>MODE SELECT</code> (the default) versus the six-byte variant (toggled with <code>--six</code>) will depend on what brand of disk you're using.

D2wn: added hardware notes

2024-11-22T04:07:28Z

added hardware notes

← Older revision		Revision as of 04:07, 22 November 2024
Line 26:		Line 26:



			hardware modding nodes:

			Smooth L7228: Drive Spindle controller?

			[https://www.ariat-tech.com/parts/samsung-semiconductor/K4T51163QJ-BCE7 Samsung K4T51163QJ-BCF8]: DDR2 memory

			LSI TNNKU873 BJR12034 - microcontroller

			MAXIC 25U40325 M1-126


	copy+paste dump for later:		copy+paste dump for later:

D2wn: more fixes

2024-11-19T10:28:29Z

more fixes

← Older revision		Revision as of 10:28, 19 November 2024
Line 3:		Line 3:
	These disks have three primary issues:		These disks have three primary issues:

	# They are formatted with 520-byte sectors. This prevents them from working with Dell PERC and HP Smart Array controllers, even when the controllers are set to "HBA-mode". They are only usable with a ''true'' HBA (or a ~~Dell/HP~~ controller flashed to the 3rd-party "IT mode" firmware).		# They are formatted with 520-byte sectors. This prevents them from working with Dell PERC and HP Smart Array controllers, even when the controllers are set to "HBA-mode". They are only usable with a ''true'' HBA (or a controller flashed to the 3rd-party "IT mode" firmware).
	# They have Hitachi Vantara firmware [https://www.reddit.com/r/homelabsales/comments/qazb88/pccan_huc101212css600_12tb_10k_sas_6g_25_drives/hh6frb5/] that locks them to Hitachi VSP systems and doesn't let them work with normal servers.		# They have Hitachi Vantara firmware [https://www.reddit.com/r/homelabsales/comments/qazb88/pccan_huc101212css600_12tb_10k_sas_6g_25_drives/hh6frb5/] that locks them to Hitachi VSP systems and doesn't let them work with normal servers.
	# They have a lock preventing them from being reflashed with the Dell / HP firmware that would allow them to be used in generic systems.		# They have a lock preventing them from being reflashed with the Dell [https://www.dell.com/support/home/en-au/drivers/driversdetails?driverId=PJ9CD] firmware that would allow them to be used in generic systems.


Line 32:		Line 32:

	sg_write_buffer -v -I HGST-Ultrastar-C10K1200-HUC101212CSS600-SAS-DELL-U440.LOD /dev/sg3		sg_write_buffer -v -I HGST-Ultrastar-C10K1200-HUC101212CSS600-SAS-DELL-U440.LOD /dev/sg3

			https://www.reddit.com/r/DataHoarder/comments/7raoz8/firmware_for_hgst_10k_sas_huc109090css600/

D2wn: minor touch-up

2024-11-19T10:18:13Z

minor touch-up

← Older revision		Revision as of 10:18, 19 November 2024
Line 3:		Line 3:
	These disks have three primary issues:		These disks have three primary issues:

	# They are formatted with 520-byte sectors. This prevents them from working with Dell PERC and HP Smart Array controllers, even when in "HBA-mode". They are only usable with a ''true'' HBA (or a Dell/HP controller flashed to the 3rd-party "IT mode" firmware).		# They are formatted with 520-byte sectors. This prevents them from working with Dell PERC and HP Smart Array controllers, even when the controllers are set to "HBA-mode". They are only usable with a ''true'' HBA (or a Dell/HP controller flashed to the 3rd-party "IT mode" firmware).
	# They have Hitachi Vantara firmware [https://www.reddit.com/r/homelabsales/comments/qazb88/pccan_huc101212css600_12tb_10k_sas_6g_25_drives/hh6frb5/] that locks them to Hitachi VSP systems and doesn't let them work with normal servers.		# They have Hitachi Vantara firmware [https://www.reddit.com/r/homelabsales/comments/qazb88/pccan_huc101212css600_12tb_10k_sas_6g_25_drives/hh6frb5/] that locks them to Hitachi VSP systems and doesn't let them work with normal servers.
	# They have a lock preventing them from being reflashed with the Dell / HP firmware that would allow them to be used in generic systems.		# They have a lock preventing them from being reflashed with the Dell / HP firmware that would allow them to be used in generic systems.

D2wn: Created page with "On XXXX we purchased ##x 1.2TB HGST HUC101212CSS600 drives off ebay for use in our servers. These disks have three primary issues: # They are formatted with 520-byte sectors. This prevents them from working with Dell PERC and HP Smart Array controllers, even when in "HBA-mode". They are only usable with a ''true'' HBA (or a Dell/HP controller flashed to the 3rd-party "IT mode" firmware). # They have Hitachi Vantara firmware [https://www.reddit.com/r/homelabsales/commen..."

2024-11-19T10:14:45Z

Created page with "On XXXX we purchased ##x 1.2TB HGST HUC101212CSS600 drives off ebay for use in our servers. These disks have three primary issues: # They are formatted with 520-byte sectors. This prevents them from working with Dell PERC and HP Smart Array controllers, even when in "HBA-mode". They are only usable with a ''true'' HBA (or a Dell/HP controller flashed to the 3rd-party "IT mode" firmware). # They have Hitachi Vantara firmware [https://www.reddit.com/r/homelabsales/commen..."

New page

On XXXX we purchased ##x 1.2TB HGST HUC101212CSS600 drives off ebay for use in our servers.

These disks have three primary issues:

# They are formatted with 520-byte sectors. This prevents them from working with Dell PERC and HP Smart Array controllers, even when in "HBA-mode". They are only usable with a ''true'' HBA (or a Dell/HP controller flashed to the 3rd-party "IT mode" firmware).
# They have Hitachi Vantara firmware [https://www.reddit.com/r/homelabsales/comments/qazb88/pccan_huc101212css600_12tb_10k_sas_6g_25_drives/hh6frb5/] that locks them to Hitachi VSP systems and doesn't let them work with normal servers.
# They have a lock preventing them from being reflashed with the Dell / HP firmware that would allow them to be used in generic systems.

== Problem 1: (520-byte sectors) ==
This is a very common problem with decommissioned enterprise SAS drives and also typically very easy to fix.

All you need is a "true" HBA in the server that is reformatting the disks. Dell PERC cards and HP Smart Array controllers cannot communicate with 520-byte sector disks whatsoever, even when put into "HBA mode".

The cards inside the [[Servers|Kurisu and Okabe]] servers have been flashed to "IT mode", as their PERC cards were old enough to be flashed using [https://fohdeesha.com/docs/perc.html a script]).

The sector size of a disk can be reformatted to 512 bytes using <code>sg3utils</code> with the command below:

<code>sg_format -v --format --size=512 --six /dev/sgX</code>

Of note, <code>--six</code> and <code>--ten</code> depend on what brand of disk you are using.

I don't know which option is correct for our specific drives, may be worth scouring the internet for a report from someone with our drives with the Dell firmware who was able to reformat it.

It is likely we won't know for sure which option to use until we unlock the firmware ourselves.

copy+paste dump for later:

sg_write_buffer -v -I HGST-Ultrastar-C10K1200-HUC101212CSS600-SAS-DELL-U440.LOD /dev/sg3