Understanding Large SATA Drive Compatibility

This post is intended to offer detailed technical information for troubleshooting issues affecting new “Advanced Format” 512e SATA disk drives.

Plugable’s full-size 3.5″ hard drive docks, the USB3-SATA-UASP1, and USB3-SATA-U3, both feature support for these new 512e Advanced Format drives. Our Plugable Storage System “PSS” products also support 512e large volumes (as soon as 2.5″ drive capacities increase and these drives become available in smaller sizes than 3.5″).

Some docks have a non-standard sector emulation feature that enables using capacities above 2TB on Windows XP 32 bit. But this requires that drives initialized and formatted in a special way, and NOT be used with other SATA controllers in desktop PC’s or other drive docking stations, unless those units also have a matching firmware version and support for this feature.

Plugable USB SATA docks do not support sector emulation for XP. Rather, we’ve chosen to support 3TB+ Advanced Format drives in the standard way without any emulation.

Background

The storage model used by disk makers for the last 10+ years is changing, and this post is an effort to explain how these changes work in detail. In case of issues moving drives between different systems, or when encountering issues using USB attached drives to host enterprise application data, the details here can help understand what factors are at play.

Data is stored fundamentally in bits (bits with a little b). Busses like USB often measure throughput in bits, like USB 2.0’s 480 Megabits per second (480 Mbps) or SATA III’s Gigabits per second (6 Gbp/s). Eight of these individual bits of data make up one of the capital-B “Bytes” that we usually measure data in, be it kilobytes (KB) in a document, MegaBytes (MB) in an MP3, or GigaBytes (GB) of data on a storage volume like a hard drive or SSD. Generally speaking, transfer rates industry-wide are measured in bits so transfer rates appear higher than if they were measured the same way the data is actually stored, in Bytes.

On modern solid state drives and traditional hard or even floppy disks, these bytes are grouped into sectors for actual read/write operations. Over the last decades, almost all storage drives used 512 Bytes per sector to store data since addressing individual bits and Bytes would be impractical.

Having all drives at 512 bytes per sector was nice, as there were few compatibility issues to think about when moving drives between systems or SATA docks, or when cloning them.

Evolution

512 byte sectors present problems with larger capacity drives (3TB+)- to make way for larger drives, sector sizes had to grow. This presents challenges throughout the “software stack” from the SATA hardware controllers and their firmware, their drivers, the operating system, and even to how applications may sometimes assume certain sector sizes or where sectors will start. Resulting issues are numerous and will take time for the industry to fully work out.

Over the past few years, Advanced Format Drives (512e, AF) drives reporting 512 Bytes/”logical” sector while actually using physical sectors of 4096 Bytes/”physical” sector have gained in popularity due to their higher data density potential, and resulting larger capacities. For more on how these drives work and why the industry switched, read this great ZD net post. Soon, we’ll be seeing drives that use 4096B logical and physical sectors.

Drive type Support/functionality
4K native (4K logical sector size) Supported on the following operating systems:

  • Windows 8
  • Windows Server 2012
Advanced Format or 512E (4K physical and 512-byte logical sector size) Supported on the following operating systems:
  • Windows Vista
  • Windows 7
  • Windows Server 2008*
  • Windows Server 2008 R2*
  • Windows Server 2012
  • Windows 8

*Except for Hyper-V. See the “Application support requirements for large-sector drives” section.

Specific requirements are listed in the following section. Run only applications and hardware that support these drives.

512-byte native (512-byte physical and logical sector size) Supported on all platforms.

*from Microsoft support policy for 4K sector hard drives in Windows

As you can see from the table above, determining whether you have a 512n (drive with reported and physical sectors of 512 bytes), 512e (the 512 emulation 4096 “Advanced Format” drives with physical 4096B clusters), or 4Kn (drive with both reported and physical sectors of 4096B) is crucial to determine which Windows operating systems will be able to recognize the drive.

On OS X, large volume support is somewhat better, with effectively no limits on volume size in modern OS X versions:

Maximum number of volumes (all Mac OS X versions)

no limit

Maximum number of files (or files and folders) in a folder (all Mac OS X versions)

up to 2.1 billion (2)

Maximum volume size and file size (Mac OS X v10.0 – 10.1.5)

2 TB (1)

Maximum volume size and file size (Mac OS X v10.2 – 10.2.8)

8 TB (1)

Maximum volume size and file size (Mac OS X v10.3 – 10.3.9)

16 TB (1)

Maximum volume size and file size (Mac OS X v10.4 or later)

close to 8 EB (1,3)

*From Mac OS X: Mac OS Extended format (HFS Plus) volume and file limits

How to check your drive’s sector size

To determine if you have an 512e Advanced Format Drive, open an administrative command prompt and run the following command on the NTFS formatted volume:

fsutil fsinfo ntfsinfo [your drive]

[your drive] will be the letter of the disk you’re testing, so c: or d: etc.

The output will look like this, with the bold portion below being the relevant information. Bytes per Sector of 512 and bytes per Physical Sector of 4096 indicate a 512e drive:

C:\Windows\system32>fsutil fsinfo ntfsinfo h:
NTFS Volume Serial Number : 0x8a0c5e820c5e68e9
NTFS Version : 3.1
LFS Version : 1.1
Number Sectors : 0x00000001d1bcafff
Total Clusters : 0x000000003a3795ff
Free Clusters : 0x000000003a3671d7
Total Reserved : 0x0000000000000000
Bytes Per Sector : 512
Bytes Per Physical Sector : 4096

Bytes Per Cluster : 4096
Bytes Per FileRecord Segment : 1024
Clusters Per FileRecord Segment : 0
Mft Valid Data Length : 0x0000000000040000
Mft Start Lcn : 0x00000000000c0000
Mft2 Start Lcn : 0x0000000000000002
Mft Zone Start : 0x00000000000c0000
Mft Zone End : 0x00000000000cc820
Resource Manager Identifier : DB59D441-7AD6-11E2-BEE8-00027232D73D’

If your drive reports:
Bytes Per Sector : 512
Bytes Per Physical Sector : 512

This one of the 512n volumes commonly used over the last decade. Support for these drives is universal.

If your drive reports:
Bytes Per Sector : 512
Bytes Per Physical Sector : 4096

The volume is operating in 512e mode, and should be recognized by most Windows Vista (SP1 and later) systems, as well as by newer Windows 7 and 8 PC’s. Windows 7 and Server 2008 users who do not have Service Pack 1 installed need a system update for 512e compatibility.

Macs should also be able to see the volume with no issue, however do not have native NTFS write capabilities, so they may only be able to read the drive.

If your drive reports:
Bytes Per Sector : 4096
Bytes Per Physical Sector : 4096

Then the drive has been formatted with non-standard sector emulation, in order to support 3TB+ drives on Windows XP.

A special release of firmware for the Plugable USB3-SATA-U3 (firmware v12092681f600) is available by special request that uses this non-standard emulation strategy. However, it is not the default and is generally not recommended as it limits moving the drive to other docks with the same emulation strategy.

How to clean and re-initialize a volume

Moving volumes initialized with 4096B sector re-emulation to other SATA controllers (other drive docks or PC SATA connections) may result in drives that are visible in disk manager but with no partitions, and no option to re-format the drive since the “new” (non U3) SATA controller doesn’t support the 4096B sector re-emulation. In this case, wiping the drive is necessary to use it on the new controller.

Wiping a misaligned drive’s partition table will allow the volume to be re-initialized as though it were new, and a new partition table may be written when the disk is re-formatted.

Performing the following steps will destroy all data on the disk selected, so these steps must be performed with extreme caution.

To check for all attached volumes, open an administrative command prompt and enter diskpart, then enter the list disk command as shown below, and select the disk of your choice to clean. After selecting whichever disk number you want to wipe, the clean command will destroy the partition table and allow you to re-initialize the volume.

C:\Windows\system32>diskpart

Microsoft DiskPart version 6.2.9200

Copyright (C) 1999-2012 Microsoft Corporation.
On computer: GRANDCENTRAL

DISKPART> list disk

Disk ### Status Size Free Dyn Gpt
——– ————- ——- ——- — —
Disk 0 Online 111 GB 1024 KB
Disk 1 Online 1863 GB 0 B
Disk 2 No Media 0 B 0 B
Disk 3 No Media 0 B 0 B
Disk 4 No Media 0 B 0 B
Disk 5 No Media 0 B 0 B
Disk 6 Online 1862 GB 0 B
Disk 7 No Media 0 B 0 B
Disk 8 Online 465 GB 0 B

DISKPART> select disk 8

Disk 8 is now the selected disk.

DISKPART> clean

DiskPart succeeded in cleaning the disk.

DISKPART>

From here the disk is blank and can be re-initialized and formatted with whatever partitions are desired. Most consumers will not need to be aware of the other issues that can arise with Advanced Format volumes in enterprise applications.

Partition Alignment

If the performance of your 512e large volume has degraded when being moved from system to system, partition alignment issues may be the cause. Issues with non-bootable cloned volumes may also be due to improperly aligned partitions. Western Digital and Hitachi offer drive alignment utilities for affected operating systems. Seagate drives should not require alignment. Customers with other brands of drives will need to check with the manufacturer of their drive to determine if an alignment tool is available and/or necessary. Partition alignment issues often affect older Windows Vista and Windows XP systems where drives are frequently accessed on other operating systems.

Before initializing a new drive, using a large volume with Windows XP, or especially before cloning an Advanced Format Western Digital drive, please check the chart below to determine if aligning your drive is necessary for maximum performance and compatibility with your software environment.

Using your WD Advanced Format Hard Drive with a Windows Operating System may require you to run the WD Align Windows software utility after you install your operating system or partition and format the drive as a secondary drive. The WD Align software aligns existing partitions on the Advanced Format drive to ensure it provides full performance for certain configurations. (Please see table for configuration details)

Running Hitachi’s “HGST Align” tool may be necessary for Windows XP or Windows Vista users. Windows 7 and Windows 8 users should not need to use this utility.

Windows support in the enterprise

Most end-users won’t need to worry about drive sector sizes, partition alignment, or any of the details above. Due to the specific issues documented by Microsoft with some of their enterprise applications, thorough planning for drive purchasing and migration is crucial in enterprise environments leveraging the following technologies:

*from “Microsoft support policy for 4K sector hard drives in Windows.

For more details on 512e and 4Kn Advanced Format support across Windows versions, application specific issues, other known issues, and unsupported scenarios, read Microsoft support policy for 4K sector hard drives in Windows.

Further details on the evolution from 512n to 512e and 4Kn drives are available in this presentation from IDEMA (International Disk Drive Equipment and Materials Association).

Deeper technical details for driver developers are available in Microsoft’s “Advanced format (4K) disk compatibility update (Windows)” article.

[usb3-sata-uasp1[usb3-sata-u3

11 comments on “Understanding Large SATA Drive Compatibility”

  1. Pingback: USB3-SATA-U3 (ASMedia 1051e) Firmware Update | Plugable

  2. Rüdiger Reply

    Hi Jeff,

    very interesting article! Especially for one searching to get his newly purchased 4TB drives attached onto a Win XP system :-)

    Thus I’m going to flah my two slipper like plugable docks with the 121026910000 firmware, initilize the drive as mbr (or GPT?) and hope I can create a 4TB volume on it…

    My concerns – resp. what is not totally clear to me after reading the article – are about compatibility. Is it right, that:

    – I can attach this special flashed dock with its drive to a WIN XP PC as well as Win 7, Win 8, etc. and I can use it to store and retrieve data from that constellation? Or e.g. doing the ini

    – I cannot use the drive from this special dock in other dock? If so, what is the reason? I think the interface from the drive to the ASM10513 is still 512Byte sectors…

    – I cannot use this special dock with other drives not initialized and volume not created by that special firmware? Again why; it’s 512Bytes interface though…

    Any other restrictions?

    What is ‘these issues’ in the 2nd paragraph of “How to clean and re-initialize a volume”?

    Perhaps a little info what the mentioned ‘work around does’ would help to understand the issues…

    Thanks,

    Rüdiger

    P.S. Think it should be 4096Bytes, not 4096kB in your article… ;-)

  3. Jeff Everett Jeff Everett Reply

    Hi Rudiger-

    Thanks for your comment. I’ll address these one by one.

    Q- I can attach this special flashed dock with its drive to a WIN XP PC as well as Win 7, Win 8, etc. and I can use it to store and retrieve data from that constellation? Or e.g. doing the ini

    A- As long as an advanced format disk initialized in the “special” firmware that the U3 is delivered with stays in a dock like the U3, it will be recognized on most any pc. The issues come when you try and use the drive in another SATA controller or dock: then you’ll have to initialize the disk- and lose all the data on it. To be clear, re-initialization is needed when changing SATA controller/dock to another, NOT when moving a drive initialized in the U3 from PC to PC.

    Q- I cannot use the drive from this special dock in other dock? If so, what is the reason? I think the interface from the drive to the ASM10513 is still 512Byte sectors…

    A- Firmware 12092681f600 re-emulates sectors of 4096B to the OS, which firmware 13022081f602, and most other SATA controllers/docks do NOT do. Those other controllers and docks would not be able to recognize drives initialized in the U3 until we re-initialize.

    Q- I cannot use this special dock with other drives not initialized and volume not created by that special firmware? Again why; it’s 512Bytes interface though…

    A- For any drives that aren’t advanced format drives (most drives 1tb and below, some drives from 1-2tb), these should work fine, even if they were initialized on another SATA controller or dock. For advanced format drives initialized in another system, since this dock expects them to be partitioned with 4096b sectors, but the advanced format drives present themselves with 512B sectors, even though they actually have the 4096B sectors: this is why only the “Advanced Format” drives using 512e (512 byte emulation) have to be re-initialized on firmware 12092681f600.

    Hope this helps- thanks for your close attention, I’ve removed the references to 4096KB sectors in favor of the correct 4096B sector.

    Any other questions, just let me know!

    Best-

    Jeff

  4. Rüdiger Reply

    …once again me:

    Cannot find those “32 bit ONLY compatible firmware versions UASP1 121026910000 and U3 12092681f600″ on your website…

    Can you post the links?

    Thanks!

  5. Jeff Everett Jeff Everett Reply

    Hi again Rüdiger

    U3 units shipping now already have 12092681f600.

    We have not offered 121026910000 publicly, nor do we have plans to do so.

    I can send this privately if you need it, however just getting a U3 would be easier. If you really need the UASP1 firmware that supports 2.19TB+ drives on 32 bit PCs, just shoot an email to support@plugable.com with your Amazon order ID.

    To summarize, for customers wishing to use a drive over 2.19TB on a 32 bit Windows PC, the U3 is recommended. Downgrading UASP1 firmware is possible, but is not an option we are offering publicly, however instructions are available for advanced users.

    For all other users, 64 bit PC users and 32 bit PC users who are OK with the 32 bit limitation of addressing only 2.19TB, the UASP1, or updating your U3, is recommended.

    Best-

    Jeff

  6. Rüdiger Reply

    Hi Jeff,

    you’re faster than Super Speed ;-)

    I just checked firmware version of one of my flat docks. It is: 12020281ab02

    Perhaps this is an elder version sold here…

    I purchased them some days ago at amazon Germany for twice the price as in the US. Unfortunately, amazon.com doesn’t ship your great products to Germany (I was in contact with Jerome on this). I was willing to buy two pieces of each (U3 and UASP), but U3 is not listed at amazon.de.

    So for me, it seems to be the only possibility as of now to change the firmware of my UASP’s…

    I’ll send my order number via email.

    Thanks again for your help!

    Rüdiger

  7. Roland Bürkel Reply

    Hi,

    I have a question to the “32 bit Windows ONLY firmware 12092681f600″:

    Is every connected HDD running in the emulated 4k mode with this firmware?

    I am searching for an USB-SATA-Controller which runs every connected HDD (e.g.500GB) in logical and physical 4k mode.

    Thanks.
    Roland

    • Jeff Everett Jeff Everett Reply

      Hi Roland-

      Thanks for your question.

      For smaller drives that come with 512n (n=native) sectors there is no emulation, only for AF/512e drives.

      I’ve never seen anything that would “emulate up” to let you simulate 4096b sectors on a 512n hardware.

      Sorry I can’t offer a more helpful response, hope the info helps.

      Best-

      Jeff

      • Roland Bürkel Reply

        Hi Jeff,

        I need a USB-SATA-Controller for advanced format HDDs (4k physical sector size) with 250G/500G/750G/1TB.

        Would this special firmware report 4k logical and physical sector size to the operating system if one of this HDDs is connected? This would be great!

        Thanks.
        Roland

  8. Roland Bürkel Reply

    Hi Jeff,

    can you please answer my question above.
    This is important for me before buying this device in Germany (70 EUR).

    Thanks.
    Roland

    • Jeff Everett Jeff Everett Reply

      Hi Roland–

      Sorry my previous reply wasn’t more clear.

      In a word, no, we cannot help with your request for a dock that would report 4096b sectors if they don’t exist on the source drive.

      If this feature is important to you, our product will *not* meet your needs. This is why I mentioned “I’ve never seen anything that would “emulate up” to let you simulate 4096b sectors on a 512n hardware.” If a solution does exist, which I do not believe it does, we’re not able to recommend where you can find it.

      For smaller drives that come with 512n (n=native) sectors there is no emulation, only for AF/512e drives. To clarify, large sector emulation only works for drives with out 512e/AF sectors- *NOT* for older 512n drives.

      Hope the details help.

      Best wishes-

      Jeff

Leave a Reply