November 22, 2011

OpenSolaris snv_130, si3124, and Port Multipliers

In general, I rarely post stuff these days. Mostly because I'm just too busy to spend more time in front of the computer going back on what happens in my life. Heck, it seems I rarely even have time to finish projects that I start.

But every once in a while, something gets enough progress that makes it likely to be interesting to the rest of the world. This would be one of those.

In April of last year, I built myself a small file server (EON Solaris) to use for reliable storage and backups using whatever parts I have sitting around plus a little eBay-ing. It has since started running out of space and also running out of SATA ports.

Naturally, I bought a port multiplier. And I found out that OpenSolaris snv_130 and Solaris 11 Express both don't support port multipliers on the si3124 driver because of changes in the SATA Framework around snv_125-ish.

Since the changes did make it into the AHCI driver, I decided to try to patch up the si3124 driver to make it work using the AHCI driver as reference.

Long story short, I expected it to be more involved, but stuff started magically working after I finished modifying si_tran_probe_port. To be bluntly honest, I have no idea why it's working.

At this point, I'm posting it up for those who might find it interesting, and for those who have more knowledge about this stuff. (...while I continue to exercise it on a test rig.)

Once again... I have no idea how safe this is.
It was developed on snv_130 on a 2nd machine since it matches my file server's OS revision. I tried to mount a test ZFS mirror pool from those two drives but that obviously didn't work since that test pool was created in snv_151a. So I blew that away and made a new pool out of it.
I also don't understand how fixing the port probe magically makes the rest of the si3124 driver's old port multiplier support work on its own.
But since getting it working, I've:
1) created a pool on 2 port multiplier-attached spanned drives
2) pushed 450GB to it by scp'ing data from another machine
3) started a large directory diff to a usb-attached drive...
4) ... while running a zpool scrub is running.

It seems be fine so far with ~108MB/sec.

Things not tested:
* It's on a sil3132 on the test rig. So the sil3124 I originally intended to be used with is still untested.
* I haven't tested what happens when there's drives on the port multiplier AND attached to the card. You can see below that port 1 on the card is empty.
* There was one hang as I tried to duplicate data from an attached usb drive, but that might be because the usb drive was clicking and dying and hanging the usb bus. Yes, I am testing with some pretty crappy hardware :P
* I doubt hotswap works. At least, I didn't try it. There's enough magic for me already.

test130:~ hchung$ cfgadm
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0                        sata-pmult   connected    configured   ok
sata0/0.0::dsk/c4t512d0        disk         connected    configured   ok
sata0/0.1::dsk/c4t544d0        disk         connected    configured   ok
sata0/0.2                      pmult-port   empty        unconfigured ok
sata0/0.3                      pmult-port   empty        unconfigured ok
sata0/0.4                      pmult-port   empty        unconfigured ok
sata0/1                        sata-port    empty        unconfigured ok
usb0/1                         unknown      empty        unconfigured ok
usb0/2                         usb-hub      connected    configured   ok
usb0/2.1                       usb-device   connected    configured   ok
usb0/2.2                       usb-mouse    connected    configured   ok
usb0/2.3                       unknown      empty        unconfigured ok
usb1/1                         unknown      empty        unconfigured ok
usb1/2                         unknown      empty        unconfigured ok
usb2/1                         unknown      empty        unconfigured ok
usb2/2                         unknown      empty        unconfigured ok
usb3/1                         unknown      empty        unconfigured ok
usb3/2                         unknown      empty        unconfigured ok
usb4/1                         unknown      empty        unconfigured ok
usb4/2                         unknown      empty        unconfigured ok
usb4/3                         unknown      empty        unconfigured ok
usb4/4                         usb-storage  connected    configured   ok
usb4/5                         unknown      empty        unconfigured ok
usb4/6                         unknown      empty        unconfigured ok
usb4/7                         unknown      empty        unconfigured ok
usb4/8                         unknown      empty        unconfigured ok

Source Code + Binary
If you happen to try this out, MAKE SURE YOU BACK UP YOUR DATA FIRST. Consider this experimental, of questionable reliability, and please do let me know your results should you be brave enough to try this on your test machine.

Posted by hachu at 06:22 PM | Comments (3)