SCST：几种常用iscsi target对比

This features comparison is intended to be a complete and fair feature-by-feature comparison between the listed targets without any bias to SCST. If you see anything wrong somewhere or anything missed, you are welcome to report it in scst-devel mailing list and it will be corrected.

As on June 2011.

General
Upstream kernel	-	Interface with in-kernel target drivers (obsolete)	-
Generic Target Engine	+	+	iSCSI only	+
Architecture	Kernel only	User space only	-	Kernel only
Stability	+	+	+	Probably (no released packages)
Performance	*****	***	****	****-
Zero-copy passing data between target and backend drivers	+	+	+	+
Support for transports without expecting transfer values (Wide (parallel) SCSI, SAS)	+	-	-	-
Interface with user space	SysFS (or obsolete ProcFS)	Custom	-	ConfigFS/IOCTL/ProcFS
Features
Target drivers in kernel space	+	+	-	+
Target drivers in user space	Via scst_local (e.g. using STGT pass-through)	+	-	Via tcm_loop (e.g. using STGT pass-through)
Backstorage handlers in kernel space	+	-	-	+
Backstorage handlers in user space	+	+	-	Via STGT (under development)
Advanced devices access control	+	-	-	+
Automatic sessions reassignment (changes in the access control immediatelly seen by initiators)	+	-	-	-
Support for Asynchronous Event Notifications (AEN)	+	-	-	-
Notifications for devices added/removed or resized through AENs or Unit Attentions (initiators can instantly see any target reconfiguration in a PnP-like manner)	+	-	-	-
Bidirectional Commands	+	+	-	+
Extended CDB (size >16 bytes)	+	+	-	+
Descriptor sense support	+	+	-	-
RESERVE/RELEASE (Windows 2003 clustering)	+	+	+	+
Safe RESERVE/RELEASE implementation according to SCSI requirements	Safe	Safe	Safe from v1.4.18	Not safe
Safe implementation of Task Management commands	Safe	Not safe	Not safe	LUN RESET - safe. Other TM commands not implemented.
Support for SCSI task attributes, including ORDERED commands	+	+	-, data corruption possible	-, data corruption possible
Persistent (SCSI-3) Reservations (Windows 2008 clustering / RHEL5 I/O fencing)	+	+ (limited functionality implemented)	-	+
Durable, i.e. transactional, save of Persistent Through Power Loss Persistent Reservation data	Durable	Not supported	-	Not durable
ALUA	+/- (Implicit only)	-	-	+/-
Failover Clustering	+	+	+	+
Different threading models to choose the best performing	+	-	-	-
CPU affinity control	+	-	-	+
I/O context grouping between I/O threads (big performance win with CFQ)	+	-	+	-
Per-initiator I/O context grouping (big performance and fairness win if several initiators access the same virtual or backend device on the target)	+	-	-	-
Protection against commands with wrong transfer size or transfer direction (may lead to crash or hard lockup of the target)	+	-	-	-
Protection against crashing target by making it to allocate too much memory for buffers and go into OOM state	+	-	-	-
Caching of allocated buffers	+	-	-	-
Latencies measurement facility	+	-	-	-
Configuration tool with ability to automatically apply changes in the config file on fly without any restarts	scstadmin	-	-	rtsadmin?
SCSI MIBs	-	-	-	+-
Supported transports and hardware
iSCSI	+	+	+	+
QLogic (Fibre Channel)	+	-	-	Preliminary
Emulex (Fibre Channel and FCoE)	+	-	-	-
SRP	+	-	-	Preliminary
iSER	-	+	-	-
Marvell (SAS)	Beta	-	-	-
FCoE	Beta	Under development	-	Alpha
LSI (Parallel (Wide) SCSI and Fibre Channel)	Alpha	-	-	-
LSI (SAS)	Preliminary (not completed)	-	-	-
IBM pSeries Virtual SCSI	+	+	-	Preliminary
Local access to emulated backstorage devices	scst_local	-	-	tcm_loop
Supported backstorage
Kernel side FILEIO	+	-	+	+
Kernel side BLOCKIO	+	-	+	+
User space FILEIO	+	+	-	-
O_DIRECT FILEIO	fileio_tgt	+	-	-
Async FILEIO	-	+	-	-
Native RAMDISK	-	-	-	+
SCSI pass-through	+	Single initiator only, not enforced	-	Single initiator only, not enforced, limited functionality for tapes
Zero-copy data read/write to/from backstorage	BLOCKIO, user space FILEIO in O_DIRECT mode, pass-through	-	BLOCKIO	BLOCKIO, pass- through
Cache safe FILEIO	Safe	Safe only RDWR backend	Safe	Safe
Cache safe BLOCKIO	Safe	-	Not safe	Safe
4k sectors support in pass-through mode	+	-	-	?
4k, 2k, 1k and 512 byte sectors emulation in modes, other than pass-through	+	+	-	+
Virtual CD devices emulation from ISO files	+	+	-	-
Possibility to write to emulated from ISO files CD devices	-	+	-	-
Emulation of virtual tape and media changer devices (VTL)	-	Experimental	-	-
Thin provisioning support	+	-	-	+
iSCSI Target
Architecture	Split	User space only	Split	Kernel only
Interface with user space	SysFS (or obsolete ProcFS)/ IOCTL/Netlink	-	IOCTL/ProcFS/ Netlink	ConfigFS/IOCTL/ProcFS
Zero-copy data send/receive	Send only	In some cases, send only	Send only	Send only
	-	-	+	+
Max ErrorRecoveryLevel	0	0	0	2
Support for limiting number of initiators allowed to connect to a target	+	-	+	-
Per-portal targets visibility control	+	-	+	-
Per-initiators targets visibility control	+	+	+	-
Support for AHS	+	+	-	-
Support for iSCSI redirects	+	+	+	-
Bidirectional Commands	+	+	-	-
Extended CDB (size >16 bytes)	+	+	-	-
Support for AENs (initiators can instantly see any target reconfiguration in a PnP-like manner)	+	-	-	-
Support for iSNS	+	+	+	-
Safe implementation of Task Management commands	Safe	Not safe	Not safe	ABORT TASK - not safe, LUN RESET - safe, other TM commands not implemented.
Safe implementation of connections and sessions reinstatement	Safe	Not safe	Not safe	Not safe
Usage of hardware instructions for digest calculations, if available	+	-	-	+
Each connection multithreaded digest calculation	+	-	-	-
Safe restart	Safe	?	Not safe before v1.4.18. After - probably safe.	?
iSCSI MIBs	-	-	-	+-
Local access target
Bidirectional support	+	-	-	+
Support for AENs (initiators can instantly see any target reconfiguration in a PnP-like manner)	+	-	-	-

REMARKS:

1. STGT has all SCSI target processing and memory management in the user space. In the kernel it has only a small library to help in-kernel target drivers to interact with it. As a direct consequence, neither iSCSI, nor iSER as well as any other user space STGT target need any STGT kernel modules loaded to run.

2. All iSCSI management implemented in user space and actual data transfers in kernel space without user space involved.

3. The result "in average" is listed. One target can be better somewhere, another one somewhere else. Although manual tuning of target and system parameters tends the restore the difference listed in the comparison. You can find example measurements , and .

4. All SCST and its drivers' kernel patches supposed to be applied and SCST with the drivers built in the release or performance build. Without the kernel patches SCST performance will be at "****+" level, except for the case, when user space backstorage handler used with iSCSI-SCST iSCSI target driver, where performance will be at "***+" level.

5. In SCST data are always passed in zero-copy manner between target and backend drivers without need for any additional kernel patches, except in case, when local access (scst_local) used with user space backend.

6. Some zero-copy functionality isn't available from user space, sometimes fundamentally. For instance, zero-copy FILEIO with page cache or zero-copy send to a socket. Also STGT can't use splice() for in-kernel target drivers, because it has memory management in user space. To use splice() with socket-based user space target drivers STGT would need a deep redesign of internal interactions between target drivers, core and backend handlers. But in some cases STGT can use zero-copy sendfile().

7. "Local access to emulated backstorage devices" means that you can access emulated by a SCSI target devices locally on the target host. For instance, you can mount your ISO image from emulated by the target CDROM device locally on the target host.

8. "Advanced devices access control" means that different initiators can see different sets of devices from the same target. This feature is required for hardware targets, which don't have ability to create virtual targets.

9. "Cache safe" means that cache synchronization commands (SYNCHRONIZE_CACHE and FUA attribute) from initiators perform what they expected to perform, i.e. push all the requested blocks from all caches, including devices' caches, to non-volatile media.

10. SCSI requires that if an initiator clears reservation held by another initiator, the reservation holder must be notified about the reservation clearance. Otherwise, several initiators can at the same time change supposed to be protected by the reservation data, which can corrupt them. This is what was called on the VMware community forum by someone working for VMware. But, sure, it can affect not only VMware, but also any other cluster implementation, relying on this functionality.

11. After a task management command completed and before the corresponding response was sent to the initiator, who sent that task management command, all the affected SCSI commands must get into a state, where they can't affect following after the tasks management response commands from this initiator. This is the safe implementation. The unsafe implementation only marks all the affected SCSI commands as aborted and then immediately send task management response to the initiator. This implementation only guarantees that the initiator will never receive responses from those commands, but it doesn't guarantee that none of those commands will get executed by backstorage *AFTER* any SCSI command, which initiator will send after it received the task management response thinking that all the aborted commands actually fully aborted. This could lead to a data corruption.

12. Both IET and LIO report in INQUIRY command response support for full task management model. But they process ORDERED commands the same way as SIMPLE commands, i.e. allow free reorder of them before they get executed. That violates SCSI standard and can lead to a data corruption to any application relying on commands order provided by ORDERED attribute.

13. LIO exports the information needed for an RFC 4455 implementation, but requires additional RFC 4455 implementing module. At the moment, there is no open source implementation of such module.

14. SCSI pass-through mode allows to export your local SCSI-capable device. For instance with it you can share your parallel SCSI tape or SATA DVD-RW device to your iSCSI network.

15. STGT and LIO don't emulate all the necessary SCSI host functionality to allow to share SCSI devices in pass-through mode to several initiators, although LIO has some necessary processing, but not all. They can only pass SCSI commands from initiators to SCSI devices and responses back. This is safe only with a single initiator. This limitation isn't enforced anyhow and both STGT and LIO don't issue any warning about it, so an user will not be notified about this limitation and can quietly corrupt his/her data. You can find more technical information about it . Also LIO in pass-through mode doesn't do necessary sense processing for tape devices to correctly return residual information, so tapes can used with it with limited functionality.

16. You can find a proposal how to implement zero-copy FILEIO in SCST on the page.

17. Doesn't need any kernel patch, except in the case, when used with user space backend.

18. Connections and sessions reinstatement is, basically, a kind of Task Management command, because it implies commands aborting. So, similarly to the safe task management above, a safe implementation of connections and sessions reinstatement must not accept SCSI commands from new connection/session until all the SCSI commands in being reinstated connection/session get into a state, where they can't affect new commands.

19. "Safe restart" means that after the iSCSI target restart, all the connected initiators will seamlessly restore all existing before the restart connections. "Not safe" means that, most likely, the connected initiators will fail to restore existing connections with some errors. However, your iSCSI initiator also should be able to handle the safe restart. For instance, old (pre-CentOS/RHEL 5) open-iscsi has problems in this area. But the latest versions do it pretty well.

20. Generic implementation, i.e. not coupled to any particular cluster implementation, which means it is needed additional effort to used with each particular cluster setup.