Lets first start by making a difference between entry level, midrange and high-end SAN's.
Entry level:
This is basically a bunch of disks with a network connection. The Oracle/Sun/StorageTek 2540 is an example for this category. This storage is aimed at lowcost shared storage.
Midrange:
This kind of storage generally features replication and more disks than entry level. HP EVA is what comes to mind for this category. This storage is aimed at a good price/performance.
High-End:
This is mainframe style storage which is made usable for open systems. Hitachi Data Systems VSP is a good example of this category. This category is for extreme performance and reliability.
Please make sure to not (mis-)judge High-End SAN by the experiences you had with entry-level storage.
Why should we use SAN/NAS storage?
- SAN's can offer more reliable storage than local storage
- SAN's offer all kinds of extra services like replication for disaster recovery.
- SAN's be used to make machines be easy replaceable. (Diskless, Boot from SAN)
- SAN's offer shared storage which can be used to make cluster setups.
- SAN's offer server-less backups and snapshots.
- SAN's offer storage pooling
- Most SAN's offer online upgrade of firmware
- SAN's offer online adding and replacement of disks
There are many issues which can arise with shared storage. I've encountered many issues with all of the above categories.
- Multipath setups which fail due to human error or due to firmware issues.
- Storage becomes slow because 1 machine is hammering the SAN. This bottleneck could arise on the network, the storage controller and on the disks.
- FC controllers are expensive and so are iSCSI controllers.
- Using ethernet might require a seperate network or a upgrade of your current network.
- Using SAN/NAS will require a storage administrator in many cases.
- Use MySQL (semisync) replication for disaster recovery instead of replication on the SAN level. This offers more efficient replication as statement based (or mixed) replication can be used.
- Use DRBD for repliation of disks for cluster setups. Or use a NDB Cluster.
- MySQL Enterprise Backup and XtraBackup can provide you with a low-impact backup wich can be used instead of a server-less backup solution. It could even be a no-impact backup when done on a slave. Using server-free backups with MySQL is not always possible.
- Using LVM will allow for flexible storage assignment and snapshots, but most filesystems only support growing and do not offer shrinking features. Think of it as the ever-growing ibdata file which happens if you're not using innodb_file_per_table
SAN or NAS
A typical NAS storage array like a NetApp Filer can be accessed using FC nowadays. And most SAN storage vendors do offer NAS heads. So this is not as big of a difference as it once was. SAN is block level storage which uses FC or iSCSI and NAS is file level storage using NFS or CIFS.
MySQL can be used on FC, NFS, iSCSI or any other type of storage. But when sync_binlog=1 and innodb_flush_trx_at_commit=1 is used MySQL will do a lot of fsyncs. This will slow down the storage a lot. A local battery backed RAID array will be much quicker as the network roundtrip will be eliminated. (Beware of raid auto learning!)
Some virtualization software like VMWare and VirtualBox do offer virtualized disks which can be located on a SAN. Make your virtualization layer is configured correctly to do a proper fsync.
Conclusion:
There is enough software in the MySQL ecosystem which can bring you SAN-like features with cheap/fast local storage. However if MySQL is not the only software which should be replicated it might be good choice to use a more generic way of managing your storage, this could include a SAN. Using a SAN might allow you to do things like upgrade the firmware of your disks and controllers without doing a failover on your master.
Do you use SAN/NAS for MySQL? Why? Why not? Do you use SAN features like replication and/or snapshots?
I absolutely agree. While I have experienced some terribly awful SANs I have also used some SANs that blow away local/DAS architectures in performance, data services, and DR/HA. In particular I'm enjoying the Sun 7420 SAN - and exactly as you mentioned above it is leagues ahead of the midrange EqualLogic PS6000 systems that I have been using at the same time. MySQL works on SAN - you just have to have someone running the SAN that knows what they are doing. Over-subscription is possible on any system, SAN or DAS or LocalDisk. Blindly saying MySQL+SAN is trouble shows that one doesn't use the correct SAN.
ReplyDeleteGood followup.
ReplyDeleteThe key part of this is, "have to have someone running the SAN that knows what they are doing." I realize this hardware is capable of being awesome but so often they are mismanaged that I tend to shy away from them all together.