Storage ideals

Storage software

NOR flash for firmware. EEPROM for parameters and history.

Block-based allocation is great for the boot partition, although extent-based allocation is great too. Extent-based allocation is great for structured data. Dynamic allocation for unstructured and semi-structured data.

Journaling for metadata, copy-on-write for the actual data. Log-free metadata would be good.

Object storage for unstructured data such as pictures, videos, and music.

An object-based filesystem like Lustre or PanFS would be great for big data analytics and artificial intelligence.

Form factors

NVMe flash arrays should be 1U or 2U, iSCSI disk enclosures should be 2U or 3U.

Parallel filesystem storage should be 1U or 2U, no larger than 3U. If any nodes should be 2U or 3U, it should be the directors or metadata servers. Blade systems might be better for parallel filesystems like GPFS or OrangeFS than for object storage like Lustre.

Protocol processing

Storage systems should have a CPU for control plane functions such as pooling, and a semi-custom application-specific integrated circuit for the actual transfer of data.

Network cards should be semi-custom application-specific integrated circuits. Standard cells would be good for traditional and industry-standard technology, Intellectual Property (IP) cores would be good for the TCP Offload Engine, hard and soft macros would be good for the Media Access Control and the iSCSI initiator, and custom blocks would be great for NVMe-over-Fabrics and other cutting-edge components.

Networking

RDMA-based fabric for NVMe-over-Fabrics and GPUDirect. TCP/IP fabric for iSCSI, Filesystem in USErspace (FUSE), S3, NFS, SMB/CIFS, HDFS, etc.

PCI Express and Compute Express Link for pooling and composing memory and accelerators and for intra-PoD (Point-of-Delivery) communication, a conventional TCP/IP or RDMA-based fabric for inter-PoD communication, sharing of certain resources such as flash memory between PoDs, and pooling storage.