Intel Optane SSD DC P4800X 750GB Hands-On Review

Intel’s rollout of Optane products, based on 3D XPoint memory, continues. Today is the release of the 750GB Optane SSD DC P4800X, a larger capacity version of the model launched earlier this year. Now Intel’s flagship enterprise SSD family has a more useful capacity while offering the same chart-topping performance as the 375GB model that launched earlier this spring.

We tested the 375GB model earlier this year through remote access to an Intel server lab. Last month, we reviewed the consumer counterpart to the P4800X, the Intel Optane SSD 900p. Today, we have the results of hands-on testing of the 750GB P4800X add-in card model.

The 750GB add-in card model was originally planned for a Q2 introduction, followed by the 750GB U.2 model and the 1.5TB models in the second half of 2017. Both 750GB models are now launching, and Intel says the 1.5TB models are “coming soon”. This is a fairly minor delay, but that is of little comfort to customers who have been eagerly awaiting drives like this since 3D XPoint memory was first announced in July 2015.

Intel Optane SSD DC P4800X Specifications
Capacity		375 GB	750 GB
Form Factor		PCIe HHHL or 2.5″ 15mm U.2
Interface		PCIe 3.0 x4 NVMe
Controller		Intel SLL3D
Memory		128Gb 20nm Intel 3D XPoint
Typical Latency (R/W)		<10µs
Random Read (4 kB) IOPS (QD16)		550,000
Random Read 99.999% Latency (QD1)		60µs
Random Read 99.999% Latency (QD16)		150µs
Random Write (4 kB) IOPS (QD16)		500,000
Random Write 99.999% Latency (QD1)		100µs
Random Write 99.999% Latency (QD16)		200µs
Mixed 70/30 (4kB) Random IOPS (QD16)		500,000
Sequential Read (64kB)		2400 MB/s
Sequential Write (64kB)		2000 MB/s
Active Power	Read	8 W	10 W
	Write	13 W	15 W
Idle Power		5 W	6 W
Endurance		30 DWPD
Warranty		5 years
MSRP		$ 1520

The performance specifications of the Optane SSDs stand out in several ways.

The latency for reads at low queue depths is far lower than NAND flash-based SSDs. This can provide a boost to almost any application that waits on storage. With no complicated wear leveling and garbage collection process going on under the hood, the Optane SSDs offer much better performance consistency than most flash-based SSDs, to the point that Intel specifies 99.999th percentile latency for the Optane SSD.

Perhaps most significant is that capacity doesn’t affect performance. For flash-based SSDs there is usually a very strong correlation between capacity and performance. The simpler, more direct architecture of Optane SSDs cuts out most of the intermediate buffering that flash SSDs need to perform well.

The sequential transfer speeds of the P4800X are not record-setting, and the read speed rating is actually 100MBps lower than that of the consumer Optane SSD 900p. However, nobody is going to buy an Optane SSD primarily for its sequential performance when flash-based SSDs can compete in that aspect at a fraction of the price. It is interesting to note that the sequential performance specifications are given for 64kB transfers, when 128kB is more common within the industry and 128kB transfers are supported by the Optane SSD.

Power consumption is the only area where the specifications differ between the 375GB and the 750GB model. The SSD DC P4800X does not implement the standard NVMe power state management or Autonomous Power State Transition features, but it does have a power governor setting controllable through vendor specific commands. This allows the P4800X to be limited to 12W instead of the default limit of 18W. Since imposing a lower power limit will reduce performance, this feature is probably not going to get much use with a product like the P4800X. There may be a few situations where it could be of use with the U.2 model that has less surface area for cooling, provided that it leads to more consistent performance than simply letting the drives thermally throttle.

The P4800X’s write endurance rating of 30 drive writes per day is far higher than most enterprise SSDs and beats any previous Intel SSD, but it isn’t actually high enough to put the Optane SSD in an entirely different class from the best high-endurance flash based SSDs. When the P4800X exhausts the rated write endurance, it will switch to a “write protect” mode where writes are throttled to 30MB/s. This is a gentler end of life strategy than the hard read-only mode that some of Intel’s previous drives have used. The mode switch happens when the “Percentage Used Estimate” SMART indicator reaches 105%.

Once the P4800X has reached its write endurance rating, it is specified for unpowered data retention of 3 months, which is standard for enterprise SSD. Intel doesn’t say anything directly about data retention when the drive is new, but the do caution that the drive will perform background data refreshing. When the drive is powered on, it will devote more time than usual to background data refreshing for a period of about three hours, to clean up any data degradation that may have occurred while the drive was off for an unknowable period of time.

Model				Dies per channel	Channels	Raw Capacity	Spare Area
Intel Optane SSD 900p 280GB				3	7	336 GB	56 GB (16.6 %)
Intel Optane SSD DC P4800X 375GB				4	7	448 GB	73 GB (16.3 %)
Intel Optane SSD 900p 480GB				5	7	560 GB	80 GB (14.3 %)
Intel Optane SSD DC P4800X 750GB				8	7	896 GB	146 GB (16.3 %)

The Optane SSD DC P4800X and the Optane SSD 900p both use the same 7-channel controller, which leads to some unusual drive capacities. The 900p comes with either 3 or 5 memory dies per channel while the P4800X has 4 or 8. All models reserve about 1/6th of the raw capacity for internal use.

Aside from the number of 3D XPoint memory packages and the number of dies they contain, the PCBs of the P4800X and the 900p differ only in the presence of a few diagnostic LEDs on the P4800X. Neither model has any large power loss protection capacitors because the Optane SSDs don’t use any volatile caches and write immediately to the 3D XPoint memory itself.

The heatsinks and PCIe brackets are slightly different between the consumer and enterprise versions, but both are predominately black aluminum. The P4800X omits the solid backplate and instead places the label with model and serial numbers on the side of the heatsink.

Along with the Optane SSD sample, Intel provided a new server based on their latest Xeon Scalable platform for our use as a testbed. The 2U server is equipped with two 18-core Xeon Gold 6154 processors, 192GB of DRAM, and an abundance of PCIe lanes.

Because this is a dual-socket NUMA system, care needs to be taken to avoid unnecessary data transfers between sockets. For this review, the tests were setup to largely emulate a single-socket configuration: All SSDs tested for this review were connected to PCIe ports provided by CPU #2. All of the benchmarks using the FIO tool were configured to only use cores on CPU #2 and only allocate memory connected to CPU #2, so inter-socket latency is not a factor. This setup would be quite limiting when doing full enterprise application testing, but for synthetic storage benchmarks one CPU is far more than necessary to stress any single SSD.

For this review, the test system was mostly configured to offer the highest and most predictable storage performance. HyperThreading was disabled, SpeedStep and processor C-states were off, and other motherboard settings were set for maximum performance. The one notable exception is fan speeds: since this test server was installed in a home office environment instead of a datacenter, the “acoustic” fan profile had to be used instead of “performance”.

Enterprise SSD Test System
System Model	Intel Server R2208WFTZS
CPU	2x Intel Xeon Gold 6154 (18C, 3.0GHz)
Motherboard	Intel S2600WFT
Chipset	Intel C624
Memory	192GB total, Micron DDR4-2666 16GB modules
Software	Linux kernel 4.13.11 FIO 3.1

The Linux kernel’s NVMe driver is constantly evolving, and several new features have been added this year that are relevant to a drive like the Optane SSD. Rather than use an enterprise-oriented Linux distribution with a long-term support cycle for an older kernel version, this review was conducted using the very fresh 4.13 kernel series.

Earlier this year, the FIO storage benchmarking tool hit a major milestone with version 3.0 that switches timing measurements from microsecond precision to nanosecond precision. This makes it much easier to analyze the performance of Optane devices, where latency can be down in the single-digit microsecond territory.

Intel SSD DC P3700 1.6TB

The Intel SSD DC P3700 was the flagship of their first generation of NVMe SSDs, and one of the first widely available NVMe SSDs. It was a great showcase for the advantages of NVMe, but is now outdated with its use of 20nm planar MLC NAND and capacities that top out at 2TB.

Intel launched a new P4x00 generation of flash-based enterprise NVMe SSDs this year using a new controller and 3D TLC NAND. However, due to the existence of the Optane SSD DC P4800X as the new flagship drive, the P3700 didn’t get a direct successor: the P4600 is currently Intel’s top flash-based enterprise SSD, and while it offers higher performance and capacities than the P3700 it does not match the rated write endurance of the P3700.

Intel SSD DC P3608 1.6TB

Intel’s first-generation NVMe controller only supports drive capacities up to 2TB. To get around that limitation and to offer higher performance in some respects, Intel created the SSD DC P3608. Roughly equivalent to two P3600s on one card behind a PLX PCIe switch, the P3608 appears to the system as two SSDs but can be used with software RAID to create a large, high-performance volume. Our P3608 sample has a total of 1.6TB of accessible storage (800GB per controller) and has the highest built-in overprovisioning ratios in the P3608 family. This gives it the best aggregate random write performance, which is rated to match a single P3700 while random reads and sequential transfers can exceed the P3700.

For this review, one controller on the P3608 was tested as a stand-in for the P3600, and a software RAID 0 array spanning both controllers was also tested.

Intel has not officially announced the successor P4608, but they have submitted it for testing and inclusion on the NVMe Integrators List, so it will probably launch within the next few months.

Micron 9100 MAX 2.4TB

Launched in mid 2016, the Micron 9100 series was part of their first generation of NVMe SSDs. Micron wasn’t new to the PCIe SSD market, but their early products predated the NVMe standard and instead used a proprietary protocol. The 9100 series uses Micron 16nm MLC NAND flash and a Microsemi Flashtec NVMe1032 controller.

Our Micron 9100 MAX 2.4TB sample is the fastest model from the 9100 series, and the second-largest. Micron recently announced the 9200 series that switched to 3D TLC NAND for a huge capacity boost and adopted the latest generation of Microsemi controllers to allow for a PCIe x8 connection, but we don’t have a sample.

Intel Optane SSD 900p 280GB

Last month, Intel launched their consumer Optane SSD based on the same controller platform as the P4800X. The Optane SSD 900p has some enterprise-oriented features disabled and is rated for only a third the write endurance, but offers essentially the same performance as the P4800X for about a third the price.

This makes the 900p a very attractive option for users who need Optane level of performance but don’t want to pay for the absolute highest write endurance. It isn’t intended as an enterprise SSD, but the 900p can compete in this space.

The Intel Optane SSD DC P4800X 750GB is expected to perform the same as the 375GB model – and the same as the consumer Optane SSD 900p for that matter. Rather than simply repeat the same tests those drives have already been been subjected to, this review seeks to dig deeper into the performance characteristics of the Optane SSD and explore what it takes to extract the full performance of the drive. There are quite a few esoteric system settings that can have an impact, since a microsecond gained or lost matters more to an Optane SSD than to a flash-based drive.

Multiple Queues

One of the most important features of NVMe that allows for higher performance than the AHCI protocol used for SATA is support for multiple queues of I/O commands. The NVMe protocol allows for up to 64k queues each with up to 64k pending commands, but current hardware cannot actually reach those limits.

Intel’s first-generation NVMe controller (used in the P3x00 drives and the SSD 750) supports a total of 32 queues in hardware: the admin queue, and 31 I/O queues. For the P4x00 generation NAND SSDs, the new controller supports 128 queues. However, the Optane SSD controller still only supports 32 queues, because that’s more than enough to reach the full performance of the drive even if each queue only contains one command at a time. The Microsemi controller used by the Micron 9100 MAX supports 128 queues, the same as Intel’s latest flash SSDs.

Achieving the highest and most consistent performance requires each CPU core that is performing I/O to have its own NVMe queue assigned to it. When multiple cores share the same queue, the synchronization overhead can increase latency and reduce performance consistency. The Linux NVMe driver currently spreads the NVMe queue assignments evenly across all CPU cores. On our server with 36 physical cores, this means that the Intel SSDs with only 31 I/O queues require several cores on each socket to share queues. Rather than patch the kernel to allow for manual queue assignment, the testbed was simply configured to only enable 16 our of 18 cores on each CPU. This caused the OS to assign one queue exclusively per core on CPU #2 that the SSDs are attached to (two cores on CPU #1 share a queue). None of the storage benchmarks in this review would benefit significantly from having two extra cores available, and 16 cores is more than enough to saturate any of these SSDs.

Sector Size

NVMe SSDs are capable of supporting different sector sizes. Everything defaults to the 512-byte sector size for the sake of backwards compatibility, but enterprise NVMe SSDs and many client NVMe SSDs also support 4kB sectors. Many enterprise SSDs also support sector formats that include between 8 and 128 extra bytes of metadata to support end to end data protection. Just as with 4k sector sizes on hard drives, using 4kB sectors on NVMe SSDs can slightly reduce overhead.

Switching between sector sizes is accomplished using the NVMe FORMAT command, which is also used for secure erase operations. On most flash-based SSDs, a NVMe FORMAT command takes only a few seconds. On Optane devices, the drive actually performs a low-level format that touches substantially all of the 3D XPoint memory and takes about as long as filling up the drive sequentially. With the 750GB Optane SSD DC P4800X, a NVMe FORMAT command takes several minutes and requires overriding the default command timeout settings. (Coincidentally, a kernel patch to fix this issue showed up on the linux-nvme mailing list while I was testing the drive.)

Interrupts vs Polling

With the performance offered by Optane SSDs, the CPU can become a bottleneck even when running synthetic storage benchmarks. The latency of 3D XPoint memory is low enough that things like CPU context switches, interrupt handler latency and inter-core synchronization can significantly affect results. We’ve already covered how the test hardware was configured for high performance, but there’s further room to fine tune the software configuration.

There are two main ways for the operating system to find out when the SSD has completed a command. The normal method and the best general-purpose choice is for the OS to wait for the drive to signal an interrupt. Upon receipt of an interrupt, the OS will check the relevant NVMe completion queue and pass along the result to the application. The alternative is polling: while waiting for an I/O operation to complete, the CPU constantly checks the status of the completion queue. This wastes a ton of CPU time and is usually only worthwhile when the CPU has nothing better to do and needs the result as soon as possible. However, polling does shave one or two microseconds off the SSD’s latency. When CPU power management is enabled, polling can also keep the processor awake when it is awaiting completion of an I/O command, potentially leading to more significant latency and throughput advantages relative to interrupt-driven I/O.

(source: Western Digital)

Recent Linux kernel versions support a hybrid polling mode, where the OS will estimate how long to sleep or run other tasks before starting to poll for completed I/O. This provides a reasonable balance between storage latency and CPU overhead, at least where storage performance is still a high priority. For this review, all drives were set to use the hybrid polling mode. However, polling is by default not used for most ordinary I/O:

APIs

At the application layer, there are several ways to accomplish I/O. Most methods (like the simple and ancient read() and write() system calls) are synchronous, making a single request at a time and blocking: the thread waits until the I/O operation is done before continuing. When these APIs are used, the only way an application can produce a queue depth greater than 1 is to have multiple threads performing I/O simultaneously.

Most operating systems also have APIs for asynchronous I/O, where the application sends requests to the OS but the application chooses when to check if those requests have completed. This allows a single thread to generate queue depths greater than one. Both asynchronous I/O and multithreaded synchronous I/O allow for high queue depths to be generated by a single application, but they are also more complex to use than simple single-threaded synchronous I/O.

Linux has also recently gained a new set of system calls for performing synchronous I/O and optionally flagging the read or write operation as high priority. This signals the OS to poll for completion of the I/O operations, reducing latency at the expense of burning CPU time instead of idling. These preadv2() and pwritev2() system calls are close to being a drop-in replacement for simple read() and write() system calls, but in most programming languages there’s a standard library providing abstracted I/O interfaces, so switching an application to use the new system calls and flag some or all I/Os as high priority is not trivial. Currently, the preadv2() and pwritev2() system calls are the only Linux storage API that can trigger the kernel to use polling instead of waiting for an interrupt.

For application developers seeking to squeeze every last microsecond of latency out of their I/O, Intel created the Storage Performance Development Kit (SPDK), an offshoot of their Data Plane Development Kit (DPDK) for networking. The projects allow applications to directly access storage or network devices without going through the OS kernel’s drivers. SPDK is an open-source library that is not tied to Intel hardware and can be used on Linux or FreeBSD to access any vendor’s NVMe SSD with its polled mode driver. Using SPDK requires more invasive application changes than any of the above mentioned APIs, but it is also the fastest and most direct way for an application to interact with NVMe SSDs. Due to time constraints, this review does not include benchmarks with SPDK.

I/O Scheduler

Most operating systems include some form of I/O scheduler to determine which operations to send to the disk first when multiple processes need to access the disk. Linux includes several I/O schedulers with various strengths and weaknesses on different workloads. For real-world use, the proper choice of I/O scheduler can make a significant difference in overall performance. However, for benchmarking, I/O schedulers can interfere with attempts to test at specific queue depths and with specific I/O ordering. For this review, all SSDs were set to use the Linux no-op I/O scheduler that does no reordering or throttling, and consequently also has the least CPU overhead.

Intel’s 3D XPoint memory is not bound by the page and erase block structure of NAND flash. Intel hasn’t disclosed what the native word size of the 3D XPoint memory array is, but the Optane SSD DC P4800X as a whole is optimized for 4kB or larger transfers. Real-world I/O isn’t always constrained to just the 4kB and 128kB transfer sizes most of our synthetic benchmarks use, so the next several tests look at single-threaded QD1 performance of transfers ranging from a single 512-byte sector up to 1MB blocks. For random reads and writes, the impact of issuing commands with suboptimal alignment is also considered. Depending on how a drive is organized internally, unaligned accesses can significantly increase the amount of work the controller needs to do.

	Vertical Axis scale:
Block Aligned	Linear	Logarithmic
512B Aligned	Linear	Logarithmic

The random read performance of the Optane SSDs is completely beyond the reach of the flash based SSDs, even for large block sizes. Even when the reads aren’t aligned to the block size or to the drive’s preferred 4kB alignment, performance is still great until the block size reaches 128kB, the largest the Optane SSD can transfer from a single command. At that point, the Optane SSDs slow down slightly before performance continues to grow at a more moderate pace.

	Vertical Axis scale:
Block Aligned	Linear	Logarithmic
512B Aligned	Linear	Logarithmic

The random write performance of the Optane SSDs is not much better than the Intel P3700, especially for writes smaller than 4kB. The Intel P3608 and Micron 9100 MAX are slower across the entire range of block sizes. For unaligned writes of larger block sizes, the Optane SSDs hit a hard limit at around 1.3GB/s while properly aligned writes can approach 2GB/s for large blocks.

Vertical Axis scale:	Linear	Logarithmic

The Intel P3700 delivers a slightly higher QD1 sequential read throughput for small block sizes, likely due to the controller caching whole NAND pages in RAM. At the larger block sizes more typically used for sequential I/O, the Optane SSDs are on top and the many of the flash-based SSDs cannot reach full performance without going beyond QD1.

Vertical Axis scale:	Linear	Logarithmic

None of the SSDs perform particularly well for writes smaller than 4kB, but the Optane SSDs do have a clear advantage. As transfer size grows, the Optane SSDs pick up speed faster than the flash-based SSDs. The Intel P3608 is the first to start hitting a performance ceiling, while the Micron 9100 is almost able to catch up to the Optane SSDs.

This next batch of tests measures how much performance can be driven by a single thread performing asynchronous I/O to produce queue depths ranging from 1 to 64. With these fast SSDs, the CPU can become the bottleneck at high queue depths and clock speed can have a major impact on IOPS.

4kB Random Reads

With a single thread issuing asynchronous requests, all of the SSDs top out around 1.2GB/s for random reads. What separates them is how high a queue depth is necessary to reach this level of performance, and what their latency is when they first reach saturation.

Throughput:	IOPS		MB/s
Latency:	Mean	Median	99th Percentile	99.999th Percentile

For the Optane SSDs, the queue depth only needs to reach 4-6 in order to be near the highest attainable random read performance, and further increases in queue depth only add to the latency without improving throughput. The flash-based SSDs require queue depths well in excess of 32. Even long after the Optane SSDs have reached saturation an latency has begun to climb, the Optane SSDs continue to offer better QoS than the flash SSDs.

4kB Random Writes

The Optane SSDs offer the best single-threaded random write performance, but the margins are much smaller than for random reads, thanks to the write caches on the flash-based SSDs. The flash SSDs have random write latencies that are only 2-3x higher than the Optane SSD’s latency, and the throughput advantage of the Optane SSD at saturation is less than 20%.

Throughput:	IOPS		MB/s
Latency:	Mean	Median	99th Percentile	99.999th Percentile

The Optane SSDs saturate around QD4 where the CPU becomes the bottleneck, and the flash based SSDs follow suit between QD8 and QD16. Once all the drives are saturated at about the same throughput, the Optane SSDs offer far more consistent performance.

128kB Sequential Reads

With the large 128kB block size, the sequential read test doesn’t hit a CPU/IOPS bottleneck like the random read test above. The Optane SSDs saturate at the rated throughput of about 2.4-2.5GB/s while the Micron 9100 MAX and the Intel P3608 scale to higher throughput.

Throughput:	IOPS		MB/s
Latency:	Mean	Median	99th Percentile	99.999th Percentile

The Optane SSDs reach their full sequential read speed at QD2, and the flash-based SSDs don’t catch up until well after QD8. The 99th and 99.999th percentile latencies of the Optane SSDs are more than an order of magnitude lower when the drives are operating at their respective saturation points.

128kB Sequential Writes

Write caches again allow the flash-based SSDs to approach the write latency of the Optane SSDs, albeit at lower throughput. The Optane SSDs quickly exceed their specified 2GB/s sequential write throughput while the flash-based SSDs have to sacrifice low latency in order to reach high throughput.

Throughput:	IOPS		MB/s
Latency:	Mean	Median	99th Percentile	99.999th Percentile

As with sequential reads, the Optane SSDs reach saturation at a mere QD2, while the flash-based SSDs need until around QD8 to scale up to full throughput. By the time the flash-based SSDs reach their maximum speed, their latency has at least doubled.

Workloads consisting of a mix of reads and writes can be particularly challenging for flash based SSDs. When a write operation interrupts a string of reads, it will block access to at least one flash chip for a period of time that is substantially longer than a read operation takes. This hurts the latency of any read operations that were waiting on that chip, and with enough write operations throughput can be severely impacted. If the write command triggers an erase operation on one or more flash chips, the traffic jam is many times worse.

The occasional read interrupting a string of write commands doesn’t necessarily cause much of a backlog, because writes are usually buffered by the controller anyways. But depending on how much unwritten data the controller is willing to buffer and for how long, a burst of reads could force the drive to begin flushing outstanding writes before they’ve all been coalesced into optimal sized writes.

Our first mixed workload test is an extension of what Intel describes in their specifications for throughput of mixed workloads. A total of 16 threads are used, each performing a mix of random reads and random writes at a queue depth of 1. Instead of just testing a 70% read mixture, the full range from pure reads to pure writes is tested at 10% increments.

Vertical Axis units:	IOPS	MB/s

The Intel Optane SSD DC P4800X is slightly faster than the Optane SSD 900p throughout this test, but either is far faster than the flash-based SSDs. Performance from the Optane SSDs isn’t entirely flat across the test, but the minor decline in the middle is nothing to complain about. The Intel P3608 and Micron 9100 both show strong increases near the end of the test due to caching and combining writes.

Mean	Median	99th Percentile	99.999th Percentile

The mean latency graphs are simply the reciprocal of the throughput graphs above, but the latency percentile graphs reveal a bit more. The median latency of all of the flash SSDs drops significantly once the workload consists of more writes than reads, because the median operation is now a cacheable write instead of an uncacheable read. A graph of the median write latency would likely show writes to be competitive on the flash SSDs even during the read-heavy portion of the test.

The 99th percentile latency chart shows that the flash SSDs have much better QoS on pure read or write workloads than on mixed workloads, but they still cannot approach the stable low latency of the Optane SSDs.

Aerospike is a high-performance NoSQL database designed for use with solid state storage. The developers of Aerospike provide the Aerospike Certification Tool (ACT), a benchmark that emulates the typical storage workload generated by the Aerospike database. This workload consists of a mix of large-block 128kB reads and writes, and small 1.5kB reads. When the ACT was initially released back in the early days of SATA SSDs, the baseline workload was defined to consist of 2000 reads per second and 1000 writes per second. A drive is considered to pass the test if it meets the following latency criteria:

• fewer than 5% of transactions exceed 1ms
• fewer than 1% of transactions exceed 8ms
• fewer than 0.1% of transactions exceed 64ms

Drives can be scored based on the highest throughput they can sustain while satisfying the latency QoS requirements. Scores are normalized relative to the baseline 1x workload, so a score of 50 indicates 100,000 reads per second and 50,000 writes per second. We used the default settings for queue and thread counts and did not manually constrain the benchmark to a single NUMA node, so this test produced a total of 64 threads sharing a total of 32 CPU cores split across two sockets.

The usual runtime for ACT is 24 hours, which makes determining ta drive’s throughput limit a long process. In order to have results in time for this review, much shorter ACT runtimes were used. Fortunately, none of these SSDs take anywhere near 24h to reach steady state. Once the drives were in steady state, a series of 5-minute ACT runs was used to estimate the drive’s throughput limit, and then ACT was run on each drive for two hours to ensure performance remained stable under sustained load.

ACT Transaction Latency
Drive		% over 1ms	% over 2ms
Intel Optane SSD DC P4800X 750GB		0.82	0.16
Intel Optane SSD 900p 280GB		1.53	0.36
Micron 9100 MAX 2.4TB		4.94	0.44
Intel SSD DC P3700 1.6TB		4.64	2.22
Intel SSD DC P3608 (single controller) 800GB		4.51	2.29

When held to a specific QoS standard, the two Optane SSDs deliver more than twice the throughput than any of the flash-based SSDs. More significantly, even at their throughput limit, they are well below the QoS limits: the CPU is actually the bottleneck at that rate, leading to overall transaction times that are far higher than the actual drive I/O time. Somewhat higher throughput could be achieved by tweaking the thread and queue counts in the ACT configuration. Meanwhile, the flash SSDs are all close to the 5% limit for 1ms transactions, but are so far under the limit for longer latencies that I’ve left those numbers out of the above table.

There aren’t really any surprises to this review. As promised, the Intel Optane SSD DC P4800X 750GB performs about the same as the 375GB model we tested earlier this year, which puts it far above the flash-based competition. The doubling of the drive’s capacity has brought it up to a far more usable level where it can more easily be regarded as either an expansion of memory or as a reasonably large storage device that is extremely fast.

The Optane SSD DC P4800X is not a one size fits all enterprise SSD, but for certain use cases it has almost no competition. SSDs using SLC NAND once offered latency close to the Optane SSD, but their high prices made them a niche product that most manufacturers decided wasn’t viable. 3D XPoint memory is significantly more expensive than current 3D MLC and 3D TLC NAND flash memory, but the low latency and high performance consistency it enables have a place in today’s enterprise SSD market.

Testing the ostensibly consumer-oriented Optane SSD 900p in the context of enterprise storage reveals the 900p to be just as competent in the performance department. There are several enterprise-class features missing from the 900p, most notably support for different sector sizes with protection metadata. Support for out of band management and is more of a convenience than a necessity except in large scale cloud hosting deployments. The missing enterprise features are not a showstopper for every enterprise customer. Otherwise, the 900p is roughly the P4800X in different capacities with a lower write endurance rating and a proportionally lower price. The 900p is essentially an entry-level Optane enterprise SSD, with a value proposition that is at least as attractive as the Optane P4800X.