IBM System z9 I/O Improvements
Part 1: Channel Subsystem
March, 2006
by Steve Guendert

About the Author

Dr. Steve Guendert, Global Solutions Architect, Brocade Communications Steve is a PMI certified Project Management Professional (PMP), and he has an MBA, and a M.S. in Management Information Systems. He is a well regarded FICON consultant and is one of the industry’s experts on the subject. He lectures and consults regularly on FICON technology as well as business case justifications for migrating from ESCON to FICON. He is currently a Ph D Candidate and is writing his doctoral dissertation on Enterprise I/O Subsystems.

[Hide]

Just when you thought that IBM achieved the peak of mainframe I/O performance with the z990, they announced the new System z9 on July 25, 2005. The IBM System z9 brought about tremendous improvements in overall I/O performance over the already formidable z990. This 2 part article will cover the I/O improvements that are specific to the IBM System z9. Part one will cover the channel subsystem improvements which include the new MIDAW facility, and multiple subchannel sets. Part two will cover the other hardware improvements made to the z9, including STI improvements and redundant I/O reconnect.

Channel Subsystem Improvements

The first major improvement we will discuss in the MIDAW facility. The second is with the logical channel subsystem.

MIDAW Facility

MIDAW stands for modified IDAW. (An IDAW is an indirect access word that is used to specify data addresses for I/O operations in a virtual environment such as we have with a z/OS environment). The MIDAW facility is a new CCW-indirect-data-address word facility being added to z/Architecture to coexist with the current IDAW facility. It is a new system architecture and software exploitation designed to improve FICON performance on the System z9. Both MIDAW and IDAW facilities offer, for FICON and ESCON channels, alternatives to using CCW data chaining in channel programs. Both facilities are designed to reduce channel, director, and control unit overhead by reducing the number of CCWs and frames processed. The MIDAW facility is usable in certain case where the IDAW facility is not because it does not have IDAW boundary and data length restrictions. The MIDAW facility is supported on z/OS 1.6 and higher.

The MIDAW facility is designed to:

1. Be compatible with existing IBM and non IBM disk control units (Note: non IBM storage devices will require support from their vendors and they should be contacted as part of the installation systems assurance process).
2. Decrease response time for exploiting I/O.
3. Increase the number of I/O operations per second that can be processed and thus move more data per second, especially on faster FICON channels.

Applications that may benefit include: DB2, VSAM, Partitioned Data Set Extended (PDSE), Hierarchical File System (HFS), z/OS File System (zFS), and other datasets exploiting striping and compression. Primary benefits include improved channel utilization and significantly improved I/O response times. Internal IBM DB2 Table Scan tests comparing MIDAW facility configurations to pre-MIDAW configurations showed:

• 36% to 58% reduction in response times.
• 35% to 56% reduction in channel busy.
• 56% to 126% improvement in I/O throughput.

The MIDAW facility provides a CCW/IDAW structure that is much more efficient for certain categories of data-chaining I/O operations. It coexists with the current IDAW facility. Pre-MIDAW we were limited in the useability of IDAWS to straight forward buffering of a sequential record. This was due to the fact that the existing IDAW design allowed the first IDAW in a list to point to any address within a page. Subsequent IDAWS (in the same list) would have to point to the first byte in a page. All but the first and last IDAW in a list was forced to deal with complete 2K or 4K units of data.

I/O data blocks that have a more complex internal structure (meaning portions of the data block are directed into separate buffer areas, i.e. scattered read/write) may be processed using CCWs with data chaining. Unfortunately, data chaining CCWs is inefficient in a modern z/OS IO environment for a variety of reasons. These reasons include control unit processing and exchanges, use of switched fabric FICON networks, and the number of channel frames required.

z/OS extended format data sets use internal structures that typically are not visible to the application program. These extended format data sets require scatter read/write operations. Therefore, CCW chaining is required, producing less than optimal I/O performance. The rapid growth in the use of extended format data sets is what prompted the development of the MIDAW facility. The scatter read/write effect of MIDAWS make it possible to efficiently send small control blocks embedded in a disk record to buffers that are separately used for larger data areas within the record. MIDAW operations are on a single block in the matter of data chaining (not to be confused with CCW command chaining). The data is therefore sent to the control unit by the channel in one block of data. For a FICON channel this means all the data is associated with the same command information unit (IU). Also, please note that there is no change to ESCON, FICON or control unit implementation. However, with MIDAW there is a greater chance that more information units (up to 16), may be sent to the control units in a single burst.

Z9 Multiple Subchannel Sets

This functionality tends to get confused with multiple Logical Channel Subsystems (LCS). It’s a new z9 feature and is not the same thing as LCS.

Typically a subchannel represents an addressable device. For example, a disk control unit with 20 drives uses 20 subchannels for base addresses. The addressable device is then associated with a device number and the device number is commonly (in error) known as the device address. Current architecture limits subchannel numbers to four hexadecimal digits. These four hexadecimal digits provide 64K addresses. This is known as a set. To date, IBM has reserved 1024 subchannels, which leaves 63K subchannels for general use. Starting with the z9-109 server, the number of reserved subchannels has been decreased from 1024 to 256.

The rapid development and release of new technologies such as global/metro mirror, GDPS-Hyperswap, and Parallel Access Volumes (PAV) has made the limitation of 63K subchannels an issue for larger zSeries installations. PAV provides a great example: a single disk drive with PAV often consumes at least 4 subchannels on its own (representing a base address and 3 aliases).

In an effort to solve this problem on the z9, IBM allows sets of subchannels (addresses) with the first iteration to include two such sets. Each of the two sets provides 64K addresses. The first set is referred to as Subchannel set 0. This set still reserves 256 subchannels for IBM use (an improvement from reserving 1024). So, 768 subchannels are now available for end users. Subchannel set 1 allows all 64K addresses to be used by the end user. The current z/OS implementation restricts Subchannel set 1 to disk alias subchannels. Subchannel set 0 may be used for both base and alias addresses.

What Multiple Subchannel sets really do is provide more room for growth in I/O device configuration. Moving the alias devices into the second subchannel set has created additional space for device number growth. The storage attachment capability of the z9 has been dramatically improved over its predecessors. For example, in the largest case, using 3390 volumes with 54 GB/volume and 768 additional volumes, you could have 41 Terabytes of additional disk storage addressability" (i.e., 54 GB/volume * 768 volumes = 41 TB).

Finally, please keep in mind that the appropriate subchannel set number must be included in your IOCP definitions/HCD definitions that produce your IOCDS). The subchannel set number defaults to zero, and IOCP/HCD changes are only needed when using subchannel set one.

In conclusion, the MIDAW facility and Multiple Subchannel sets provide some complimentary improvements to the channel subsystem. Next month’s article will include a discussion on some of the internal hardware improvements with the STIs.