FICON Channel-to-Channel (CTC): A Primer
Part I
May, 2005
by Stephen Guendert

About the Author

Dr. Steve Guendert, Global Solutions Architect, Brocade Communications Steve is a PMI certified Project Management Professional (PMP), and he has an MBA, and a M.S. in Management Information Systems. He is a well regarded FICON consultant and is one of the industry’s experts on the subject. He lectures and consults regularly on FICON technology as well as business case justifications for migrating from ESCON to FICON. He is currently a Ph D Candidate and is writing his doctoral dissertation on Enterprise I/O Subsystems.

[Hide]

This two part article will examine the topic of FICON CTC. Part I will offer a brief review of ESCON CTC, and describe some of the advantages FICON CTC presents over ESCON CTC. Part II will discuss some of the details you need to know in implementing FICON CTC. Unless stated otherwise, the reader is asked to assume that the mainframe model we are discussing is a zSeries machine.

What exactly is CTC?

CTC stands for channel to channel (connectivity). It is used for server to server connectivity in the mainframe environment. Server to server communications support data movement and processor-to-processor communications. Other common terminology used for server to server connectivity includes clustering, grid computing, and server area networks. In the open systems world we typically see server to server connectivity done via Gb Ethernet, Fibre Channel (Virtual Interface over Fibre Channel), or Infiniband. For today’s mainframe environments, server to server access would be either CTC over ESCON or CTC over FICON.

CTC is arguably the most efficient way of getting data in and out of the mainframe. CTC control units and their associated devices provide THE infrastructure for intersystem communication. CTC was originally developed by IBM prior to the development of the Systems Network Architecture (SNA), and it uses a sparse instruction set that simplifies mainframe access for certain applications such as file and print distribution. Furthermore, CTC bypasses the Virtual Telecommunications Access method (VTAM) which is how SNA traffic gets processed. By eliminating VTAM from the equation we realize faster data transfer rates with decreased response times for the systems that have been developed to exploit the inherent advantages in using CTC.

The CTC function instruction set mentioned above actually simulates an I/O device that will then be used by one system control program to communicate directly with another system control program. In the course of doing so, it provides the synchronization and the data path used for the data transfer between two channels. When used to connect two channels associated with different systems, we have what is called loosely coupled multiprocessing systems. The two channels connected by the CTC connection view this connection as an unshared I/O device. The CTC is selected by the IOS and responds in the same exact manner as any other I/O device. How it differs from these other I/O devices is that it will use commands to open a path between the two channels it connects and then synchronizes the operations that are to be performed between the two channels. For an analogy, think of this as resembling the transporter technology used in Star Trek but without the shimmering lights, sound effects, and the need to say "energize".

IBM currently supports CTC for:

• Parallel channels via IBM’s 3088 MCCU
• ESCON channels
• FICON channels via a 9032-5 ESCON Director with the FICON Bridge (FCV) feature
• FICON Native CTC (FCTC)

The primary z/OS, OS/390 and z/VM exploiters of CTC communication include:

• IMS read/write devices

• XCF (Cross System Coupling Facility) for both pathin and pathout devices used for sysplex intersystem communication (z/OS and OS/390 only). For small messages, CTC can often be more efficient than passing messages via the coupling facility (CF).

• VTAM read/write devices
• TCP/IP read/write devices.
• VM pass through facility
• Remote Spooling Communications Subsystem (RSCS)

CTC can be used in the following configurations:

• Base Sysplex. In this configuration the CPCs are connected by CTC channels and a shared data set used to support communications. If more than 1 CPC is involved, we will use a Sysplex Timer to synchronize the time on all involved systems.

• Parallel Sysplex. Multi system data sharing at high levels of performance will be enabled by the Coupling Facility. In a parallel sysplex, the CTCs are used to support cross-system Coupling Facility (XCF) signaling paths between systems in the sysplex.

• Loosely Coupled. (Mentioned earlier). This configuration has more than one Central Processing Complex (CPC), shares DASD, but does NOT share central storage. The CPCs are connected by CTC channels and are managed by multiple z/OS images.

Prior to the general availability of ESCON in the early 1990s, the IBM 3088 Multisystem Channel Communication Unit device (control units) provided parallel channel-attached CTC support. The 3088 control unit is no longer marketed by IBM. As of Dec 31, 2004 the IBM 9032-5 ESCON director and FCV feature are also no longer marketed by IBM. Today’s best options for CTC are ESCON and FICON Native CTC, and those technologies will be the focus of the rest of this article with an emphasis on FICON.

ESCON CTC Technology

ESCON CTC enables direct communication between multiple CPCs and/or CPCs with LPARS as illustrated below. A stand alone CTC control unit is no longer used to provide CTC adapter and switching functions. ESCON CTC communication requires a pair of ESCON channels. One channel is defined as an ESCON CTC channel on one side of the connection. The ESCON CTC Control Unit Function is provided in the microcode (LIC) of the ESCON CTC channel. The second channel is defined as an ESCON CNC channel to operate on the other side of the connection. The CTC channel path will communicate with a CNC channel and vice-versa. Connections can be either point to point with no intervening ESCON director, or switched point to point via an ESCON director as seen in the figure below.

3 additional key points:

1. The ESCON channel that is defined as CTC cannot be used for other I/O device support. It can only support the ESCON CTC Control Unit Function and CTC devices. In other words, it cannot also support disk, tape, etc on the same defined channel. You use this ESCON channel solely for CTC functions.

2. The ESCON channel defined as CNC can support ESCON CTC control unit definitions AND other I/O control unit type definitions such as disk and tape. However, this can only be done if the channel is connected in a switched point to point topology.

3. ESCON CTC channels support a maximum of 512 unit addresses (devices) and up to 120 logical control units (LCUs). This limitation means that multiple pairs of CTC and CNC channels must be used in an installation with a large number of interconnected LPARs on the same physical processor or to LPARS/images on other processors. In other words, for an environment with large z990 servers that have high LPAR counts, ESCON CTC can use up many of your available channels in short order.

Shared vs. unshared channels

Recall that ESCON channels can be defined as shared, dedicated, or reconfigurable. Channels that reside in a CEC without Multiple Image Facility (MIF) cannot be shared by the CEC’s LPARs. I.e. these unshared channels are dedicated to a single partition. If this unshared channel were to be further defined as reconfigurable, it can be reconfigured from one partition to another partition. MIF allows for channel sharing. LPARs can share the channel paths, and consequently they can also share any control units and associated I/O devices that are also configured to the shared channels. So, if this same CEC is running in LPAR mode, the logical partitions can share channel paths. This reduces the number of physical connections between processor complexes. Both ESCON CTC and CNC channels can be defined as shared channels.

The topic of z890/990 spanned channels is best left to a separate discussion. The author recommends referencing the IBM Redbook IBM eserver zSeries 990 Technical Guide for a detailed discussion on the Logical Channel Subsystem (LCSS) and the concept of spanned channels.

FICON CTC Technology

Note: FICON protocol and its characteristics are not discussed here. This section focuses solely on FICON CTC, and in particular, what has changed from ESCON CTC. Aspects of the FICON protocol will be important for the following section which will make the case for why you should use FICON CTC over ESCON CTC, but it is assumed that the reader is already familiar with the basics of the FICON protocol and technology. Finally, the remainder of the discussion on FICON will refer to FICON Native CTC. Therefore, unless otherwise stated, "FICON" in the remainder of this article should be assumed to refer to FICON Native.

FICON CTC consists of two FICON (FC) channel FCTC control units. This method of CTC connectivity, which has been available since October 2001, increases bandwidth available for CTC by up to a factor of 10. At least one of the two FCTC control units needs to be defined on a zSeries server at Licensed Internal Code (LIC) driver level 3C or later. There are multiple options for implementing a FICON CTC configuration, which will be covered in Part 2 of this article. With that said, there are three commonalities that will apply to all FICON CTC configurations:

• The FICON channel at each end of the FICON CTC connection, supporting the FCTC control units, can also communicate with other FICON control units (disk, tape, virtual tape). This is very different from ESCON.

• The mainframe at each end of a FICON CTC connection will use a FICON native channel (defined as CHPID type FC).

• The FC channel at each end of the CTC connection has a FICON CTC control unit defined. An FCTC control unit can be defined in IOCP on any FICON native channel; however, the FCTC control unit function will only be provided by a zSeries server at LIC driver level 3C or later. The FCTC control function on a zSeries server at LIC driver level 3C or later can communicate with an FCTC control unit defined on a FC channel on any of the following:

  • S/390 G5 or G6 server
  • Another zSeries server

• FICON CTC communication does not require a pair of channels because it can communicate with any FICON (FC) channel that has a corresponding FCTC control unit defined. This means, that unlike ESCON CTC, FICON CTC communications can be provided using only a single FICON channel per processor. However, even though a single FC channel per processor can provide CTC connections across multiple processors, for large FICON configurations it is best practice to use at least one pair of FC channels. This will allow you to maintain the same CTC definition methodology that you used with ESCON, but with an added benefit: the FICON channels can simultaneously support the definition of FCTC control units and disk/tape/virtual tape control units.

Please note that from the above points that even though FICON CTC control units are defined at each end, only one end provides the FICON CTC control unit function. A fairly complex algorithm is used to determine which channel will provide the FICON CTC control unit function. The algorithm is run during the initialization of the logical connection between the two ends of the FICON CTC connection. Ideally, this algorithm results in balancing the number of FCTC CU functions provided at each end of the logical connection. More details of the algorithm are discussed in part 2 of this article, as well as some z990 and z890 specifics.

A comparison of channel technologies for CTC

The above table summarizes where the differences are in the two technologies for CTC communication. The author feels it is very safe to conclude that FICON is a better technology for FICON CTC. The multifold gains in bandwidth, number of supported concurrent I/O operations, and addressability all result in much higher performance for your CTC environment. FICON also is a much more efficient protocol in terms of CCW (Channel Command Word) management, resulting in being able to go further distances unrepeated and maintain high levels of performance without experiencing the phenomena known as data droop. (Data droop is quite prevalent in the ESCON world). This performance gain allows for the reduction in the number of channels used for CTC, which in turn results in a lower total cost of ownership attributable to CTC needs.

FICON CTC sounds too good to be true doesn’t it? Well, as the famous saying/acronym goes in Economics 101: TINSTAAFL. There Is No Such Thing As A Free Lunch. FICON CTC can be much more challenging to plan and implement when compared to ESCON so there is a tradeoff. That subject will be the focus of Part 2 of this article.

TO BE CONCLUDED.....