In this section, we are going to understand the Mainframe testing, which is used to test the software or the applications and services developed on Mainframe Systems.

And we also learn about the mainframe attributes, types of mainframe testing, how to perform it, the different challenges and Troubleshooting while performing the mainframe testing, various commands used in Mainframe Testing, some common issues faced during mainframe testing, and Mainframe Automation Testing Tools.

Before understanding the concept of Mainframe testing, we are going to learn about the Mainframe.

What is Mainframe?

The mainframe is a multi-user, high performance and high-speed computer system. The mainframe is the most reliable, scalable, and secured machine systems.

In other words, we can say that these systems are used for the intend of larger-scale computing, which involves great availability and safety. The mainframe systems are mostly used in various subdivisions such as Retail, insurance, finance, and other critical areas where massive data are processed several times.

In this, we can perform some million Instructions per second [up to 569,632 MIPS] with the help of the below aspects:

• Maximum Input/output bandwidth: The connections between drives and processors have a few choke points if we have the extreme input and output bandwidth.
• Reliability: Frequently, the mainframes agree to the graceful degradation and service though the system is running.
• Reliable single-thread performance: It is important for practical operations against a database.
• Maximum Input/output connectivity: The maximum input/output connectivity implies that the mainframes excel at providing huge disk farms.

After understanding the mainframe concept, now we have come to our main point of discussion on mainframe testing.

What is mainframe testing?

The main objective of mainframe testing is to make sure the dependability, performance, and excellence of the application or service by verification and validation approaches and check if it is ready to deploy or not.

The tester only needs to know about the CICS screens’ navigations while performing the Mainframe testing as these are custom-built for specific applications.

And the tester does not have to worry about the emulator set up on the machine if any modification happens to the code in COBOL, JCL, and so on.

Where we performed the mainframe testing?

Generally, the mainframe testing is executed on the deployed code with multiple data combinations set into the input file. In other words, we can say that the Mainframe Applications must be tested completely before the production run.

The Mainframe application, or else known as the batch job, is tested in contradiction of the test cases developed with requirements.

The software or the application which runs on the mainframe can be retrieved through the terminal emulator.

After understanding the mainframe testing, we will look into the several characteristics of mainframe testing.

Mainframe Testing Methodology

In Mainframe testing, the Software or an application are retrieved by end-users in a way, which is diverse from Web applications.

And the application tester should know these significant differences, which are shown below:

Web Applications	Mainframe Applications
The web applications are acquired via two-tier architecture [Client/Server] or three-tier architecture [Presentation/apps/DB storage layers].	In the mainframe application, the end-user must log into the system directly.
The web applications retrieve across Browser or UI.	Terminal Emulator must retrieve the web applications.
In this, the testing can be performed straight on the application screen.	In this, the tester should have precise knowledge of mainframe operations.
In the web application, some of the processing is done on the Client-side, and the applications should install at the right place before the testing begin.	In mainframe testing, the Terminal Emulator is the only software, which has to be loaded in a client machine for testing where the applications exist in the main server.

Mainframe Attributes

In mainframe testing, we have the following mainframe features; let us see them one by one:

• Multiprogramming
• Time sharing
• Virtual storage
• Spooling
• Batch processing

Multiprogramming

• The multiprogramming attribute is a facility, which allows us to make efficient use of the CPU.
• The computer implements various programs at the same time.

Time-Sharing

• Time-share processing is also known as Foreground Processing, whereas batch job processing is known as Background Processing. Therefore, it is known as an Interactive Processing because it allows the user to relate with the computer directly.
• In a time-sharing system each user can access to the system over the terminal device.

Virtual Storage

• Virtual storage uses disk storage as an extension of real storage.
• It is a technique to use memory efficiently to store and perform many sized tasks.

Spooling

• The Spool means Simultaneous Peripheral Operations Online, which is used to accumulate the output of a program or an application.
• If it is required, then the spooled output is directed to output devices like a printer.

Batch Processing

• Batch processing is a technique where we can accomplish any task in units also known as jobs.
• We can perform one or more programs in an order based on jobs.
• The Job scheduler concludes about the sequence where the jobs are implemented.
• Jobs are planned as per their importance and class to maximize the average output.
• The batch processing gives us the necessary information with the help of JOB CONTROL LANGUAGE(JCL).

How to do Mainframe Testing

The mainframe testing can be performed in two ways, either manually or using some automation tools such as QTP, REXX, and IBM application performance analyzer etc. But generally, we will execute the Mainframe testing manually.

To do mainframe testing, we need to follow the below steps:

Step1: Planning

Firstly, the business team or the development team constructs the test plans including the Business requirement document, System requirement document, other project documents and the inputs. And it controls how a specific item or process is going to be changed in the release’s cycle.

Meanwhile, to prepare the test scenarios and test cases in advance, the testing team will coordinate the development and the Project management teams.

Step2: Scheduling

Once the requirement document is prepared successfully, it will hand over to the development team and the testing team. And the testing schedule should be writing with the project delivery plan, which should be accurate.

Step3: Deliverables

After receiving the document, they will check the deliverables. And the deliverables should well describ without having any uncertainty, and it should be fulfilling the scope of test objectives.

Step4: Implementation

After that, the implementation should be done as per plan and deliverables.

Generally, the 15-25% of the application will be affected directly by the modified requirement in a release. And the other 60-75% of the release will be depend on the out-box-features such as testing the applications and processes.

Therefore, we need to test the Mainframe application in two ways:

• Testing Requirements
• Testing Integration

Testing Requirements: We will test the application for the features or the modification disclose in the requirement document.

Testing Integration: Regression Testing is the main attention of this testing activity. And we will test the entire process or other applications that receive or send data to the precious application.

Step5: Reporting

After that, we will share the test results with the development team on a periodical basis. To maintain continuity, the testing team should be in touch with the development team for instant modification in crucial conditions.

Categorization of Manual Testing in Mainframe

Mainframe Manual Testing can be categorized into the following two types:

• Online testing
• Batch job testing

Online Testing

• Online Testing refers to the testing of CICS screens that is equivalent to web page testing where the functionality of the existing screens can alter, or new screens can be added.
• Various applications can have inquiry and modification screens where screens’ functionality needs to be tested as part of the online testing.

Batch Job Testing

• Generally, in batch job testing, the testing process contains batch jobs for the functionality, which is executed in the existing release.
• The test result will extract from the output files, and the database should be tested and documented.

Mainframe Testing commands

In mainframe testing, we have used various commands, which are very helpful while testing an application or software.

Some of the most commonly used commands in mainframe testing are shown in the following table:

Commands	Description
COPY	The COPY command is used to copy a dataset.
SUBMIT	The SUBMIT command is used to submit a background job.
RENAME	The RENAME command is used to rename the dataset.
DELETE	The DELETE command is used to remove the dataset.
JOB SCAN	The JOB SCAN command is used to fix the JCL with the libraries, program, file, and so on without implementing it.
CANCEL	The CANCEL command is used to cancel the background job.
ALLOCATE	The ALLOCATE command is used to allocate a dataset.

Note: We have some other commands while performing mainframe testing, but they are not used regularly; therefore, we used those commands whenever it is needed.

Prerequisite statistics on Mainframe operations

A tester should have the following information on mainframe operation while performing the mainframe testing on an application:

• ISPF [Interactive system productivity facility] has to be used for menu operations.
• TSO [Time-sharing option] is a method that is used to access virtual storage and manage datasets with the help of commands.
• The FTP [file transfer protocol] or another transfer protocol must be used while testing software.
• If the mainframe is the backend, we should use the cross-platform operations.
• SDSF [System display and search facility] has to be used for controlling operations and system resources.
• Batch job management has to be followed.
• CICS transaction has to use for testing the IBM mainframes.

Mainframe testing Challenges and Troubleshooting

If we are performing the mainframe testing on the application, we might have encountered some challenges, and for those challenges, we have some specified approach, as we can see in the below table:

Challenges	Explanation	Approach
Unclear or Incomplete Requirements	Sometimes a user may have access to the user manual or the training guide, but which are not similar to the documented requirements.	The Test engineer should be actively part of the Software Development Life Cycle (SDLC) from the requirements phase to overcome the unclear requirement issue.And if the requirements are testable, then it will help them to validate easily.
Identification or Data Setup	Sometimes, the tester might have a condition to re-use the current data as per the requirement. But occasionally, it is hard to find the mandatory details from the current data.	To overcome the data setup challenges, we can use the homegrown tools as per the requirement.And to get the existing data, queries should make in advance.In case of any trouble, a request place to the data management team to generate or replicating the vital data.
Job Setup	The job needs to be setup in the QA region when the jobs are saved into PDS. Therefore, the jobs are not submitted with a production qualifier.	For this, the Job set-up tools can be used to overcome human errors during setup.
Ad-hoc Request	Sometimes, we may encounter some situations where the end to end testing needs to be maintained due to a problem in upstream or downstream application. And the Ad-hoc requests enhance the time and effort in the execution cycle.	To overcome the particular challenges, we can take help with some regression scripts, automation scripts, and skeleton scripts, which decrease the time and effort overhead.
On-Time Releases for scope alteration	Sometimes, we have the condition where the code effect may entirely modify the look and feel of the system. And the modification could be in the test cases, scripts, and data.	For this, the impact analysis and the scope change management process should be in placed properly.

Steps to follow in Mainframe Testing

The below steps need to be followed while performing the mainframe testing:

Step1: Smoke Testing

In the first step, we will be performed smoke testing, where we check whether the code installed is in the correct test environment. And it also makes sure that there is no critical issues with the code, which saves the effort of testers’ time in testing a faulty build.

Step2: Functionality Testing/ System testing

After performing the smoke testing, we will do one round of functionality or system testing to test various models’ functionalities independently and concerning each other.

Following are the types of testing which has to done while implementing the System Testing.

• Batch testing
• online testing
• Online-batch integration testing
• Database testing
• Batch Testing

We will perform the batch testing to authenticate the test result on the output files and data modification completed by the batch job with the testing specification.

Note: Batch jobs are a set of events that got implemented without any user interaction with accessible computing resources.

• Online Testing

In online testing, we will test the front-end features of the mainframe applications. The online testing covers various aspects such as user-friendliness, data input validations, look & feel, navigations within the screen etc.

The application should be tested for exact entry fields such as interest on the plan, an insurance plan, etc.

• Online-Batch Integration Testing

The online-batch integration testing can be performed on the systems with the batch processes and online application. And here, we also tested the Integration features of the online process with the backend process.

Basically, in this testing, we validate the data flow’s accuracy and the interactions between the screens and the backend system. And the batch job is used to check the data flow and communication between the online screens.

• Database Testing

The database testing is used to test the data stored by the transactions for conformance with the system specification. And the databases validated their layout and the data storage, which contains the data from the mainframe application such as IMS, IDMS, DB2, VSAM/ISAM, Sequential datasets, GDGs are

And in database testing, we can also authenticate the data integrity and other database parameters for their ideal performance.

Step3: System Integration Testing

The system integration testing is used to check the systems’ functionality relating to the system under test. It is executed after unit-level tests because it is important to test the interface and serval types of messages such as Job Successful, Job Failed, Database updated, etc.

And for correctness, we will test the data flow across the modules and applications. The system integration testing is performed to make sure the readiness of the build for deployment.

In system integration testing, we can perform the below testing:

• Batch Testing
• Online Testing
• Online -Batch Integration Testing

Step4: Regression Testing

The most important phase of any testing is regression testing. The regression testing is making sure that the batch jobs and the online screens cannot directly relate to the system under test, which are not affected by the current project release.

The regression testing guarantees the modification done in a module and does not affect the parent application’s and the overall function along with the integrated application.

A specific set of test cases should be accepted based on their complexity, and a Test cases repository should be created to get a successful regression testing. And the particular test should be modified whenever there is a new functionality moved into the release.

Step5: Performance Testing

In mainframe testing, the next step is performance testing. In performance testing, we will try to find the bottlenecks in important areas such as front-end data, upgrading online databases, and project the application’s scalability.

In the Mainframe applications, we may encounter the below performance bugs:

• The online response time could be slow, which leads to user disappointment.
• Sometimes the Batch jobs and backend process takes extra time, which cuts into system accessibility to the online users.
• Scalability issues.

To overcome the above bugs, we should test the application appropriately with the help of the following:

• System integration parameters
• Application and Database design
• Coding
• System and database parameters
• Timing of back end jobs

Step6: Security Testing

The security testing includes evaluating the threats, risks, vulnerabilities and recommend remedial actions for applications and networks.

The security testing should cover use cases in identity and access management, Risk & Compliance Management, Data protection & privacy policy adherence.

In other words, we can say that security testing is performed to check how well the application is designed and developed to conflicting anti-security attacks.

Security testing should be completed on the two types of security systems: Mainframe security and Network security.

In security testing, we need to test the following aspects:

• Authorization
• Integrity
• Authentication
• Confidentiality
• Availability

Step7: Agile Methodologies

The Agile methodology is used to simplifies the gradual development of application and responds to modification quickly.

Note: In an agile development scenario, we can use the Incremental Testing approach.

Mainframe Automation Testing Tools

We have various types of mainframe automation testing tools available in the market. Some of the most commonly used mainframe automation testing tools are as follows:

• QTP
• REXX

QTP [Quick Test Professional]

QTP tool is used to test functional regression test cases of the web-based application. QTP stands for Quick Test Professional, and now it is known as Micro Focus UFT [Unified Functional Testing].

It is very helpful for the new test engineer because they can understand this tool quickly. QTP is designed on the scripting language like VB script to automate the application.

Features of QTP

Following are the most common features of QTP:

• In this tool, we can perform BPT [Business process testing].
• QTP uses the scripting language to deploy the objects, and for analysis purposes, it provides test reporting.
• Both technical and non-technical tester can use QTP.
• QTP supports multiple software development environments like Oracle, SAP, JAVA,
• We can test both desktop and web-based applications with the help of QTP.

REXX [Restructured Extended Executor]

It is an interpreted programming language, which is established at IBM. REXX is a high-level, structured programming language designed for reading and learning. REXX stands for Restructured Extended Executor.

It is used as a scripting and macro language. In other words, we can say that REXX is frequently used for processing text, data, and generating reports.

It supports various operating systems such as MVS, OS/2, AmigaOS, VM. And REXX can also be used as an internal macro language in some other software, for example, KEDIT, ZOC terminal emulator, SPFPC, etc.

Features of REXX

Following are the most common features of REXX:

• It supports case-insensitive tokens, which contain the variable names.
• By using REXX, we can access system services and commands easily.
• The REXX can support various procedures, functions, and commands which are related to a particular situation.
• It supports basic input/output advantages.
• It supports dynamic data typing with no declarations.

Advantage of performing the Mainframe testing

The advantages of performing the mainframe testing process will help us in the following aspects:

• It utilizes the resources optimally.
• It helps to evade the redundant rework.
• It enhances the user experience.
• It decreases the production downtime.
• It helps us to expands customer retention.
• And it also helps us to reduces the overall cost of IT operation.

Overview

In the mainframe testing section, we have learned the following topics:

• To test the application efficiently, the test engineer should participate in design meetings scheduled by the business and development teams.
• Mainframe testing is like any other testing process starting from Requirement gathering, test design, test execution and result reporting.
• We have understood the mainframe attributes such as Multiprogramming, Time sharing, Virtual storage, Spooling, and Batch processing.
• We have understood that mainframe testing classification is divided into two parts as Online testing and batch testing. The online and batch testing can implement effectively without missing any functionality mention in the Requirement specification document.
• We have understood the various mainframe testing Challenges and approaches.
• We have learned the most commonly used command of mainframe testing.
• We have understood the several mainframe automation testing tools, which help us enhance the process of mainframe testing.