ISTQB – CTFL Certified Tester Foundation Level – Testing Throughout The Software Life Cycle Part 2

7. Testing Levels : Integration Testing [CC]

Now that each unit has been proven to be working correctly, individually and ready to go, the next level would be to put those units together to build the system. This is what we call integration. At this stage, testers concentrate slowly on the integration itself. For example, if they are integrating module A with module, they are interested in testing the communication between those modules, not the functionality of the individual module as that was done already during component testing.

Thus, the test basis for integration testing can include the software and system design, a diagram of the test architecture workflows and use cases. The test objects would essentially be the interface code between those modules. This can include subsystem database implementations. Before integration testing can be blended, an integration strategy is required.

This involves making decisions on how the system will be put together. Barrier to Testing there are two major types of integration big bang integration and incremental integration. In big bang integration we integrate a bunch of units together at once, resulting in a complete system. The problem with this kind of integration that it looks like building the system faster, but in real life it’s much, much slower and time consuming.

Either because we would have to wait till we have a bunch of units ready to integrate or for the fact that when testing of the system is conducted, it’s difficult to isolate any errors found. We have an error caused when few units have been added, so each unit caused the error. It’s hard to know. In incremental integration, we overcome the major issue of big bank integration by adding units for integration one by one.

There are different kinds of incremental integration top down integration, bottom up integration and functional integration. To understand incremental integration, let’s imagine we have a system where we have menus for Students, employees and teachers. And under student there are add student functionality and under employees. There is add employee functionality and calculate salary. And under teacher, there is add teacher functionality. Also at a deeper level, under Calculate Salary there is the Calculate Bonus functionality.

So Components mean Cold Component in Brie and Component embry coles Component calculate Salary and Component calculate Salary coles Component Calculate Bonus and so on. Let’s look at the different incremental integration approaches one by one. First, top down Integration this is where the system is built in stages, starting with components at the top level, then going to lower levels till we finish the whole system. In our example, the sequence of building the components will be as follows from top to bottom.

Remember, when each component is built, it will go first through the component testing stage. So we assume that each component is working very well by itself at this point. So the order of integration will be one and two, then one and three, then one and four, then two and five, then three and six, then three and seven, then four and eight, then last seven and nine. If we assume the situation where we have just built combine seven top down integration testing requires that all the interactions with combination seven Calculate Salary are tested. But combiner nine is not ready yet. Still, we want to test the combination seven. Then what we need to do here yes. Correct. We will create a stub instead of components nine calculate bonus the second approach of incremental integration is bottom up integration. This is the opposite of top down integration and the components are integrated in a bottom up order. In our example, the sequence of building the components will be as follows from bottom to top nine and seven, then two and five, then six and three, then seven and three, then eight and four, then two and one, then three and one, and finally four and one.

If we again assume the situation where we have just built component seven, bottom up integration testing requires that all the interactions with component seven Calculate Salary are tested. But again, component three Emblem, is not 3D yet. Still, we want to test component seven. Again, what we should do here correct. Create a driver instead of component three. Employees functional integration is similar. The sequence of creation will depend on building and finishing a full functionality before moving on to the next. In our example, this means that we finish the student’s functionality first, then the whole employee functionality, then the teacher’s functionality again, stops on drivers may be used for testing. There may be more than one level of integration testing.

For example, component integration testing focuses on the interactions between software components is done after component testing. This type of integration testing is usually carried out by developer. The second type of integration is system integration testing, which focuses on the interactions between different systems and may be done after system testing of each individual system. For example, a trading system in an investment bank will interact with the stock exchange to give the latest prices for its stocks and shares on the international market. This type of integration testing is usually carried out by testers. Ideally, testers should understand the architecture and influence integration planning. If integration tests are planned before components or systems are built, those components can be built in the order required for most efficient testing.

8. Testing Levels : System Testing [CC]

System Testing now that we know that all units are working together, the next step is to consider the functionality of the whole system from end to end perspective. This activity is called system testing. In system testing. The test environment, which is the hardware and software used as an environment for testing, should correspond to the final target or production environment as much as possible in order to minimize the risk of environment specific failures not being found in testing. System testing is considered with testing all the scenarios that the system might go through. Thinking of all the possible scenarios is tricky and needs good understanding of the system domain.

Most often, it’s carried out by specialist testers that form a dedicated and sometimes independent test team. Within development, report into the development manager or project manager or quality manager. System testing may include tests based on risks and or on requirement specification. Business processes use cases or other high level text descriptions or models of system behavior, interactions with the operator system and system resources. The test object will generally be the system under test.

9. Testing Levels : Acceptance Testing [CC]

The final level of testing is acceptance testing. Acceptance testing is simply a form of yes or no testing. Should we accept the software? Yes or no? So acceptance testing should answer the question can the system be released and deployed? Acceptance testing is often the responsibility of the customer or users of the system. The goal in acceptance testing is to establish confidence in the system. Finding defects should not be the main focus in acceptance testing. So remember that acceptance testing objective is acceptance yes or no, whereas the component integration and system testing objectives is to find defects. Therefore, test levels have different objectives. Remember that acceptance testing is a yes or no kind of testing. So any way we would test something to get a yes or no answer, then it’s a form of acceptance testing. Testing a commercial off the shelf or in other words COTS software product before purchasing, you test it to say yes or no to purchase it or not. Testing a component during component testing to decide if we should continue with it or not. Again, it’s a yes or no. So it’s some sort of acceptance testing test.

Basis for acceptance testing includes the user requirements, the system requirements, the use cases, the business processes and any risk analysis. Reboots test objects for acceptance testing include the business processes on fully integrated system, the operational and maintenance processes, the user procedures, forms and reboots and any configuration data. There are different forms of acceptance testing which include the following user acceptance testing, operational acceptance testing, contract and regulation acceptance testing and finally alpha and beta acceptance testing. Let’s look at each one of those in detail. First, user Acceptance Testing in user acceptance testing will verify the fitness of for use of the system by business users.

This can be either at the developer site, which might be called factory acceptance testing or at the customer site which might be called site acceptance testing. In this case, customer can perform any test they wish, usually based on their business processes. It can happen at any time during the project. So yes, you can demo the software to the customer in the middle of the project and they can decide not to continue. So user acceptance testing can happen at any time. But usually it also happened as the final stage before users sign off. The second type of acceptance testing is operational acceptance testing.

This is where we test the system for acceptance from the system Administrators board interview. It validates whether the system meets the requirement for operation and maintenance. The operational acceptance test may include testing of backup data, restore data, disaster recovery, maintenance tasks and beyond, a check for security and so on. Contract and Regulation Acceptance Testing in contract acceptance testing, sometimes the criteria for accepting the system are documented in a contract. If the customer needs to buy COTS software component of the shelf but he wants to add a minor requirement, say he wants to add the company name in the first displatch screen of the software.

Then I guess you will agree with me that we don’t need to change the whole requirement document for this and go through the software lifecycle again. Such a minor change could be documented simply in the contract and that’s it. Testing is then conducted to check that these criteria have been met before the system is accepted in regulation acceptance testing. In some industries, systems must meet governmental, legal or safety standards. Examples of these are defense and banking industries. For example, if a software company wants to add the ISO logo to its software, then it should follow ISO guidelines in developing the software.

Testing is then is conducted to test whether we should pass the ISO guidelines or not. Again yes or no. Last alpha and beta testing some systems are developed for the mass market. For example commercial off the shelf software where there is no actual customer who can provide you with his requirements. But rather the marketing team for example of the company suggested some requirements for you and built the system based on those suggestions. Still, you want to get some feedback of how the user who will buy the system feel when using the system before actually boating the system for sale. Very often this type of system undergoes two stages of acceptance test.

Alpha testing takes place as a developer site. We invite some potential users to our site using our machines and let them play with the software for a while and get feedback from them. Beta testing takes place as a customer site. We send the software to some potential users and ask them to play with the software using their machines and ask them to send us feedback when they are done. So actually, anytime you are downloading a beta software, you are acting as a beta test. So which is better from your point of view, alpha or beta testing? We can discuss that as the discussion board. Thank you.

10. Test Types [CC]

We learned that each test level has specific testing objectives. In this lecture we will look at the types of testing required to meet these objectives. We need to think about different types of testing because testing the functionality of the component or system alone may not be sufficient to meet the overall testing objectives. The different test types include functional testing, non functional testing, structural testing and testing related to change. Let’s look at each one of them in detail. First functional Testing the function of the system or component is what it does. The function of the system is typically described in the requirement specification, a functional specification or in use cases. Some functions are assumed and not necessarily required by the customer, such as copy and paste, for example, even if the customer didn’t ask for it explicitly. Functional tests are based on these functions described in the documents or understood by the testers.

Notice that a type of functional testing is security testing. This is confusing to many of us. Security testing in Isttb is a type of functional testing which investigates the functions relating to detections of threats such as viruses. Another type of functional testing is interoperability testing, which evaluates the capability of the software product to interact with one or more specific components or systems. The techniques used for functional testing are often specification based to derive test conditions and test cases from the functionality of the software or system. But other techniques can also be used. We will be talking about those techniques more in the Design Test Cases Techniques section. Second Non Functional Testing Imagine an accounting system where every functionality works perfectly well, but it’s very slow. You might need hours to print a report. Such a system is still not usable even though it performs its functionality quite well.

That’s why we need the second type of testing, which is testing the quality characteristic of the software or what we call non functional testing. Here we are interested in how the system does its functionality. The functional Attribute what it does might be print a report. The nonfunctional attribute is how it does print the report. There are many types of nonfunctional testing which includes, but not limited to performance testing. How many users can connect to the system and how will that affect the performance of the software? Low Testing how will the system perform if we do a single transaction so many times? Stress Testing how will the system perform under very tough circumstances? Many users, many transactions, low memory and so on. Usability Testing is the system easy to use? Mental ability Testing is the system easy to maintain if we need to fix the defect? Reliability Testing is the system reliable or does it crash eventually? Portability Testing is the system easy to boat from one platform to another? Non functional requirements can be stated in the requirement document as a direct customer request. For example, the system must support 1000 users simultaneously or as an industry standard. For example, the web page response time should be less than 6 seconds. We will talk more about some nonfunctional attributes in the tool sections of this cross. Structural Testing structured testing is also known as white box testing, which we will talk about more deeply in the Design Test Case Techniques section. The structured testing simply means that we use any information about how the software is constructed to test the software. So for example, if you know the components of the system’s database of how a specific function is implemented, then you can use this to your advantage and use structured testing. The most famous kind of structured testing is when you are having the source code written by the developer.

Specification based testing cannot guarantee that we have exercised every line of code in the system under test. Therefore, we use a structural testing, usually after a specification based testing, to measure how much of the code has been covered by our test execution efforts. So the main idea of a structural testing is to measure the coverage. Coverage is expressed as a percentage of the items being covered. If coverage is not 100%, then more tests may be designed to test those items that were made to increase the coverage. Confirmation and Regression Testing first confirmation testing, or sometimes it’s called retesting. When the test fails and we determine that the cause of the failure is a software defect, the defect is reported to the developer and we can expect a new version of the software that has had the defect fixed. In this case, we will need to execute the test case again to confirm that the defect has indeed been fixed.

This is known as confirmation testing or retesting. When doing confirmation testing, it’s important to ensure that the test is executed in exactly the same way as it was the first time using the same inputs, data and environment. If the test now passes, does this mean that the software is now correct? Well, we know now that at least one part of the software is correct where the defect was. However, this is not enough. The effects may have intuit, used or uncovered a different effect elsewhere in the software. The way to detect disease unexpected side effects of the fixes is to do regression testing. The word regress is the opposite of the word progress. Progress is moving forward, so regress is moving backward. In regression testing we move backward and test areas that we have already worked fine to make sure it’s still working fine after any kind of a change to the software.

11. Test Types vs Test Levels [CC]

One thing we need to talk about is the relationship between test levels and test types. What kind of test types can be performed at each test level? Let’s look at them one by one. In component testing, of course, we test the functionality of the component and sometimes we can perform non functional testing like the component’s performance, its mountability and so on. The developer uses structural testing to go through the code and if he fixes a bug, he will have to do retesting and nutrition testing to the change of code. So all the four types of testing can be applied at the component test level. In integration testing, testing of specific non functional characteristics such as performance may be included in integration testing as well as functional testing.

Knowing how the modules interact is true a sort of structured testing. And again, if a bug is fixed we will have to do retesting and navigation testing to do the change of code in system testing, that’s for sure. Where every test is applied, you may ask structured testing too?

The answer is yes for sure. Knowing the menu structure and the database design of the system would sure help in a structural testing last acceptance Testing well, believe it or not, we can do all four types of testing as acceptance testing as well. And again, some customers can do structural testing too. Think of the It department of any institution nowadays. They are the people who you have to deal with when you deliver a new system to their company. And sure, they have some knowledge of programming to do. White box Testing so we can do all the four types of testing at the four levels of testing. Why you didn’t say so from the start instead of wasting our time? You may ask. Well, because it’s written like this in the ICTB curriculum and I have to be honest delivering the information to you. You can thank me now. Thank you. Thank you.

12. Maintenance Testing [CC]

Once a software is delivered and deployed, it could be in operation for years. During this period it may become necessary to change the system and after any change we must test the system to make sure everything is still working fine. Testing that takes place on a system which is in operation in a live environment is called maintenance testing. Maintenance testing is done on an existing operational system and is triggered by different causes. It could be triggered by modifications which include blend enhancement, changes corrective and emergency changes, and the changes of the environment such as blend operating system or database upgrades.

Blend upgrade of commercial off the shelf software or patches to the correct newly exposed or discovered security problems in an operating system. Maintenance testing could be triggered for migration which is moving the system from one platform to another should include operational tests of the new environment as well as of the changed software. Migration testing or conversion testing is also needed when data from another application will be migrated into the system being maintained. Maintenance testing for the retirement of the system may include the testing of data migration or archiving if long data retention periods are required. So maintenance testing is triggered by modifications or migration or retirement.

Maintenance testing is a kind of art by itself in my point of view. Unlike regular testing, there are many aspects we need to be careful about when we do maintenance testing as we are testing on a live environment. Imagine if your system is a banking system with tons of data about bank clients and transactions I cannot test. For example, does delete all records is still working or not? Or can I test transferring $1 million to my account? That would be nice. So in addition to testing what has been changed, which is retesting, maintenance testing includes extensive regression testing to bars of the system that haven’t been changed. But a lot of questions could arise. What could this change have an impact on? How important is a fault in an infected area? Should we test what has been affected? If yes, how much? What are the most important affected areas or areas most likely to be affected, or the whole system? The answer is it depends.

The scope of maintenance testing is related to the risk of the change, the size of the existing system, and the size of the change. Determining how the existing system may be affected by changes is called impact analysis and is used to help decide how much regression testing to do. The impact analysis may be used to determine the regression test suite. One point about maintenance testing is that it can be difficult if a specification are out of date or missing or testers with domain knowledge are not available. So what would you do if you face the situation yourself? Well, you should consider what the system is currently doing, for example, examining existing system and look in user manuals or guides and ask the experts the current user. Okay, now the thumb up. What kind of questions can we see in the exam related to this section? Well, we have learned many terminologies in this section and we need to know what those terminologies mean and who do what.

The Vmodel is a sequential software model where we can do early blending and designing of testing. Iterative incremental models like Crad, Agile and Rub are where we need to use automation testing, component testing, integration testing, system testing and acceptance testing are test levels, functional, nonfunctional and structure V testing. Regression testing are types of testing. Maintenance testing is done to a live environment, alpha testing is done at the developer site, beta testing is done at the user site and both are kinds of acceptance testing.

A question might be what kind of testing can be done by a developer from all the terminologies that we have just heard? Well let’s see component testing, integration testing, functional testing, nonfunctional testing and for shows structural testing plus suite testing and regression testing can be done by the developer. Another question what kind of testing can be done by the customer? Acceptance testing for sure and the four types of testing. Well have fun and feel free to ask me anything in the discussion board on the right if you have any questions. Thank.

• Category: Uncategorized