Test Plan vs Test Strategy

As a software tester, you may find yourself tasked with writing two documents that sound very similar: test strategy and test plan.

Both of these are important documents that software testers prepare before beginning a project, but each serves a different purpose. And knowing the difference can be useful so that you can understand how a client's testing project will be carried out. Writing a test plan and test strategy is an important skill that every software tester should master—and knowing the difference between test plan vs test strategy documents can come in handy when you're interviewing at a software testing firm.

Here's everything you need to know about the two documents.

A test plan is defined as a document created by the software testing firm that outlines the various actions to be executed during the testing process. Test plans answer the "how" questions about a software testing project, such as:

• How are we going to test the software?
• How will the testing occur?
• How many cycles of testing will there be?
• How will the testing be coordinated?

You can think of a test plan as an outline of your software testing project, as it describes the assets, approach, and the time table for each portion of the testing. It will detail the features to be tested, the test strategies and techniques that the software testing team will use, and more.

Test plans are fluid in nature and can vary from one software product to another. They're important because they help software testing firms and their clients stay on the same page in regards to the type of work that will be carried out.

Test plans are usually written by test managers or test leads based on input by team members. They contain several fields to be filled out, such as the following:

• Introduction: a brief introduction of the software application to be tested
• Purpose and scope: the overall purpose and scope of the testing
• Testing strategy: the testing approach or approaches that will be used
• Features to be tested: describes what specific features of the software will be tested
• Features not to be tested: exactly the opposite; lists the features of the software that will not be tested
• Test deliverables: the different types of test findings to be reported such as test cases and test results
• Environmental needs: any environment-specific need for the project
• Responsibilities: lists the team members and other resources involved in the testing project and their responsibilities
• Training: the training requirements of specific resources
• Tools: list of tools used if applicable

To summarize, a test plan goes into great detail about the testing project and how it will be carried out.

A test strategy is a more high-level document that outlines the project in broader terms, usually including a set of instructions or guidelines about how the testing will be carried out. Where the test plan goes into a lot of detail about a project, the test strategy gives a 50,000 foot view of it. Yet it is needed to clearly define the exact software testing approach and testing objectives of the software application.

The test strategy addresses the "what" questions surrounding the testing project, such as:

• What are the high level test objectives? Or, what is it we're trying to accomplish?
• What types of testing will we do such as functional, aggression, etc.?
• What are some of the risks and mitigations to those risks?
• What is the defining point in which testing is done on the project, or how we will recognize when testing can be considered complete?
• What are the things that we are going to test?

Your test strategy has to be one single document, whereas there may be multiple test plan documents to support that strategy.

It is tailored to your audience - stakeholders and senior management team you need approval from. And unlike a test plan, which can be modified during the course of a project, the test strategy is a static document that doesn't change.

There are usually four components or sections of a test strategy:

1. The scope; what are the things within the scope of testing and the things not in the scope

2. Test approach - what types of testing will be conducted

3. Environments - where are we going to be doing our testing?

4. Key deliverables - your clients want to know what are they paying for

• Introduction – A brief introduction to the purpose and scope of the document
• Standards to use – The different standards or set of guidelines to be followed
• Risks and Mitigations – The different risks associated with testing and their mitigation strategies
• Entry Criteria – The pre-requisites that must be performed before testing can start
• Exit Criteria – The criteria defining when the testing can be stopped
• Test design techniques – The test design techniques to be used like – equivalence partitioning, boundary value analysis, etc.
• Test environment – This defines which hardware/3rd party is needed to fulfill the requirements, of where the product is going to be tested
• Configuration management of testware – Specification of the right version of testware for testing
• Test process improvement – The approaches to use for improving the test process
• Approvals – The persons approving the test strategy document

Still confused? It can be useful to see the differences between a test plan and test strategy document at a glance:

Test plan: Specific to a particular project

Test strategy: Usually for a complete organization

Test plan: Prepared by the test lead of test manager

Test strategy: Usually prepared by the project manager

Test plan: Describes the testing activities in detail such as the techniques used, schedule, and resources

Test strategy: Describes the high-level test design techniques to be used such as environment specifics

Test plan: outlines features to be tested, components not to be tested, approach to testing, pass-fail criteria, and more

Test strategy: includes scope, test approach, testing tools, test environment specs, and more

Test plan: Is a fluid document that can be changed or updated

Test strategy: Static document that is not modified

Many people get these two documents confused, but it's important to know the difference between a test plan and a test strategy because both set expectations for the testing team and the client. With practice, you can become a pro at writing both and knowing what information each one covers, boosting your confidence when you outline your testing plan to clients.

A. It doesn't matter which of these documents you write first; what matters is that each one is uniques and tailored for the project and client at hand. You don't want to copy a test plan and test strategy from one client to another.

You can think of a test strategy as helping to plan the test procedure while the test plan is then used to carry out the testing process.

A. Knowing the difference between test plan and test strategy documents helps set expectations for your team and your clients before each software testing project.

We are often asked what software testing is. The video below shares a solid definition of the term.

But we thought a software testing project case study might be helpful to better understand what software testers do on a typical day. This includes testing software, writing requirement documents for our clients, and creating user guides to ensure compliance for our clients to use for quality assurance and auditing purposes.

Iterators LLC was hired to complete accessibility testing for a few projects for the Library of Congress (LOC).

Accessibility testing is required on all government websites, with Section 508 and WCAG 2.2 requirements used. To become a Trusted Tester an employee must complete the DHS Trusted Tester online training course requirements and pass the DHS Section 508 Trusted Tester Certification Exam of Homeland Security in Accessibility, and we are in a unique position to help on this project.

We cross-train all our employees so that we can work on several projects at one time or several different aspects of a project to complete the work and reduce the cost to our clients.

Our first project assigned by LOC was testing their new braille feature on BARD Mobile for Android. We were tasked with testing the braille feature with several refreshable braille displays.

During our testing we used the Orbit Reader 20, and two different braille displays from Freedom Focus 14 and Freedom Focus 40. There are plans to use other refreshable displays such as Humanware, but this testing has not occurred yet. We needed to test refreshable braille displays and their use in tandem with Google BrailleBack and Google TalkBack.

This work was to ensure that all hardware worked as expected with the apps we were testing. For this testing, we had to complete functional testing, smoke testing, exploratory testing and had a user panel to ensure we caught all issues that a visually impaired individual might experience while using the app.

Initially, our client was unsure if we would find any bugs and hesitant to have us enter bugs into Bugzilla as they stated the software was “complicated”. Bugzilla is a web-based general-purpose bug tracking system and not unlike other bug tracking systems we use every day such as Jira, TestRails, PractiTest, and Click-Up.

Testing was completed over several agile sprints with many significant software testing bugs found. Our testing had us test against the National Library Service requirements document. Next, we had to create an up-to-date user manual. While the manual had been updated several times, the testing had not been.

For example, when downloading a book or magazine from the Now Reading section of the mobile app, the download would end up at the bottom of the page. For years, the user guide had listed the download being at the top of the page once the document was downloaded.

Our testing team, on several occasions, said this was an error in the documentation and that the download ends up at the bottom of the page. This was corrected in the user document and sent to the development team to fix per the requirement document.

Over the next several months, we reported 30 high priority bugs with about half fixed at this point. We have encouraged our client to test in an agile fashion because once the development team is finished, it’s harder to get these bugs fixed.

Our bugs were reported and based on the requirement document used to create the software. Lastly, the user guide had to be rewritten to reflect the app's behavior and general updates.

Once the app was tested and created, the user guide was sent to Communication Services to ensure the style used per other requirement documentation. This document had to be approved before being disseminated to the public. For example, how does the library determine what the Most Popular Books are and over what period.

Once the document was returned from COS, this PDF document had to be remediated. The process of remediation is to make a PDF, create the heading for the document, alt text added to meaningful images, and decorative images either ignored or taken out of the digital document altogether.

Once the remediation process is complete and validated, the document becomes ADA compliant. We then provide an accessible PDF that can be read with the use of a screen reader and create the HTML output so that the document can be added to the Library of Congress website.

You can fine the current user guide we completed here: https://www.loc.gov/nls/braille-audio-reading-materials/bard-access/bard-mobile-android/#creatingfolders3.3

Iterators LLC has been named a Top Certified Women-owned Business, again in 2022. (https://IteratorsTesting.com) and was built to ensure that companies do not settle for shoddy software that has not been properly tested. Iterators works on an array of software testing services for small start-ups, Enterprise software and works for local, state, and federal government.

Our team is thrilled to announce that we have recently been named a Clutch top certified women-owned business for 2022.

Clutch is an online review and rating platform that primarily serves the B2B industry. They spotlight leading B2B firms that might not otherwise get it with their awards and certification programs. Our team invests so much time and work into ensuring that our clients get the best possible results from their projects. Knowing that we have not only succeeded in those endeavors, but we did enough for them to award us for it also means more to us than words can express.

We express our gratitude to our clients in Boston (our home base) and beyond, without whom this award would not be possible. Our team recognizes that it takes time and effort to be able to provide enough support to make this accolade possible. They did so on our behalf is something we truly appreciate.

Discover what an award-winning service feels like by visiting our website and contacting our team today. Let’s make sure your software is the best it can be together.

Is My Website ADA Compliant?

If you are asking this question, then your website is most likely not ADA compliant. It does take some design and care to make a site ADA compliant using the WCAG requirements.

Some items are fairly well known, such as including alt-text for any images on your website. However, other requirements are fairly specific so that your site works well with screen readers or other assistive technologies.

Also, recall that screen reader compatibility is not the only criteria to make your site ADA compliant as there are a range of disabilities that your users encounter.

For example, what if you are right-handed and your right arm is in a sling or a cast and you start using the keyboard to navigate your website? Can you perform all activities on your website if you only use a keyboard?

Note that making your website accessible can be done affordably. See the other topics in this blog for more information on next steps for you to consider.

Table of Contents:

- Is My Website ADA Compliant?
- What Is Accessibility Testing?
- How to Test Accessibility of a Website
- Accessibility Testing for Mobile Apps
- How Much Does Accessibility Testing Cost?
- What is a VPAT for Accessibility Testing?
- Why is accessibility testing required?
- Why is color contrast important for accessibility?
- How do you perform an accessibility test?
- What are accessibility testing tools?
- Frequently Asked Questions

Accessibility Testing, as related to software or a website, focuses on if the software/website meets the WCAG requirements and therefore users with a range of disabilities can access your site.

An analogy to consider is that some physical buildings are not accessible to wheelchair users if there are not ramps or elevators to allow them access to your building. Is your website like this?

For example, we were told that some students at the Perkins School for the Blind that their teachers must provide them with research for their projects because most sites on the internet are not accessible. What if your child was doing a report on Electric Cars, but none of the websites were accessible to them?

Your website should be accessible to people beyond your direct customer base.

In comparison, “Functional Testing” of software refers to if your website functions as expected. For example, if you click on “Search”, does it then perform a search. This feature might be functional, in that the feature works, but it may also be inaccessible and cannot be used by some users with disabilities. This requires “Accessibility Testing” to determine if the WCAG requirements are met.

Iterators LLC strongly recommends using a Trusted Tester for your accessibility testing. The Trusted Tester certification is a challenging program that is administered by the US Federal Government in the Department of Homeland Security to provide Section 508 and WCAG testing certification.

The principal tool used here is ANDI along with the training needed to perform the testing needed for all WCAG requirements.

Other tools are available on the internet, such as automated code checks or site reviews.

• Deque captures 57% of accessibility issues
• Testing automated accessibility tools

In particular, be careful with “widgets” that claim that they will make your website accessible. These can only check some accessibility issues. Some people use these as a perception that this will save money for your accessibility project. However, accessibility does not have to be a significant cost as described elsewhere in this blog.

In summary, the best approach for accessibility testing is to use a Trusted Tester that is familiar with all WCAG requirements and then use a blend of automated testing and manual testing in order to evaluate all of the requirements. This should include creating a VPAT (voluntary product accessibility template) to summarize the testing completed.

Mobile App accessibility testing is more challenging than testing a website but can still be done affordably as part of your software development project. The WCAG requirements apply to mobile apps, and indeed for all software, and not just to websites.

The four main principles of WCAG are: Perceivable, Operable, Understandable and Robust. These principles are crucial for a mobile app for disabled and non-disabled users. It is most economical to combine accessibility testing with manual functional testing as you develop your app.

An example of some of the automated testing tools for mobile apps can be found at this Perfecto blog. However, this article does not list the manual testing that is also required in order to ensure that your app is accessible.

One reason that manual testing is needed is that accessibility requires trained judgement in addition to making sure that all of the appropriate attributes are filled in. For example, are the accessibility labels meaningful? Can a user easily avoid incorrect actions or be informed if required information is needed? Did you consider more than vision impairments as the website was tested because there are a wide range of disabilities for potential users.

Accessibility testing for your website or mobile app can be affordable. The “Test Left” principle should be used meaning that you should include this testing as part of the ongoing project. Waiting until the very end of the software project can lead to more costly redesigns or reworking code to overcome any issues that are found.

At Iterators LLC, we find that it is useful to allocate 5 – 10% of the development budget for both Functional and Accessibility testing. As mentioned, the full cost is reduced if these issues are found as your developers progress through the initial development or a redesign.

A VPAT is a report that provides a scorecard regarding how well your website, app, or software meets the accessibility requirements. VPAT stands for Voluntary Product Accessibility Template and is administered by the Information Technology Industry Council. A VPAT is required for Section 508 compliance with the US federal government, or for EN 301 549 compliance for the EU (European Union), or for the WCAG guidelines.

An example VPAT can be found for the City of Boston website. Also, see other areas of this blog for how a Trusted Tester can complete the accessibility testing effectively at a reasonable cost to your project.

This depends on who you ask.

Asking a lawyer may indicate that you can be sued if your site is not accessible. Dominos Pizza tried to avoid accessibility by taking their case to the US Supreme Court and recently lost. More importantly, if you ask a disabled person, they would greatly appreciate if your website or app were accessible.

It is of course extremely frustrating to reach a website and then find that you cannot access it due to your disability. This would be like arriving at a movie theatre and finding that you could not get into the building or watch the movie.

See this website for some “humorous” examples of locations that are inaccessible to wheelchair users. Of course, these are only humorous if you are not the person inconvenienced.

We heard a story recently from the Perkins School for the Blind in which some students could not independently complete their homework because performing research on the internet typically brings up websites that are not accessible. These students may not be your direct customer, but would you think of building a building without accessibility features?

The City of Boston summarized their position in several blogs in which they upgraded their site with more WCAG accessibility features. Another excellent article is “I Used The Web For A Day With Just A Keyboard”. You can help everyone by applying the WCAG standards to your website.

Color Contrast is one of the WCAG guidelines for accessibility. It is one of the tests that has some clear guidelines, such as:

• Contrast of typical text should be a minimum of 4.5:1
• Contrast for large text can be a minimum of 3:1
• Contrast in company logos does not apply because you have hopefully designed your logo effectively.

These color contrast ratios can be evaluated with several tools. We use Color Contrast available on the macOS App Store. These tools work by manually finding the foreground and background of text and providing the color contrast and if it passes the WCAG requirements.

Color Contrast also must account for color blindness. The macOS has the ability to simulate different color blind conditions by adjusting the color scales. Activating this feature gives you the chance to see if your color scheme will still work well for all users.

A good place to start accessibility testing is to summarize the different templates that your website is using.

Your website may have dozens or hundreds of pages, but only a few different templates. For example, a Home Page, a Blog Page, a News Page may be set up with different templates. Each of these templates can be evaluated as described below.

Next, we perform an initial audit of each page before performing a full suite of tests. This reduces the cost by providing a quick triage of the page. If there are issues to fix, then this will identify them quickly without the need for extensive testing.

The following are several of the common areas for testing:

• Keyboard accessibility: This is our first test and it is easy to see if a website has tried to satisfy the WCAG requirements. Load the website and then start pressing the tab key on the keyboard. You should first see a Skip to Main Content link, which can be tested. Continuing to press the tab key should show you which item is currently selected. Also look for tabs that lead to invisible links.
• Image accessibility: Most people are familiar that images need alt-text. However, “decorative” images should have an empty string as alt-text (“”). The reason for this is that there is no value to a screen reader user in announcing alt-text such as “stock photo of a family entering a bank.” The rule is that you can delete a decorative image and the user would still have all of the information that they need from your website.
• Links: All websites have links, and are sometimes listed as “click here”, or “learn more.” It is important that each link has a descriptive name that a screen reader, or other assistive technology, can report to a user. You can still use links that say “Learn More”, but if you have multiple links, then you can provide an accessible name that is different from the visible name. This name is then used with a screen reader to provide a full description to the user.

The following are some accessibility testing tools that could be used by a general user. Note that some training is still needed in order to know which tests are needed for all of the WCAG requirements.

ANDI is a tool provided by the US federal government at no cost. The user interface is sparse, but it provides a large number of tools and information needed for accessibility testing.

For example, it will quickly list if you have several links that all have the same descriptive name, such as “Learn More.”

A11yTools is a Safari extension for macOS. The cost is reasonable at $12, and it is then used to highlight images, headings, forms, tables, and other elements that need to be examined.

For example, the images are highlighted and the alt-text is made visible so that you can quickly check if the alt-text is appropriate. A common mistake is that decorative images should actually have an empty string as alt-text.

WAVE is also a browser extension which works with Chrome or Firefox and also provides accessibility information for images, links and other elements.

What is your exposure to the market of services for the visually impaired? Are you familiar of other players who compete with Freedom Scientific's JAWS? (And how do they distinguish themselves)

• Freedom Scientific is undoubtedly the gold standard and an excellent product. We test with many refreshable braille displays from Freedom Scientific, Orbit, Humanware, etc. We also have employees who are legally blind and technology specialist.

When testing new products for the visually disabled, what do you try to be the most cognizant of.

• Some in the blind community believe in the pure blind experience. This means that only blind users should complete the testing. Everyone with a visual impairment is not blind. Testing different disabilities requires different strategies and, frankly, a diversity of thought that women possess because they've often been excluded in the technology field. We excel at our work by leaving no stone unturned and working with user panels to ensure we are testing with product users.
  
  Decades ago, most individuals with visual impairments were blind due in part to premature births and exposure to oxygen causing blindness. With advances, this is not true today. the leading cause of visual impairments today is CVI – Cortical Vision impairment, a disorder caused by damage to the parts of the brain that process vision. Times change and strategies must change too.

Are mobile applications prevalent or considered in the visually impairment space in your line of work?

• Yes, we consider apps in our work, but they are not mandated to be accessible, so business leaders need to make that a priority.

What would you recommend for new entrants into the software for the visually impaired to be cognizant of?

• Basically, it's good for people to champion their own area of expertise or interest but welcomes others working towards a common goal.

What is pdf remediation?

• PDF Remediation is the process of correcting accessibility errors that make it impossible for users with visual impairments and other disabilities to read a PDF document without the aid of remediation and a screen reader. Remediated documents are ADA compliant. Meaningful images have Alt-text added during the remediation process, and decorative images are removed.

Table of Contents

- What is Software Testing
- How Does Software Testing Work?
- What Is a Software Bug?
- Different Types of Software Testing
- Common Terms: Types of Testing
- Types of Software Testing
- Agile Software Testing Strategy
- Agile Software Testing Is an Important Part of the Development Cycle
- Do I Need Software Testers?
- What Kind of Training Do Software Testers Need?
- Software Testing Best Practices
- Software Testing Resources
- Software Testing Training Resources
- Software Training Books on Amazon
- Frequently Asked Questions

The Psychology of software testing is related to the idea of wanting to find software defects. It is much better for you to find a bug rather than your customers.

Our training program for software testing specifically teaches that it is good to find a bug, because new software testers are often disappointed when a defect is found. A good day for a software tester is to find a bug.

The Economics of software testing is related to how software testing can actually reduce the cost of software development. This can be counter-intuitive, but it makes sense when you remember that software testers are generally less expensive than developers. It can also be very costly to redesign software at the end of a project if a significant bug is found that requires significant development to fix.

This is related to the “Test Left” paradigm in which the software testing should occur as early in a timeline as possible, as discussed in the Agile Software Testing Strategy section.

Example: What is Software Testing?

A software bug is simply a difference between an expected result and what actually happens when the software is tested. The Expected Result can come from a list of software requirements, or more often it comes from the expected behavior for an app that follows best practices.

For example, it is best for buttons to be labeled consistently, respond quickly, give feedback that an action was taken, etc. If your app differs from a typical app, then users will find the software confusing.

Example Bug: The Mother and Father data entries were stored in the wrong fields

There are many types of software testing. For example, Unit Testing is normally completed by developers as they add Unit Tests to their own code so that a set of expected results are verified for a set of test cases.

A simple example would be to verify that a function that adds numbers would verify that 2+2=4. Technically, this is part of Test Driven Development (TDD), which is more of a development team activity than a Quality Assurance Software Testing activity.

Therefore, many people think of software testing as User Acceptance Testing (UAT) or Software Acceptance Testing (SAT). This is considered as “final” testing at the end of a software project to do a final validation or verification (V&V). This is an important activity, but it is better and more economical to follow an Agile Software Testing Strategy as discussed in a later section.

• Functional Testing is done with the goal of finding functional defects. This means that a certain software function is not working as expected. Some examples that Iterators has found are a “First Name” field stores a result as a “Last Name”, or clicking on a given button leads to a crash, or an inaccurate action.
• Regression Testing means that a previously working feature has regressed and is no longer working after a software update.
• Manual Testing means that a software tester is manually using the software by that individual. This manual testing could be Functional Testing, Regression Testing, or any of the other testing types listed below.
• Automated Testing means that the testing is performed by another computer program that automates the use of the software application. For example, Selenium can be used to program a set of tests for a website in order to execute the web app and compare the results to the specified requirements. Automated Testing is normally done as Automated Regression Testing because automation is generally more expensive to set up than Manual Testing, but the automation can then be run every day or after every software build and so the overall costs are reduced over time.
• Accessibility Testing is another major area in which the bug is related to how easy the software is to use for users of all abilities. For example, some users with disabilities would use a keyboard switch or a screen reader for a website or app. Software bugs can impact the accessibility of the software product for some users. WCAG provides a set of requirements for websites to be accessible. Iterators is a Trusted Tester certified by the Department of Homeland Security for accessibility testing.
• Integration Testing or End-To-End Testing refers to doing the software test once all of the components are all put together and is most similar to how an end-user would use the software product. Testing each individual component may not find all of the errors that occur when all components are put together into a complete system or app.

Agile refers to a project management strategy often applied to software development projects. For our purpose, a simplified way of thinking about this is to emphasize that each part of the software should be tested at the time they are developed rather than waiting until the end of the project. Good Agile testing can be lower cost and much lower risk than the “Waterfall” strategy of waiting until the end of a project to perform the testing.

For example, assume that a project takes 6 months to complete. A development sprint could be completed every 2 weeks by several developers. Perhaps one day of software testing is done for each of these 2 weeks. This ensures that the features are actually working well and there are no regression bugs.

The alternative is to wait until the end of the project, and spend perhaps 2 weeks testing, find bugs, and then need to spend another 2 weeks or so to fix those bugs.

That can delay the release of the software product by a month, or you may need to compromise the quality of the software to meet your deadline.

Some people mistakenly think that the Test Driven Development completed by the developers are sufficient to make sure that everything is working. This needs to be balanced with manual testing to perform Integration Testing to see how the software works with all of the components completed.

Short answer is “Usually.” Let’s go through a few scenarios to consider.

If you are creating a relatively simple website, then perhaps not.

If you are creating a responsive website suitable for desktop and mobile browsers perhaps with some animation elements, then yes.

But won’t your web developer test the site? Of course, but it is often beneficial to have trained software testers also test the site. At Iterators, we have a device farm with different mobile devices, we test different browsers, window sizes to test the responsive break points, check the animations.

We always find issues, such as images that do not display correctly on certain window sizes, or animations that are not compatible with some browsers. It is more economical to hire a well-qualified software tester than to expect your developer to spend extra time testing all of these configurations.

What if you are developing a mobile app? The answer is definitely Yes!!

In fact, it is best to test with the Agile strategy of testing new features every Sprint, which could be every week or two. Iterators has had a few customers that have had their apps fail when tested at the very end. One customer had a “guarantee” by the developer to fix all problems that were found in the last 3 weeks of testing. After multiple rounds of testing, the app was still not functional, and the app was abandoned. Reduce your risk and your overall costs by performing testing during development.

It is again more cost effective to have testers test and developers develop. The other approach is for the client to test the app, because “we all use apps.”

However, there is a great benefit to using trained software testers to find those examples that your end users will end up complaining about on the launch of your app.

At Iterators, we have a custom training program that incorporates several useful components. We include the ISTQB Foundation Level training for software testers. This has several valuable techniques for software testing and some specialized terminology.

We also find the exploratory testing approach useful. A good introduction to this it is the Explore It! Book by Elisabeth Hendrickson.

Many people feel that software testing means writing extensive Test Cases. However, the real goal is to find bugs. Iterators uses an exploratory approach with enough structure to make sure that all areas are examined.

The advantage of exploratory testing is to allow an experienced tester to find the best areas to test. For example, if defects are found in a certain area, then it needs even more testing. Finally, we include a good amount of practical examples because there is no substitute for actual experience.

The most significant best practice is to make sure the software testers are included in your project. Some developers will include QA software testers on their team, and some are more individual contributors. The key thing to remember is that the Product Owner is responsible for the overall product quality and should decide on the strategy for software testing.

Iterators can provide an independent view of the testing needs. We have had some developers state “it should only take 50 hours to test at the end of a project.” These same projects have then had to scramble to get functioning software to deliver.

A better approach is to follow the Agile methodology in which new features and testing are done at the same time so that new features are found to be functional at the time they are created.

Check out our software testing project case study here.

There are many software testing resources available on the internet. Here are some of the Top 10 links for software testing in different categories. There may not be 10 in each category, but they are all good resources.

What do you find are the current biggest challenges with software services for the visually impaired?

• Most websites are not accessible to individuals with a visual impairment. Individuals with visual impairments are not all blind. Some have low vision, color blindness, or multiple disabilities, including blindness. If a website is not accessible, it prevents users from using it.
  
  We have a colleague teaching technology to students with visual impairments. As part of their curriculum, they are assigned to create a report such as one of the fifty states. However, the websites are not accessible. While the students are taking a technology course and are to use their technical training to create the report, they cannot find an accessible website to complete the project. The instructors have to search and find a site that is accessible to complete the assignment.

Aside from adoption, what user difficulties do you find are hard to overcome?

• Iterators LLC is in our fifth year as an inclusive software testing company, women-owned and managed. As software testers we are certified Trusted Testers certified by the Department of Homeland Security. We test against blindness, low vision, color blindness, other visual impairments, deafness or hearing impairment, motor and developmental disabilities, elder status, light-induced epilepsy, and more. Many businesses know that their websites must be accessible, but often decide to comply only after a penalty has been charged.
  
  We branched out and secured an award from the federal government and currently work on several projects for the Library of Congress. I leave a link here: https://hbr.org/2020/03/do-your-di-efforts-include-people-with-disabilities of what I believe is an important article from Harvard Business review. "Although 90% of companies claim to prioritize diversity, only 4% consider disability in those initiatives, according to a report from the Return on Disability Group."

Why is a quality assurance tester needed on a software development team?

• Software testing does not need to be completed by the development team. Ideally, ideally there should be independent software testers working in tandem with a development team. If you only have testing completed from the development team, too many issues are lost, and the client may sign off on a project that has had insufficient testing completed.

Why is software testing important?

• Without software testing, users would not have a good experience with the software they use every day. With a lack of security, accounts can be hacked, causing a security breach such as users' passwords being compromised or other critical data compromised. We work in any industry, from small startups to large corporations and all, have software bugs that reduce the performance of the software. As testers on Enterprise software, we know when software is new, untested, and brittle, there are many bugs, and we test the software in an unfriendly environment to find the bugs.

What is a bug in software testing?

• Software bugs are coding issues where the software doesn't act the way it is expected to and, therefore, doesn't meet the client's requirements. For example, a user wants to log into a new app. The user has read and agreed to the terms of service, created a user name that follows the written guidelines, set up a user name and password that follows the requirements yet cannot log into the website. That is a bug. As software testers, we test against the written criteria.

As of February 1, 2022

Summary

• Internet Explorer 11 was the last update as of October 17, 2013
• Support for Internet Explorer on most Windows 10 computers will stop as of June 15, 2022
• IE Mode in Microsoft Edge will be supported until at least 2029

Conclusion

• All customers are encouraged to stop using Internet Explorer and the Internet Explorer compatibility mode provided in several browsers.
• However, the compatibility mode will still exist until at least 2029 and it is possible that some people will still have an internet explorer required website or app, or they may be use the internet explorer compatibility mode.
• It is therefore up to customers if software testing should occur using Internet Explorer

Wikipedia Page:

• Link

Microsoft Support site, May 20, 2021

• Link

Internet Explorer (IE) 11 is the last major version of Internet Explorer. Starting June 15, 2022, the Internet Explorer 11 desktop application will no longer be supported on certain versions of Windows 10*.

Customers are encouraged to move to Microsoft Edge, which provides support for legacy and modern websites and apps. For organizations with a dependency on legacy Internet Explorer-based sites and apps, sites will need to be configured to open in Microsoft Edge using Internet Explorer (IE) mode.

Internet Explorer mode in Microsoft Edge enables backward compatibility and will be supported through at least 2029. Additionally, Microsoft will provide notice one year prior to the retirement of IE mode. Go here for a list of Microsoft Edge supported operating systems.

See the following resources for more information:

· Internet Explorer 11 end of support and IE mode announcement

· IE mode website

· IE announcement technical FAQ

· Microsoft Edge and IE Lifecycle FAQ

· Windows OS support dates

· Internet Explorer supported operating systems

· Microsoft Edge supported operating systems

* Go here for a list of supported operating systems for Internet Explorer, including versions impacted by this announcement.

• Manual Testing
• Exploratory Testing vs. Test Cases vs. Agile
• Test Cases in TestRail with Modular Organization
• Automated Testing, Selenium, ID tags, Locators
• Enterprise Projects

One link to explore for more information is:

• https://www.atlassian.com/agile/software-development/testing

When working with clients on testing projects, Iterators LLC follows guidance from two key sources:

• ISTQB Certification: https://www.istqb.org/certification-path-root/foundation-level-2018.html
• Elisabeth Hendrickson's Explore It! Link on Amazon

The following case study summarizes some key points that your project should consider when you are evaluating the use of manual or automated testing techniques.

Manual Testing

It is well-known that manual testing is faster to get started, faster to identify bugs and an important part of an agile development process. Perhaps less well-known are some best practices regarding how to implement these techniques. The following outlines the progression of this particular project with more details given below:

• Existing test cases were provided that were out of date;
• Exploratory testing procedures were implemented; and
• Test cases were updated in a test management system (TestRail) and made more modular.

A later case study will examine the automated testing phase of this project.

Existing Test Cases

A common goal for using test cases is to identify the requirements that are covered by a given set of tests. In theory, a team could list all the software requirements and then provide acceptance criteria for those requirements. In practice, this can be challenging as requirements and implementation details change which result in extensive maintenance of the test cases themselves.

An example of this is given in the following test case example that was stored in Excel or Google Sheets:

A few challenges were found during the initial phase of these trials.

• The initial steps included the login sequence and then navigating to the page to be tested. However, these steps were out of date because those modules had been updated.
• Additionally, these initial steps were repeated in other test cases, so the maintenance effort was compounded.
• The steps for the page being tested had similar maintenance issues, but also, they were very specific to do an exact sequence of operations. As we will see below, this level of specificity is useful and necessary when creating automated tests but can be detrimental for manual testing.

In practice, the testers were spending considerable time in decoding the overall intention of each test case, which button to click, and when to evaluate the expected results. As we will discuss later, many of these issues were addressed by improving the test cases to increase their maintainability and utility.

Benefits of Exploratory Testing

Test cases are useful in some circumstances, but other manual testing strategies can also be utilized as long as the goals for the project are clear. For example, specific test cases are useful for final acceptance testing, having precise control of the coverage of testing, or ensuring that regression testing performs a specific sequence of tests.

Exploratory testing is an alternative approach that Iterators has success with on both large and small projects. Iterators starts with the goals and principles articulated in the ISTQB Foundation Level Software Testing training. In particular, our goal is to find significant bugs and help developers to fix them. Exploratory testing has several advantages over creating and executing test cases.

• Test cases tend to focus on the “happy path” of users following the intended path through the software, while there are an infinite number of non-happy paths that actual users will follow.
• This leads to the benefit of testers using their creativity and judgement to find areas that are likely to contain significant bugs or new areas that need testing. Also, diverse testers can find additional areas to test using their unique creativity and judgement in order to improve test coverage.
• We have also found that test cases tend to constrain testers as they focus on executing the exact steps in the test case and not exploring important aspects of the software.

We use Explore It! by Elisabeth Hendrickson as an excellent reference for the techniques and benefits of this approach. We have found the Explore It! approach is more representative of what actual users will do when they are trying to use the software, because they do not follow the exact sequence expected by developers (or test cases). A classic example is as follows:

Requirement:

A college course can have more than one instructor and these instructors can contribute at different levels that add to 100%, such as 50:50, 60:40, etc.

Test Case Approach:

There are at least 20 test cases that could be written for this requirement. For example, create zero, one, two, or three instructors and try to enter values that do not add up to 100%. Or, try to enter in negative numbers or a value of 0% and verify that the software correctly warns the user when the data are validated.

Bug Found with Exploratory Testing:

• Create two instructors and try to enter values that do not add to 100%. It was verified that the dialog could not be left without valid entries that lead to 100%, such as 60% for the first instructor and 40% for the second.
• Go back into the dialog and then delete one of the instructors and leave the dialog. In this case, that leaves one instructor with 60% of the course. BUG: This was allowed while the expected result would be that the 60% should be adjusted to 100%, either manually or automatically.

It is very time consuming to create test cases that cover all eventualities, and the exploratory approach allows test cases to be created ad hoc as the software is examined. There is a time and place for both approaches, and Iterators applies the most cost-effective and appropriate strategies to find bugs to support the current stage of development.

Future case studies will provide details of the next stages of this project with the following examples:

• Automated end-to-end testing of the user interface using Selenium with a Python/Excel framework
• Updated test case management using TestRail with modular test cases

At Iterators, we have experience working on complex mobile apps and we understand why testing teams are essential for a positive user experience. Clients often think mobile app developers do the software testing instead of ISTQB-certified software testers. However, dedicated software testers are invaluable when creating mobile apps; they have the approach and knowledge to thoroughly examine each facet required.

In addition, software testing tends to have the smallest price point. So, this part of the mobile app testing process can be underestimated. Iterators’ software testers know how complex the testing can be and will provide enough detail to address any issues, or bugs, prior to launch.

In this case study, we will share how essential proper software testing is for a successful app to launch.

At Iterators, we have experience working on complex mobile apps and understand why software testing is essential for a positive user experience.

Jill Willcox, Managing Member

Client: Pharmaceutical Company

Our client is a pharmaceutical company with mobile apps for IOS and Android devices. This 2.0 version will track aspects of their treatment such as medication administration and lab results. We are suing IC Agile as part of our project management strategy.

The first thing to note is pharmaceutical companies are highly regulated, which always adds another level of complexity to the project. Second, we created test scenarios as the app must be extremely well-tested to ensure that software bugs will not affect its overall use. And third, the IOS and Android apps must be identical on both platforms for an industry so highly regulated.

Below is an overview of the areas that must be tested for the app to work effectively:

Introduction Screen, which was created to grab the user’s attention

• App to capture name, user’s email address and password which must be verified
• User must accept the Terms of Agreement and system must verify the Terms of Agreement
• Once accepted, the user will be able to move to the Login page

Login page

• User must be able to log into account
• Email address and password are required and this step must be verified before moving onto next step
• Link to Reset Password must be tested and successful before moving onto the Reset Password feature

Reset Password

• User must be able to reset the password
• User must receive email to confirm the password reset works
• Software testers must test the server to ensure settings are correct for email to be received by patient, without an error

Introduction of App

• User must see a welcome tour on the app, which only occurs the first time the app is opened. Therefore, as a software tester, we will set up many accounts to ensure that the welcome page does not show again when a user logs in again
• The user should be able to go through all the pages of the app, without interruption, and once complete, move on to the Homepage
• The client must determine how they want a user to access the screen (e.g., swipe from the left or right, hit another component to log your information, or both). As software testers, we have to test all options

Dashboard

• User should be able to see the Dashboard, which encourages them to explore the site
• The app should encourage users to track symptoms on the more than 10 screens that follow, including rash, fatigue, weight gain or weight loss, sleep disruptions, etc.

Reminder Tab

• User can go to Reminders and use the Reminder Tab, which allows user to sign up for reminders about taking the medication

Enter Symptoms

• User should be able to enter symptoms of the disease, such as rash, with ease. User can either touch the screen to indicate the areas affected or type in the symptom, and should note the severity of the symptom
• There is also a Help button in case a symptom is not displayed correctly and the user needs to report it

Enter Mood Symptoms

• User should be able to enter symptom for mood disturbances; mood only has a single value
• There is a tip on the Mood page as well as a Help button

Enter Sleep Symptoms

• User should log the quality of sleep. There is one value for sleep in contrast to multiple values for rashes.
• This data is sent to the server for analysis; therefore, the software tester must test the server to ensure the process works without a glitch

View Body Map History

• User can view how symptoms like the rash have progressed or decreased over different periods of time
• The calendar will allow you to change date ranges over a week, month, or year

Notes

• User can write notes to themselves, the pharmaceutical company, or their healthcare provider. Notes are created by using a drop-down menu and, in this case, only one note can be written on a given day, regardless of the category.
• The software testers verify that this functions correctly
• User should be able to view previous pages of notes, symptoms, or results from prior dates
• User sees a default date of “Today.” If the date is “Today,” future dates should be disabled
• User sees a default of “Today” and if the dates are prior dates, the app should not be disabled

Enter Lab Results

• User should be able to enter lab results on any day
• User should be able to receive Helpful Tips on the Enter Lab Results page
• The app is not connected to the patient’s doctor; the app is to help the patient keep track of their results in an easy way

In summary, what appears like a simple app for a mobile device is actually a complex program with numerous components. The above outline – though briefly touching upon the steps – illustrates the importance of mobile application testing for quality assurance.

With that mindset, we then identified weaknesses, or objections potential customers express that can be turned into strengths or opportunities. For example, one we sometimes hear is an organization can use an offshore software testing company. This is true; however, we have an opportunity there to emphasize the benefits of using a company in the same time zone and without any language barriers that could lead to miscommunication—both of which are also strengths. Another is a lack of awareness of Iterators, the importance of software testing, or that developers are not the right people to test their product. In this case, the opportunity exists to write and publish case studies. Third, we can limit delays caused by development and mitigate losses, which is another strength to highlight. Finally, some clients cannot provide recommendations on Clutch for example, https://clutch.co/profile/iter... and word of mouth, or written recommendations can lead to growth. Government entities usually abstain from leaving a "recommendation" but do write about our work https://www.boston.gov/news/bu..., which leads to increased recognition.

Going through this exercise and having direction helps me better understand the process and the end goal. But I am more of an introvert, a bit shy, I confess with complete honesty. I would not ask a current client if they could recommend us to their counterpart at a similar company. "What do you think will happen?" one of my mentors asks. I respond that I would be fearful the customer would get annoyed and stop using us. Then my mentor poses another question: "Do you think they really will stop using you because you ask for a referral?" I respond and say no, but that's my fear and shows the level of my unease. Then my mentor shares a story: They were out of college and working their first job, a job they did not expect to have and commission only, selling a product door to door. As a condition of employment, you had to ask for referrals from every customer you had. My mentor admits he was uncomfortable with this requirement, fearing they would cancel the order they just placed. I asked what happened and am told: "I asked everyone for a referral—some would not give a referral, others did provide a referral, and no one canceled their order," and adds that asking for these referrals positively impacted his business.

There can be a couple of definitions for the word vulnerable. I'm not even sure it's the right word to use. Still, I so appreciated hearing about my mentor's vulnerability and that they were capable of feeling as uncertain as I feel. I knew that if my mentor could share this, I could ask for a referral, and I did so. I have not gotten a response, but I'm not fearful any longer. Since reading our story in The Boston Globe, our client has been a great customer, has written a testimonial about us that lives on our website, https://Iteratorstesting.com, and did so, not at my request, but because of the work we provide every week, 52 weeks a year, for the last four years. That day, I stopped worrying about something that hadn't happened yet, and it's an excellent lesson to learn and one I will not forget anytime soon.

I learned something else valuable that day. As the weeks go by, I realize that this is not just my mentorship—it's our mentorship. My mentors did not know anything about software testing or the struggles that individuals with disabilities encounter while attempting to access their Information Communication Technology, which is usually not in an accessible format and lacks ADA compliance. And maybe tomorrow or sometime down the road, women will be recognized for their significant contributions in technology and all other fields they choose to work in, with set-asides being unnecessary to level the playing field for anyone. Hopefully, my mentors will positively remember our mentorship, just as I will.

During our discussion, one mentor states being a small business is a weakness, and I have it listed as a strength. He explains that companies are concerned small businesses will close and cites the effect COVID-19 has had on small businesses. I hear it’s also a risk, and his job is to mitigate risk and find the best vendor that can complete the job.

It is hard to hear and harder to wrap my head around. My immediate thoughts are it’s a risk to hire anyone ‒ any employee, any vendor, anyone. Just look at the rate of attrition. In a study done by Mercer, the national average for total separations is 20.1%.¹ I think later I cannot imagine a business that has not taken risks to be successful, big or small, but I’m beginning to understand his point. I believe having dedicated employees and being a passionate owner of Iterators means we are going to make every effort to meet our deliverables ‒ possibly more so than those who are merely a small part of a large organization. However, hearing my mentor’s point of view on the risks associated with hiring a small company is eye opening. Examining our business through a SWOT analysis is invaluable, and I love being able to get this data. I don’t have to agree, but to succeed, I have to understand the parameters we are up against.

As a small business, we did everything any business ‒ large or small ‒ would do during a crisis. For example, we moved computers to all our employees’ homes with each having an array of phones, tablets and other equipment for testing purposes. We moved to a different office, with less square footage, and when we did not have client work, we trained our employees on additional testing specialties. Two new employees took part in training offered by the Department of Homeland Security on accessibility testing to be knowledgeable of WCAG 2.1 while another wrote a program for automated regression testing in accessibility, ensuring that we catch software bugs by running the program rather than re-testing, which reduces our client’s overall cost and frees up extra time for us to take on an extra job.

I ask my mentors if being a women-owned small business is important and am told “it’s a checkbox I either fill in or leave blank.” I have written before about the diversity of thought women bring to technology, as an example, and cite examples of mistakes made when there’s a lack of diversity in the room.² However, learning my mentors’ priorities and how others think when hiring a vendor has been enlightening.

My mentorship session is over for today and I now have to wait two weeks to speak with them about this again. We didn’t get through the entire SWOT analysis and will continue at our next meeting. It’s so valuable to have these conversations. I realize the importance of knowing others’ perceptions. What I think is a strength or weakness is only my point of view. If someone is thinking of it as a weakness, it’s good to know, and in the coming weeks we will discuss that too.

1. https://www.imercer.com/articl...
2. https://iteratorstesting.com/blog/benefits-in-tech

In full disclosure, I used the term "procurement" in the application process to mean obtaining something in the way that governments use this term when they seek our help testing websites and mobile apps. Our company, Iterators LLC (found at IteratorsTesting.com), performs software testing services for websites, mobile apps, and more. In reality, I was not suggesting I needed assistance in the area of the supply chain. Instead, I wanted to work on securing more clients. While initially embarrassed by my error, I embraced my slight discomfort. (I will write more about this later.)

Of the three mentors, two are in the procurement area, and I would not change a thing. One mentor was hired during COVID-19 and has never been to his office or met the other two mentors in person. He is authentic, analytical, and creative. The second mentor has been with the company for more than five-plus years and previously had a mentorship of her own during her career and found it beneficial. She is intelligent, thoughtful, and confident. The third mentor has been in her position for over 15 years. Working in the area of diversity, and experiencing all the events we are living through, such as Black Lives Matter, makes this an exciting space to work in with many opportunities to look forward to. She is a mentor to all, intelligent and generous with her time.

When it was my time to introduce myself, I started with my initial career path as a registered nurse and finding that all my jobs surrounded accessibility in one way or another. What was most interesting to me about our initial meeting is that my mentors didn't know anything about software testing or the extent to which it's used every day. They were completely unfamiliar with Accessibility testing, WCAG 2.1, or even the lack of access that disabled individuals encounter. When I mentioned the Voluntary Product Accessibility Template (VPAT) and explained it's a document that evaluates how accessible a particular product is according to Section 508 Standards, it blew their minds. Although this is not their area of expertise, they liked my passion, and as one of my mentors said, "I would not hire someone without passion." This mentor explained that in procurement, you interview and meet lots of vendors, and if you don't have passion, no one else will.

I learned a lot the first week, allowing myself to feel some initial embarrassment while not letting it consume me, about the generosity of my mentors even outside of our scheduled time. They have passed on many pieces of beneficial and constructive advice based on their experience, but my most valuable lesson was to own my passion. Sometimes, I try to hide my passion because it's often considered a negative for women, like the word "emotional" in the workplace. However, passion can go a long way in teaching people about a topic they are not familiar with, help pique their interest and learn its value, and maybe even encourage them to become a client should they ever need our services. From now on, I will own my passion.

When launching a new software product, it's tempting to forego hiring an outsourced QA firm to have your own in-house team test the product. After all, you'll be saving yourself money, right? But hiring a software testing company can actually end up transforming your own business in several positive ways.

Less Stress For Your Own Employees

Hiring an outsourced software testing team immediately takes the added pressure off of your own employees. It means they can focus on their own job duties without the added stress of working software testing into their schedule, which is its own project in itself that may require longer hours. This will keep productivity levels high as they work on the next big project on the agenda.

And if your product requires localization, a QA team can test it to ensure the language translates properly for your customers outside of your country. Chances are you may not have staff members that can ensure the software conforms to any cultural differences as well, particularly if the product is to be released in several foreign countries. A software testing company can thoroughly test localization in products for you.

We all know how stressful it is when there's a functionality issue with your website. Hiring a company dedicated to keeping things running smoothly through the use functional testing ensures you don't lose prospective clients due to overlooked glitches.

You'll Save Money

It may seem counter-intuitive, but paying a fee for software testing services may save you money in the long run. For example, testing mobile apps requires running the software on several devices including tablets and mobile phones. You could end up spending thousands of dollars buying these devices yourself if they're not readily available in your office, and using valuable time setting them up, you could hire a company with its own in-house testing lab that already owns and is familiar with these devices as well as the latest operating systems and browser versions.

Using outsourced software testers will also uncover any bugs in the software program early on so they can be remedied before its released to your customers. It's always best to repair any glitches or problems before a product is released to avoid the cost and time associated with trying to fix it for the next version, and to avoid any backlash from unhappy customers.

You'll Gain Competitive and Customer Insight

While an outsourced QA team certainly isn't going to divulge any of your competitors' secrets, if they've worked with products in your industry before then they may be able to offer insight into what features are popular and how customers are using them. They will also provide you unbiased, fresh feedback about your software product and be able to view and use it in the same manner your customers would.

You'll Gain a Trusted Partner

If all goes well with your chosen software testing outsourcing company, then you'll have a trusted partner to go to when you're ready to release your next new software product or updated version of a current product. You'll save time asking colleagues for recommendations or researching outsourced QA teams for the right one.

When introducing a new software product or mobile app it may be tempting to have your in-house employees test the product themselves; after all, they understand your business, your product, and your customers, and you may think this decision will save money in the long-run. But there are actually a few compelling reasons why you may want to pay for an outsourced quality assurance team.

It may sound counter-intuitive but for starters, paying for outsourced software testers can actually help save you money in the long run. A quality software testing company usually maintains its own testing lab stocked with a myriad of devices to test products on to mimic how your customers will use the product in real life. Most software companies don't have the budget to spend on extra devices and equipment for this purpose.

Instead of rushing a software product with bugs to the marketplace, hiring a software testing company ensures that any glitches are caught and can be repaired immediately. This saves time and money that would be required to make adjustments to the product after its release, and will also help you avoid unnecessary customer complaints.

Hiring an outside team of software testers that offer manual and automated regression testing services also allows your employees to focus on their jobs and daily tasks without worrying about the extra time and commitment that may compound when there's a software product that requires careful vetting before a deadline. Leaving that job up to a software testing company saves you and your own employees time and reduces stress.

Your employees may also not be completely objective about your product and company, while paying for software testing services means receiving an impartial opinion about the user friendliness of your interface. An outsourced team may also be able to provide feedback on the software based on what features your competition offers and how their customers are using the product.

Software testing companies usually also ensure that their testers have at least three to five years' experience when they're hired as well as ISTQB (International Software Testing Qualifications Board) certification, which chances are your employees do not have. This means the outsourced testers have passed exams and training to ensure they're well educated in what to look for when testing software products.

If your company is planning on releasing the software product in foreign languages to customers outside of the U.S., then you may want to pay for software testing services. Many QA teams have a deep knowledge of how software translates and what cultural differences need to be addressed, which your own employees may not be knowledgeable about, particularly if your business is confined to one America-based office.

If you do decide to turn to a software testing company, you're not alone. Generally, only very large software companies can afford the costs of a dedicated in-house testing team. Most companies turn to outsourcing to help save time and money, and to ensure their product is the best it can possibly be when it's rolled out to customers.

When your company is introducing a new software product, it's imperative that it is as glitch-free as possible to ensure your customers will be happy. Very often, hiring a software testing and quality assurance company can deliver benefits that your in-house employees may not be able to provide, but it's not always easy to find the right one, even with thousands of available choices. Here are some general tips on what to look for when hiring a software testing company.

==> Learn more about the art and science of software testing here.

How They Work With Clients

What is the on-going communication with clients like with the testing company you have in mind? Will they meet with you in person, or over video conferencing? How long will it take them to test your product and how do they provide feedback? Finding out about their testing process and how closely they work with clients will give you insight into whether they're the suitable team for your needs.

ISTQB Certification and Experience

A quality software testing and QA company only hires experienced testers -- those with an average of three to five years' experience testing software under their belt. Their testers and test managers should be ISTQB (International Software Testing Qualifications Board) certified, meaning they have passed training and exams that require a deep knowledge of software development. It's also a good idea to seek out companies that have experience testing products in your particular industry because chances are they're familiar with what your competitors have released and can provide advice on how to make your software more user friendly and competitive.

Experience Testing Multi Platforms

Software testing companies understand that the world has gone mobile and therefore, testing a software product on various platforms and browsers if applicable is extremely important. Whatever your customers will be using is what a QA company should be using to conduct its testing. Some software testing companies also feature an in-house testing lab where hundreds of the most commonly used devices are readily available to test the product.

Variety of Testing Services

From installation to user interface testing, a reputable software testing firm has experience testing all facets of software and its functions, including e-commerce platforms. If your software is to be released in foreign language versions, you'll want to choose a software testing company that has experience in localization testing and can ensure that translation is accurate on all areas of your product across other languages and conforms to the foreign user's culture.

Client Testimonials

Reviews from past clients speak volumes when choosing a software testing firm; check to see if the business you have in mind has testimonials and if they cite which industries their clientele is from and the types of software they had tested.

These are just some initial guidelines, but are a good start to help determine which software testing company is right for your business needs and type of software product.

Automated vs Manual Testing

When it comes to testing a new software product, you basically have two choices: you can choose to have people (either your in-house employees or an outsourced QA company) conduct the testing, or you can use automation testing services.

If the idea of testing your software product for bugs and other issues with a click of a button sounds too good to be true or cold and impersonal, think again. In many ways using an automated testing service company or product to vet your software offers advantages that a human test team cannot deliver.

For starters, manual testing requires a lot of labor and work. It takes time for testers to actually examine a product and to see how it performs on various devices. But with automation testing, the testing software usually only requires a few hours to examine your application and can deliver results in the same amount of time. Automation testing software also provides you with detailed data about your product and areas of needed improvement. It also tends to be more accurate than human-based testing.

However, just because the system runs automatically once integrated with your product doesn't mean that humans are not involved in the process.

The Automation Testing Process

Automation testing usually considers of three methods of testing: graphical user interface (GUI), application programming interfaces (API), and continuous testing. GUI testing mimics how a human user would use the software, by keeping track of fields that need to be filled in, clicks, and other interactions with the software interface. It is also helpful in uncovering any early bugs with the software. API testing goes deeper, essentially examining how the software interacts with other software and ensures that it meets security and compliance standards. Continuous testing is exactly that; the software is continuously tested to provide updated feedback on the software and any glitches with it.

However, just because the system runs automatically once integrated with your product doesn't mean that humans are not involved in the process. Automation testing services still require careful writing of code to direct the testing tools on what to measure, although some companies offer codeless test automation.

Types of Software Testing Companies

Where traditional software testing companies have their own lab full of equipment to test software on various devices, an automated testing services company requires minimal set-up and can usually implement their product as a cloud-based application. Most automated software testing tools require a minimal setup time and can be seamlessly integrated with your software product, including third party tools. An automated software testing company can usually offer several QA testing products depending upon your needs, and provide you with a live demo and free trial before you commit to a product.

The decision on whether to hire manual testers or to use automation testing services is entirely up to you and your specific needs as well as that of your employees. Automation testing services offers an alternative to ensure your software product is as problem-free as possible and will keep your customers happy.