Many testing organizations become frustrated with the limitations of a legacy test platform. They are also burdened by the overhead of keeping up with aggressive product lifecycles. But changing test platforms can be too daunting, leaving legacy solutions in place years beyond their useful lifetimes.
This blog addresses the challenges of migration to a new test automation platform and lays out the first step on a path for successful migration, a Proof of Concept (PoC).
What is a Proof of Concept?
Before launching a new product, adopting a new methodology, or migrating to a different platform, your organization will benefit from evaluation of the proposed solution. Defining the scope for a PoC is key to obtaining useful results. In the case of test automation, that evaluation should replicate as many attributes of the new use case as possible. ,
With rich software environments, such an evaluation calls for a Proof of Concept, where the outcome can help stakeholders to make informed cost:benefit decisions about migration.
Challenges and Risks of Test Platform Migration
As with any type of legacy software, moving to a new test automation platform presents risks in configuration, programming, and operation:
Need to learn a new testing framework, a different architecture, or an unfamiliar programming language
Compatibility and interoperability with existing test software and hardware instrumentation
Stability and quality of the migration target
Utility and usability of solution outputs
Possible interruption or degradation of existing operations, e.g., ability to collect results data and maintain testing continuity
Rather than taking a “leap of faith”, a proof-of-concept lets you test and evaluate important metrics and parameters in a real-world setting.
PoC Benefits
A proof of concept may seem like a lot of effort. Formalizing your platform evaluation, however, delivers significant benefits, in general and for Test Automation migration.
Benefit |
Description |
Test Automation Migration |
| Feasibility | Highlights whether a project is feasible. Discovering flaws in assumptions at an early stage helps you to decide whether to pursue further or to look elsewhere | Will the new test automation platform handle legacy test cases and instrumentation? |
| Cost Prediction |
Can provide a realistic basis for real-world costs. A PoC can uncover hidden costs or find new ways to save money once the actual project is underway. | What type and scope of resources will be needed to migrate test code and instrument interfaces to the new platform? |
| Risk Identification |
Helps to spot risks and obstacles early, so you can find ways to mitigate problems before they develop | Performance gap? New skills required? Sufficient interoperability? Downtime required? |
| Outcome Verification |
Shows you what to expect from the completed project. You may discover new uses or additional applications during the proof of concept stage | Create workflows and integrations that can carry forward into production-ready test plans |
| Early Feedback |
Elicits feedback before the full project is in motion, helping you to make necessary modifications before beginning implementation. | Test prototype plugins for DUTs and evaluate test case mapping from legacy to the new platform |
| Stakeholder Showcase |
Can show your project has a high likelihood of success, spurring management to approve it. | Demonstrate value of a new test automation platform to stakeholders |
An OpenTAP PoC
Engaging in a Proof-of-Concept before committing to a migration to OpenTAP (or any other platform) gives you and your team a preview of what’s to come. If nominally successful, a PoC also enhances your ability to justify such a move to management.
PoC Checklist
Following are questions you should ask going into and coming out of a migration PoC:
What are the high-level requirements for the product or service under test, and for the test platform?
Are the PoC parameters realistic? Can you reliably use the platform in a real-world setting?
Is the scope of the PoC small enough to be manageable but extensive enough to yield useful results?
Are software, hardware and human resources available to support to PoC?
Are actual stakeholders and users involved in the PoC?
Can you estimate and defend the costs of the PoC and the migration?
What other technologies are needed to integrate/operationalize the new solution?
How are you going to collect and analyze feedback from the PoC?
What metrics will you use to compare the new and legacy approaches?
This checklist may not be exhaustive, but it offers a starting place for planning your PoC.
Tell Us Your PoC Story
The PoC (a.k.a. “pilot”) in the OpenTAP video Seeing Is Believing gave Francisco, Gabriel and their team sufficient confidence to move their OpenTAP migration forward, towards production. We’d love to hear about migration proofs-of-concept carried out by our readers in the OpenTAP ecosystem.
You can share your PoC experience or pose questions
