Read the prior installments here:
Our final topic for the Keys to Success for Complex NetSuite Implementations blog is focused on Testing. This is another area where the approach for small projects greatly varies from large enterprise projects. On a small project, testing may be limited to a few hours by the implementation team and some ad hoc User Acceptance done by the business users. Meanwhile, enterprise level clients usually have a dedicated team and follow rigorous processes for test planning, testing status reporting and defect resolution.
It is important, of course, to understand and align with the client’s testing methodology to achieve project success. While there are certainly many benefits to these more structured and comprehensive methodologies, it is also worth noting that this often adds administrative overhead and time to the project.
Rather than leave testing as an afterthought to be considered once configuration and development activities are nearing completion, it is key to engage with the QA team early in the process. I recommend having at least one QA Lead or representative attend requirements sessions to understand the full context of the solution and business needs.
Documenting test cases directly from requirements is another best practice. Another advanced technique we like to use on large projects is Test Driven Development which we will delve into in a separate blog in the near future.
When working with mature Enterprise level organizations, there are usually well defined roles and responsibilities for testing. It is important for implementation teams to understand and align with these. Typically, these roles and responsibilities are loosely defined as this:
First and foremost, test cases should be created by the client QA team, or at least the client users who will be performing the testing. In some cases, clients ask implementation teams to assist with this task. However, in my experience, this gets into dangerous territory.
In order for test cases to be well understood and easy to execute, they need to be written by people who truly understand the requirements, the real-life use cases and expected results. Relying on the implementation team to write test cases, makes them only as good as the requirements, creates risk for things to be lost in the translation and creates additional difficulties in performing testing. In some cases, where smaller clients ask for this type of assistance we offer to collaborate with them, but resist entirely taking on this effort.
Another best practice I recommend is carefully designing the test plan to minimize overlapping test cases. Multiple test cases for the same functionality, be it configuration or customizations, make defect management more complex. For example, when multiple test cases share common functionality elements and some of the test cases pass while others fail requiring a fix, then all the test cases must be retested once the defect(s) is corrected.
This also places constraints on promoting defect fixes across environments, as all defects around all the functionality in the impacted test cases must be resolved at the same time to re-test. This can lead to a vicious inefficient cycle of multiple rounds of executing the same test cases.
Testing execution is usually planned to begin once development, configuration and unit testing are complete. There is typically a plan to complete a certain percent of test cases per week and daily defect resolution meetings to keep progress on track.
Defect Resolution (aka Testing Execution) meetings are used to keep testing progress on schedule. To be successful these need to be attended by all stakeholders.
Identified defects need to have appropriate priority assigned to them to help the teams prioritize the order in which to address them. At times, when a project is running out of time and/or budget it may become necessary for the business to de-scope some lesser priority defects to be addressed at a later phase.
For this reason, especially on large projects, it is important that the whole project team understand, follow and enforce the process. When QA team members reach out directly to individual implementation team members to triage defects prior to business validation and prioritization, a lot of time may be wasted on low priority work while higher priority items lack traction.
Finally, planning for the move-up and re-testing of defects is integral to successful testing execution. The technical, functional and testing teams need to collaborate closely on the following:
To the last point, about promoting defects to the QA environment. The importance of collaboration here cannot be understated. QA must control everything that is promoted to their environment. Test cases and various functionality are often intermingled and making changes/fixes without the QA team’s knowledge and planning is a recipe for conflict.
Making an unexpected fix has the potential to invalidate multiple test cases and cause a lot of lost time and re-work for the QA (and often UAT) team. For this reason, I recommend having a QA Lead act as a gatekeeper for any changes being moved up to the QA environment. The implementation team should not have the access to promote changes directly, unless given formal permission.
Testing can be a stressful time on projects as the best designed solutions sometimes unravel, missing requirements and expectations are discovered all while the project hurls towards the impending Go Live date. Heeding these best practices around Test Planning, Clear Roles and Responsibilities, Test Case Design and Testing Execution has proven to make this phase easier for me and my team to manage.
Now that NetSuite is officially under the Oracle umbrella; I expect the number of complex implementations to continue to increase as NetSuite expands into many more Oracle data centers. As we embark on this new exciting journey together, we expect the implementation methodologies and best practices to continue to mature to meet the needs of a larger more international customer base. I hope you found this blog series helpful and welcome your thoughts and feedback!
Learn more about Techfino on our resources page.