Acquia Site Studio (previously known as Cohesion) is a low-code Drupal-based platform that allows content editors and marketers to create unique websites using components and layouts. It uses a drag-and-drop editing feature which provides fast and seamless learning for QA Engineers having less coding knowledge. We recently delivered a low-code, enterprise scalable, self-service platform and CMS for a German multinational pharmaceutical and life sciences company to build 1000+ websites and migrate 170 consumer brands. Based on our experience, we will be describing our QA process and the best practices for Site Studio testing.
Understanding Site Studio
We applied the W2H1 (Why-What-How) approach for understanding Acquia Site Studio from a Quality Assurance perspective.
Why is it important for your QA Engineer to understand Acquia Site Studio?
- When it comes to Acquia Site Studio (Cohesion) projects or any new platform implementations; the QA team must be well versed with the platform to deliver good quality results
- To identify the root cause of an issue - whether it is an issue with the component or backend functionality
- To validate whether the component is built and working as expected i.e. if it meets the business and user requirements
- To validate if the integration between the components (frontend) and the functionalities (backend code) is in sync with the interface requirement
- To validate the usability and compatibility of the website built
Top 6 things your Quality Assurance team should know about Acquia Site Studio
- Site Studio elements and templates
Site Studio (cohesion) provides a wide range of in-built form elements that can be used to create custom components as per project requirement. Site Studio also provides options to create various templates for the website - Master, Content, View and Menu Templates.
For example, a QA Engineer has created the header menu links at the backend but it is not rendering as expected at the frontend. In such cases, the QA Engineer should know which template to refer to, to identify the cause of the issue.
- How to create a custom component
Site Studio provides the ability to create components with very less code knowledge. This enables QA Engineers without much coding knowledge to create components and perform testing efficiently and effectively.
Let me share an example of our project, we were required to validate if the content of the “Modal” custom block reflects properly on to the frontend. This “Modal” component was yet to be developed by a 3rd party vendor.
So we created a component using the Site Studio “Modal” element and used it to validate the “Modal” block feature on the frontend!
- Purpose of the components and how they work individually
This knowledge helps the QA Engineer to validate if the component built meets the requirement and serves the purpose of reusability. In addition to this, it helps the QA Engineers to provide a workaround for any enhancement/condition-specific use cases, using the existing components.
- How components are integrated with the backend system to achieve the requirements
In our project, there was a requirement to launch a modal pop-up on clicking a menu link. In this case, a QA Engineer should understand how to sync the Drupal menu link and the Site Studio Modal component. And should also validate that clicking a menu link triggers the launch of the correct modal pop-up.
- For multi-site solutions with multiple themes, QA Engineers should know ‘Style Guide Manager'
Style Guide Manager helps to change styles and appearance depending upon the theme/subtheme. This would help the QA Engineers to validate if the changes made to style values in SGM are reflected correctly on the frontend for the corresponding theme.
- How to import/export Site Studio package on a site
Acquia Site Studio facilitates sharing of components, templates, styles and configurations with other websites using the import/export package option.
QA Engineers can import and export packages using either UI or Drush. For Site Studio projects, QA Engineers usually need to validate that the latest package (provided after each release) is imported and rebuilt successfully on the website. And the updated package is not breaking existing features.
This also reduces the dependency of the QA Engineers on the dev team for importing packages on various websites.
How to perform Quality Assurance testing for Site Studio?
Focus areas for Site Studio Testing
Compatibility Testing
- Site Studio provides responsive support for different devices. QA Engineer needs to verify whether a particular component renders and works as expected across different devices - desktop, laptop, phone, and tablet.
- For example; a QA Engineer must check if there are any horizontal scrolling, alignment or font issues w.r.t the buttons on different devices and multiple screen sizes.
- QA Engineers need to verify compatibility across different browsers. Sometimes, a component may work correctly on Firefox, however, it may fail on Safari. So, we need to validate if the components are rendered as expected across browsers.
Regression Testing
Regression testing is to check if the modification/upgrade of the application has not led to a functional break. This testing involves the below activities:
- Ensure that the existing functionality still works after the Site Studio version is upgraded
- Selected test cases are re-executed to ensure changes have not impacted previously tested features
- Check if the end to end testing workflow works fine
- The data flow between components is working fine
- Check if the styles and user interface is not impacted
Integration Testing
Integration Testing is carried out once the individual components have been unit tested and are working as expected; without waiting for the entire system to be ready. This ensures early detection of defects and also helps in isolating the defects to particular component interaction.
For example, we had to ensure that the “Product Teaser View” and “Slider component” are ready at Drupal and Site Studio end respectively. QA had to validate if fields of the product teaser view are rendered correctly in the slider component. And an end-user can slide through products and navigate to the product's PDP page via click event.
UI Testing
The user interface is crucial when it comes to creating a reliable website. To ensure all the elements of the front-end work as expected and are displayed equally well for users across all devices and browsers.
There are a couple of factors to validate as part of UI testing:
- Styles/Icons/Font
- Templates
- Field widths
- Navigational elements
- Progress bars
- Menu items
- Table/Slider scrolling
- Error logging
- Action buttons
- Modal Pop-up
Multilingual + RTL
When content is translated into languages other than English, it gets expanded in length, this might affect the UI/UX of the website. English texts are usually short compared to other languages. Therefore, translated content leads to truncation, overlay on or under other components and so on.
Testing of components using Right-to-Left language requires special considerations such as the focus on text alignment, ordering of components within the layout, positioning of buttons, dropdown, data entry fields, so on. Therefore, QA Engineers need to make sure that no components and design are breaking for multilingual and RTL language sites.
Automated Regression Testing
To achieve enhanced product quality and team agility, it is always ideal to automate regression testing. The regression test suite should include both functional and UI test cases.
- Functional Regression:
Functional regression testing is essential for verifying that components of our system are working as expected after a code change. For Site Studio, we automated functional regression test cases such as the creation of contents using components, interaction with components and so on, using an automation tool called Behat.
After every release, we ran the automated functional regression test suite. This ensured that existing core features were working as expected and also increased the chance of detecting bugs caused due to the change.
- Automated Visual Regression Testing:
Apart from having fully functional components in the system, it is also important to be concerned about the user interface. Styling or CSS change can have undesirable effects on other components and might break responsive pages at different resolutions. Therefore, we need visual regression testing.
Visual regression testing is a visual comparison between the base and test image versions. However, today there are a huge range of browsers, devices and screen resolutions and testing them manually has become impossible. Therefore, we need to automate this testing using a visual regression framework/tool such as Backstop JS.
Challenges we encountered while testing Site Studio
- Multi-site testing
Brand sites have specific design requirements apart from the Master Design System. When a new package is rolled out on multiple brand sites, we need to test that components and styles are rendering as expected on each brand site. This increases the testing effort and time. To minimize the testing efforts and find UI bugs, we can use visual regression testing tools.
- Managing package import/rebuild
QA Engineers need to verify that the package import and rebuild are successful. Sometimes, a package is imported successfully on the website but the rebuild fails. QA Engineers also need to ensure nothing breaks after import and rebuild, from both style and feature perspectives.
- Site Studio upgrades
Site Studio upgrades sometimes have an impact on the websites. Therefore, we need to perform regression testing on websites using upgraded versions on lower testing environments, to identify if there any impact or risk. Once the entire checklist is tested and passed, upgrading is performed on higher environments.
For example, with Site Studio 6.1.1 upgrade, there was a change: “Themes with Acquia Cohesion enabled and not set as default will no longer generate assets automatically”.
In our project, this impacted the websites using the sub-theme and as a result, components were not rendered correctly for that theme. The solution was to check the Build Site Studio assets checkbox in the appearance settings of the theme under the Site Studio section. Once this solution was applied successfully to existing sites using a non-default theme and to new sites, an upgrade was performed on higher environments.
- API outage
API Outage of Site Studio has a huge impact especially when we are developing, testing and deploying features. This blocks the activities related to site studio testing. Initially, when we started to use Site Studio in our project, we often faced API Outage. Now, Site Studio is very stable and consistent in terms of API Outage.
Conclusion
We covered the purpose and process of understanding Acquia Site Studio from a Quality Assurance perspective, along with the test approach that should be applied to any Site Studio and Drupal integrated project. We also focused on the areas of impact and the challenges that we encountered in our project. If you’ve worked on a QA project for Site Studio, we would love to hear from you!
Happy Testing!