A/B Widget Testing Tool

A/B Widget Testing Tool

I redesigned our internal split testing tool to improve usability and cut reliance on third-party platforms. The new experience simplified the test setup and result tracking. Two months after its launch, usage doubled, resulting in a $21,000 monthly savings for the company.

I redesigned our internal split testing tool to improve usability and cut reliance on third-party platforms. The new experience simplified the test setup and result tracking. Two months after its launch, usage doubled, resulting in a $21,000 monthly savings for the company.

Role

Lead Designer

Timeframe

1.5 Months

Toolkit

Figma, FigJam, Chat GPT

Collabortors

Product Manager, Developer

Our in-house A/B testing platform, Inline Media, had low adoption rates, with only 5% to 7% of internal teams utilizing it. At the same time, the company spent over $150,000 per month on overlapping third-party tools. A complete rebuild wasn’t feasible, so we aimed to boost adoption through targeted UX improvements. As the lead designer, I managed the entire design process, including discovery, research, prototyping, and usability testing, to make Inline Media an intuitive tool for experimentation, reduce our reliance on external platforms, and consolidate test data.

Problem

User interviews and internal analytics revealed several key issues that explained the low adoption:

  1. The setup flow for split testing was confusing, especially for non-technical users.

  1. There was no clear way to preview or compare test variants before or during a test.

  1. Historical test data were hard to access and lacked visual clarity.

  1. Teams relied heavily on third-party tools, such as Looker, to interpret test results, which fragmented the workflow.

To deepen our understanding, I facilitated an empathy mapping session with seven key stakeholders—marketers, content managers, and data analysts who regularly run tests. These sessions uncovered common frustrations and helped identify two high-impact areas to prioritize in the redesign:

  1. Improve the split testing experience and flow.

  2. Enhance asset preview functionality before and during test setup.

Where do we go, and what does it look like?

After brainstorming, I hypothesized that low adoption came from missing features and limited testing. Our goals were to enhance test feedback, streamline split test workflows, and clarify historical data.

I conducted secondary research on tools like Google Looker, Storyblok, and Salesforce to understand how they present and retrieve data, which directly informed our approach.

From there, I developed a product requirements document, user stories, and outlined key pages and inputs. After aligning with the product manager and confirming feasibility with engineering, I focused on designing MVP test preview pages. This clarified two main areas: split testing and the ad server, with some overlapping insights, but the priority was enabling teams to set up and evaluate tests effectively.

Drawing to discover

After thoroughly reviewing the existing split testing process, I redesigned the flow to simplify the experience. This included introducing new pages and sections while eliminating redundancies to improve usability and efficiency.

Through collaboration with engineering, we quickly validated feasibility and aligned on which improvements we could deliver within the 2-month timeline.

Key design questions emerged:

  • How might we make test creation intuitive and informative?

  • How can we effectively display historical performance and comparative variations?

  • How might we reduce the need for users to consult external dashboards?VP scope to focus on scalable improvements that could deliver immediate value and pave the way for future iterations.

Through collaboration with engineering, we quickly validated feasibility and aligned on which improvements we could deliver within the 2-month timeline.

Key design questions emerged:

  • How might we make test creation intuitive and informative?

  • How can we effectively display historical performance and comparative variations?

  • How might we reduce the need for users to consult external dashboards?

    MVP scope to focus on scalable improvements that could deliver immediate value and pave the way for future iterations.

With user insights and technical constraints in hand, I began creating the new flow and sketching low-fidelity concepts for an MVP focused on practical usability gains.

Initial concepts included:

  • Creation Date Column: To help users sort and identify older tests.

  • Inline Asset Previews: Thumbnail previews next to each variant for quick visual reference.

  • Test Overview Section: A dedicated view summarizing test metadata, dates, and statuses.

  • Contextual Tooltips: Embedded explanations during test creation to clarify field purposes.z

    Using this feedback, I adjusted the MVP scope to focus on scalable improvements that could deliver immediate value and pave the way for future iterations.

With user insights and technical constraints in hand, I began creating the new flow and sketching low-fidelity concepts for an MVP focused on practical usability gains.

Initial concepts included:


  • Creation Date Column: To help users sort and identify older tests.

  • Inline Asset Previews: Thumbnail previews next to each variant for quick visual reference.

  • Test Overview Section: A dedicated view summarizing test metadata, dates, and statuses.

  • Contextual Tooltips: Embedded explanations during test creation to clarify field purposes.z

    Using this feedback, I adjusted the MVP scope to focus on scalable improvements that could deliver immediate value and pave the way for future iterations.

With user insights and technical constraints in hand, I began creating the new flow and sketching low-fidelity concepts for an MVP focused on practical usability gains.

Initial concepts included:

  • Creation Date Column: To help users sort and identify older tests.

  • Inline Asset Previews: Thumbnail previews next to each variant for quick visual reference.

  • Test Overview Section: A dedicated view summarizing test metadata, dates, and statuses.

  • Contextual Tooltips: Embedded explanations during test creation to clarify field purposes.z

    Using this feedback, I adjusted the MVP scope to focus on scalable improvements that could deliver immediate value and pave the way for future iterations.

With user insights and technical constraints in hand, I began creating the new flow and sketching low-fidelity concepts for an MVP focused on practical usability gains.

Initial concepts included:

  • Creation Date Column: To help users sort and identify older tests.

  • Inline Asset Previews: Thumbnail previews next to each variant for quick visual reference.

  • Test Overview Section: A dedicated view summarizing test metadata, dates, and statuses.

  • Contextual Tooltips: Embedded explanations during test creation to clarify field purposes.z

    Using this feedback, I adjusted the MVP scope to focus on scalable improvements that could deliver immediate value and pave the way for future iterations.

Bringing the design to life

To save time and maintain UI consistency, I bypassed wireframes and designed directly in high-fidelity using our internal design system. This approach enabled rapid prototyping and made stakeholder reviews more efficient.

Key design changes included:

  • Terminology Refinement: Changed “Traffic Sources” to “Properties” to reflect how internal teams already referred to these segments.

  • Preview Toggle: Introduced a simple switch between thumbnail and expanded views, giving users control without clutter.

  • Improved Filtering: Implemented filters by business vertical, date range, and test status, grouped for usability.

  • Test Overview Page: Created a separate view where users could see test results side-by-side, including performance metrics and winning variant indicators.

  • Embedded Dashboards: As an interim measure, we embedded Looker dashboards to show test performance while we scoped out future native visualization components.

  • Historical Data & Auditability: Added overview pages that include a complete list of split tests, with dates created and last edited, to provide teams with visibility into experiment history and ownership.

    Every design element was reviewed with the product manager and engineers to ensure alignment with backend capabilities and product goals.

To save time and maintain UI consistency, I bypassed wireframes and designed directly in high-fidelity using our internal design system. This approach enabled rapid prototyping and made stakeholder reviews more efficient.

Key design changes included:

  • Terminology Refinement: Changed "Traffic Sources” to “Properties” to reflect how internal teams already referred to these segments.“

  • Preview Toggle: Introduced a simple switch between thumbnail and expanded views, giving users control without clutter.

  • Improved Filtering: Implemented filters by business vertical, date range, and test status, grouped for usability.

  • Test Overview Page: Created a separate view where users could see test results side-by-side, including performance metrics and winning variant indicators.

  • Embedded Dashboards: As an interim measure, we embedded Looker dashboards to show test performance while we scoped out future native visualization components.

  • Historical Data & Auditability: Added overview pages that include a complete list of split tests, with dates created and last edited, to provide teams with visibility into experiment history and ownership.

    Every design element was reviewed with the product manager and engineers to ensure alignment with backend capabilities and product goals.

Feedback during early stakeholder reviews revealed areas that needed refinement:

  • The filtering system in the admin dashboard lacked logic and flexibility.

  • Preview functionality was inconsistent, either too small to be useful or too large to be efficient.

  • There was no way to view which business unit or vertical a test belonged to.

  • Teams wanted better visibility of test progress over time, but engineering advised deferring complex charting until after the MVP.

Using this feedback, I adjusted the MVP scope to focus on scalable improvements that could deliver immediate value and pave the way for future iterations.

Testing & Iteration

With one week left before design handoff, I conducted remote usability tests with four internal team members. These sessions were focused on test creation, result interpretation, and filtering.

Findings and solutions:

  • Unclear Preview Scaling: Users were unaware that they could resize previews. I replaced the manual resizing mechanism with a toggle-based approach that was easier to find and use.

  • Lack of Onboarding Cues: New users weren’t sure how to get started. I added inline tooltips and empty-state messages with action prompts.

After incorporating these updates, I created a clickable prototype and validated it with stakeholders before passing it to the engineering team.

Final Results

The MVP was launched on time and received strong support across all teams.

Measured impact two months post-launch:

  • Adoption increased by 14%, nearly doubling the user base.

  • $21K/month saved through reduced reliance on third-party tools.

  • Improved efficiency in test setup and result visibility.

  • Greater confidence among teams, thanks to a centralized platform and precise results.

The tool transitioned from being overlooked to becoming a foundational resource for experimentation.

Reflections & Next Steps

What worked

  • Early user input kept us focused on real problems.

  • The design system enabled fast, high-quality delivery.

  • The MVP scope allowed us to ship impact quickly and build a foundation for growth.

What to improve

  • Engage non-technical users earlier for clarity and accessibility.

  • Prototype graphs to validate data comprehension sooner.

Next steps

  • Replace embedded dashboards with scalable native visuals.

  • Standardize admin tools to reduce context switching.

  • Add a guided setup flow for new, non-technical users.

  • Implementing and then redesigning the ad server and other sections of the app.

Takeaway
The project showed how UX connects user needs with business goals, improves workflows, and reduces reliance on external tools. It also underscored the value of alignment, focus, and continuous iteration.

What worked

  • Early user input kept us focused on real problems.

  • The design system enabled fast, high-quality delivery.

  • MVP scope helped us to ship impact quickly and establish a foundation for growth.

What to improve

  • Engage non-technical users earlier for clarity and accessibility.

  • Prototype graphs to validate data comprehension sooner.

Next steps

  • Replace embedded dashboards with scalable native visuals.

  • Standardize admin tools to reduce context switching.

  • Add a guided setup flow for new, non-technical users.

  • Implementing and then redesigning the ad server and other sections of the app.

Takeaway
The project showed how UX connects user needs with business goals, improves workflows, and reduces reliance on external tools. It also underscored the value of alignment, focus, and continuous iteration.

You like my work? Get in touch and learn more about me.

You like my work? Get in touch and learn more about me.

You like my work? Get in touch and learn more about me.

You like my work? Get in touch and learn more about me.