First the decision, then the variations
“We want better creatives” isn't a test question yet. A better one is: “Does an opening with a specific customer question generate more relevant inquiries than one that directly shows the application?” Now it's clear what needs to change and which next production step depends on the result.
A hypothesis connects the target group, the suspected obstacle, and the variant. For example: “Prospects don't yet understand how to use our offer. A visible application could help them understand it.” This is a plausible assumption, not a proven psychological effect.
Before testing, also check the offer and the follow-up page. If the form doesn't work or the service remains unclear, a different opening won't address these issues. Creative, target group, and landing page must be considered together, without changing everything simultaneously for every comparison.
What can change between two variations?
For a clearly defined question, the other relevant conditions should remain as consistent as possible. For example, you can compare two approaches with the same main body, offer, and next step. If the presenter, price, target audience, and landing page change simultaneously, you won't know later which single difference was responsible.
Comparing entire concepts is still possible. However, the statement should be "Concept one versus concept two under these conditions," not "This single sentence works better." This distinction determines what the production can learn from the result.
Constructed variant matrix for a consulting provider:
| Component | Variant with customer question | Variant with application |
|---|---|---|
| Entry point | A real, recurring question from a consulting session. | A specific situation where consulting is needed. |
| Main part | The same professionally reviewed process. | The same professionally reviewed process. |
| Evidence | The same approved example. | The same approved example. |
| Next step | The same description of the initial consultation. | The same description of the initial consultation. |
| Goal and follow-up | Identical target page and criteria for matching inquiries. | Identical target page and criteria for matching inquiries. |
The matrix is a planning example without results data. For Video production it shows which additional shots are actually needed. It is not an invitation to cut out arbitrary variations from every detail.
A/B test or comparison in normal ad operation?
In a controlled A/B test, the variations are compared using appropriate allocation. The TikTok documentation on split testing describes separate, equally sized target groups without any overlap. You can check the specific availability and configuration in the respective ad account.
If you run multiple ads in an automatically optimized campaign, the platform may deliver them differently. Differences in the number of people reached, the timing, or the number of impressions can influence the comparison. Such observations are useful for further planning but shouldn't be considered experimental proof without further consideration.
Even two organic posts don't usually constitute a clean A/B test. They can reach different people at different times. Use them as clues for new questions, rather than deriving a general rule from a single successful post.
Which key performance indicator (KPI) is decisive?
Choose a key metric that aligns with your decision. For example, in a campaign for inquiries, the effort per qualified inquiry might be relevant. Define beforehand what "qualified" means: Does the request match your offering, are the essential prerequisites met, and is the contact reachable?
| Observation | Helps with this question | Not yet answered |
|---|---|---|
| Video views and playback | Is the content even being viewed? | Whether suitable customers are generated from it. |
| Clicks on the next step | Does the ad generate interest in joining? | Whether the expectations on the landing page are met. |
| Inquiries | Is contact made? | Whether the contacts are relevant to the offer. |
| Qualified inquiries | Are you receiving inquiries that are relevant to your field? | Whether offers are accepted. |
| Orders and economic contribution | How is the business developing? | What contribution a single video made. |
A video with low click-through rates can raise many unsuitable expectations. A different approach might generate fewer leads but more relevant inquiries. Therefore, feedback from sales should be included in the analysis. You can find the basics of the cost metric in the article about Cost per Lead.
How long should a creative test run?
There is no reliable minimum duration or fixed number of purchases that makes every test robust. Planning depends on factors such as the initial level, the economically relevant difference, the expected data volume, and the test methodology used.
Before starting, define when the evaluation will take place and under what conditions you will stop. With a firmly planned test, you shouldn't search for a winner every day and stop at the first suitable interim result. Technical errors or unacceptable business consequences may still justify termination.
With only a few inquiries, the results are often uncertain. "No significant difference yet" is a possible result. This is neither proof of equivalence nor a call to allocate unlimited budget. A smaller range of options or a different test question might be more appropriate.
A calculation example – and why it doesn't prove a winner
Let's assume, in a hypothetical comparison , €500 is spent on each option. The first option generates ten inquiries, two of which qualify according to your criteria. The second option generates six inquiries, three of which qualify.
The cost per inquiry is €50 or approximately €83. In contrast, the cost per qualified inquiry is €250 or approximately €167. The decision will vary depending on the key performance indicator (KPI).
These small sample sizes don't demonstrate a reliable advantage. Different delivery methods, random fluctuations, or inconsistent qualification can alter the impression. This example merely illustrates why defining the target metric before starting is crucial. It's not a client result and not a budget recommendation.
What goes back into production after the test
Document the test question, variations, time period, data source, and known limitations. Record the decision you make based on this: adopt, discard, or continue with a more precise question. If no clear difference is apparent, a simpler or cheaper-to-produce variant might still be viable—just not as a proven performance winner.
Furthermore, declining results don't automatically indicate creative fatigue. Also check delivery, offer, season, measurement, and the post-click journey. A new video created on a hunch might address the wrong issue.
At Vertikal Viral, we combine delivery and optimization with the next production. Bring along existing ads, your goal definition, and any known feedback from inquiries. From this, we can derive a concrete next test. Let’s think BIG.
Differentiating between creative testing, performance creative, and fatigue
Creative testing is the method you use to compare assumptions. Performance creative refers to the advertising material tailored to a campaign objective. Creative fatigue describes a potential decline in effectiveness over time. The terms are related but not interchangeable.
This helps with developing a new advertising medium. Performance Creative BriefingIf a previously effective ad becomes less effective, start with the fatigue diagnosisbefore hastily considering a new test as the solution.