How many variants should a playable A/B test have?

Two is cleanest: control and one variant. Add a third only if the budget still gives each variant the planned sample, since every extra variant splits impressions.

What is a statistically significant result for a playable test?

Usually a result where the chance the gap is random is below 5% (95% confidence). Google lets you read App experiments at 80%, 85% or 95%; 80% is a directional read, not proof.

How many impressions does a playable test need?

It depends on your baseline and the lift you want to detect. Detecting IPM 15 vs 18 takes about 28,000 impressions per variant at 95% confidence and 80% power. Smaller lifts need several times more.

Does AppLovin have a creative A/B testing tool?

AppLovin does not document a dedicated creative A/B test for UA campaigns. Teams test through creative sets attached to live campaigns and read set-level results. AppLovin MAX A/B testing is for monetization.

Can I A/B test playables on TikTok?

Yes, but put each playable variant in its own ad group. TikTok aggregates results at the ad group level when video and playables run together.

What should I test first in a playable?

The hook and the fail state. They are quick to change and affect how many players start and how many tap through after play.

Should the winner be the variant with the best IPM?

Only if its D7 retention and early ROAS hold up. IPM picks finalists; quality metrics pick the winner.

How to A/B test playable ads (with a worked example)

To A/B test a playable ad, change one element (the hook, mechanic, difficulty, fail state, end card or CTA), run the original and the variant on the same network, geo and budget at the same time, and decide only after both reach a sample size you set before launch. Judge the winner on IPM first and D7 retention or ROAS second, never on a single day of CTR.

The hard part is not the idea. It is the discipline: one variable, enough data, one read. Here is how to set that up on each major network.

Key takeaways

Change one element per test. If you change the hook and the difficulty together, a win teaches you nothing you can reuse.

Decide the sample before launch. Telling an IPM of 15 from an IPM of 18 takes roughly 28,000 impressions per variant at 95% confidence and 80% power.

Each network tests differently. Unity has dedicated Creative Testing campaigns; Meta has a creative testing tool in Ads Manager; Google ranks assets and runs experiments; AppLovin works through creative sets.

Name every build so the change is readable from the file name. Results you cannot map to a change are wasted spend.

Genie Playable builds each variant from the last version, without a developer, so the variable you meant to test is the only thing that changes.

What should you test in a playable ad?

Start with the parts that move the most players. A playable has a clear structure (we break it down in anatomy of a playable ad), and each part answers a different question.

Element	Example variants	Primary metric to watch	Question it answers
Hook (first one to three seconds)	Instruction text vs hand pointer; near-win board vs empty board	Engagement rate, IPM	Do players start?
Core mechanic	Swipe vs tap; merge vs match	Completion rate, IPM	Which part of the game sells it?
Difficulty	Three moves to win vs five; slower timer	Completion rate, CTR	Is the challenge the right size?
Fail state	Forced fail at the last move vs player can win	CTR after fail vs after win, D7 retention	Does losing make players want more?
End card	Reward shown vs character shown; static vs animated	End card CTR, CVR	Does the pitch land?
CTA	"Play now" vs "Continue level"; always visible vs end only	CTR, CVR	Does the button get the tap?
Length	15-second loop vs 30-second level	Completion, IPM	How much play is enough?

A rule of thumb from teams we talk to: test the hook and the fail state first. They are cheap to change and they tend to move IPM more than art polish does. Skins and themes are worth testing later, once the mechanic wins, to fight fatigue.

Why test only one variable at a time?

Because the point of a test is a reusable lesson, not a single winner. If variant B has a new hook, a shorter level and a new end card, and it wins, you do not know which change did it. You also cannot carry the lesson to the next concept.

The exception is a concept test. When you compare two different playables (say, a match-3 rescue vs a merge puzzle), you are testing the whole idea, and that is fine. Just call it a concept test, and follow it with one-variable tests on the winner. Our guide to iterating a winning playable covers that second stage.

One variable in. One clean variant out. In Genie Playable you iterate from the last version, so changing the fail state does not mean rebuilding the hook, the art or the end card. No developer ticket.

Try Genie

How much data do you need before you call a winner?

More than most dashboards suggest. Small samples produce big, random gaps.

A worked example in plain language

Two playables run on the same network and geo. You are comparing IPM.

Scenario	Impressions per variant	Variant A installs (IPM)	Variant B installs (IPM)	Is B really better?
Small test	5,000	75 (15.0)	90 (18.0)	Not yet. A gap this size happens by chance about one time in four.
Bigger test	20,000	300 (15.0)	360 (18.0)	Probably. The chance this gap is random drops to about 2%, below the usual 5% cut-off.

Same IPMs. Different answers. The only thing that changed is how much data you had. (These come from a standard two-proportion test; any A/B significance calculator gives the same result.)

Planning the sample before launch

To plan, you need three inputs: your baseline IPM, the smallest lift you care about, and how sure you want to be. With a baseline IPM of 15, a target of 18 (a 20% lift), 95% confidence and 80% power, you need about 28,000 impressions per variant. Want to detect a 10% lift? Plan for roughly four times as many.

Three practical consequences:

·Fewer variants, bigger samples. Every extra variant splits the same budget. Unity's own docs note that each added creative pack reduces the impressions each pack receives.

·Downstream metrics need much more data. Installs per impression is a large-number metric. D7 ROAS is built from a few paying users and is far noisier. Use IPM to pick finalists, ROAS to confirm the winner.

·Stop at the planned point. Peeking daily and stopping when it looks good inflates false winners. Unity and Google both point you to significance checks before acting.

For how many playables to put into a test round given your budget, see how many playables to test.

How do creative tests work on each network?

Each network gives you different tools, and some give you none built for playables. This is what each one documents.

Network	Testing setup	What to know for playables
Unity Ads	Dedicated Creative Testing campaigns. Each creative pack gets a fair chance to serve; billed on CPM, not CPI.	Use a control pack with history, do not add or remove packs mid-test, test in the countries you will run in. For playables, the overview compares clicks across creatives.
Meta	Creative testing in Ads Manager: duplicate an ad into two to five test ads, with part of the campaign or ad set budget set aside (Meta suggests up to 20%). A/B tests in the Experiments section for larger questions.	Requires the Highest Volume bid strategy. Pick cost per result (cost per install) as the test metric.
Google Ads (App campaigns)	Asset reporting ranks each asset Low, Good or Best against assets of the same type in the campaign. App asset experiments split traffic between a base and trial campaign.	Up to 20 HTML5 assets per ad group. Documented asset experiments cover video uplift on Android, with a recommended 50% split and 80% confidence. For playables, rely on asset labels or separate ad groups.
AppLovin	No documented creative A/B tool for UA campaigns. Creatives go into creative sets (up to ten files per asset type) that you attach to campaigns; reporting is consolidated at the creative-set level.	Put one concept or one variant per creative set so set-level results map to one change. AppLovin MAX A/B testing is a monetization feature, not a UA creative test.
TikTok Ads	Playables run in Smart+ on the TikTok placement. TikTok recommends at least six videos and one playable per ad group.	When an ad group mixes video and playables, results are aggregated at ad group level, with no per-playable breakout. Give each playable variant its own ad group to read it.

On AppLovin, practitioners who publish their method add new creative sets to campaigns that are already spending and judge each set against the campaign goal (cost per conversion or ROAS), rather than running a separate test campaign. If you are setting this up for the first time, our walkthrough of a first AppLovin campaign with a playable covers the structure.

Whatever the network, compare results within that network only. Each one counts impressions, engagements and clicks its own way. For how networks differ beyond testing, see playable ad networks.

How should you name playable test variants?

So that anyone can read the change from the file name, six months later, without opening a spreadsheet. A simple pattern: GAME_FORMAT_CONCEPT_VERSION_CHANGE_NETWORK_DATE

Part	Example	Why
Game	PZL	Short code per title
Format	PLY	Keeps playables apart from video (VID) and static (IMG) in reports
Concept	rescue	The core idea, stable across versions
Version	v04	Increments with every build
Change	failearly	The one variable that differs from the previous version
Network	ALV, META, GADS, UNI, TTK	Network-specific builds differ in packaging
Date	261026	Launch date

Full name: PZL_PLY_rescue_v04_failearly_ALV_261026. Keep the same name on the build file, the creative set and your test log. If you use in-playable events, keep the event names identical across versions too, as we describe in what to track inside a playable ad.

How do you read playable test results?

01Check the test was fair. Similar impressions per variant, same dates, same geos. Unity flags uneven delivery with a warning; on other networks, check it yourself.

02Check the sample. Did each variant reach the planned impressions or installs? If not, there is no result yet.

03Read delivery. IPM and CPI vs the control. Is the gap significant?

04Read quality. D1 and D7 retention, early ROAS. A variant that wins IPM and loses D7 is not a winner.

05Read the in-ad funnel. Where engagement, completion or end card CTR moved, so you know why it won.

06Write down the lesson. One line: "Early fail state lifted IPM on AppLovin US, D7 flat." That line is what you reuse.

The full cut, iterate or scale framework is in how do you know a playable ad is working.

Same variant, every network. Genie Playable exports optimized builds for Meta, AppLovin, Unity Ads, Liftoff, Google Ads, TikTok Ads, Pangle and Mintegral (Moloco coming soon), so a test on two networks compares the same creative, not two hand-ported versions.

Book a demo

What are the most common playable A/B testing mistakes?

·Changing several things at once. You get a winner and no lesson.

·Calling it on day two. Networks explore new creatives before they settle. Google suggests ignoring the first five to ten days of an App campaign experiment.

·Judging on CTR alone. Playable clicks often come at the end card, after play. CTR mixes hook and end card quality.

·Comparing across networks. A playable's IPM on AppLovin and a video's IPM on Meta are not the same measurement.

·Mixing formats in one ad group on TikTok. You lose per-playable reporting.

·Adding variants mid-test. Unity's docs warn this can produce inaccurate results. It applies everywhere.

·Testing without a control. Without a creative that has history, you cannot tell a good result from a good week.

·Not logging the change. If the file name does not say what changed, the test was spend without learning.

Where does Genie Playable fit?

Testing well needs a steady supply of clean variants, and that is usually the bottleneck. Genie Playable builds a playable from your APK or HTML build plus the assets in Genie, from a format that is already working, or from scratch from a brief with Genie Agent. Each new variant comes from the last version, so you change the hook, the fail state or the end card and keep everything else fixed. No developer needed at any step.

What it does not do: Genie Playable does not run your tests, pick winners or replace your network reporting and MMP. We do not publish performance lift numbers for it.

It fits teams that want to test more playable variants without adding engineering time. It is not the right fit if your in-house playable team already ships variants as fast as you can test them, or if you mostly buy on a network we do not export to yet.

Self-serve starts at $490 a month, or $249 a month for startups, and sign-up is free, with enough tokens to create your first two playables before you pay. The custom plan, usually for larger companies, is a full one-month pilot where the Genie team onboards you and helps you build the playables you want.

Test more, rebuild less. One brief in. A clean variant out. Genie Playable gives your tests the steady supply of one-change variants they need.

Try Genie

FAQ

Sources

A playable ad from your own game is a 30 to 90 second interactive slice of your game: your characters, your art and one clear mechanic, packaged as a small HTML5 file an ad network can run. You can't ship your game build as it is. Networks cap playables at about 5 MB in a single HTML file or ZIP, with strict rules on code and clicks. So you have three options: export a stripped-down scene from your engine, rebuild the moment in an HTML5 framework, or use a tool that builds the playable from your game's build and assets. This guide walks through all three.

Key takeaways

What does "a playable ad from your own game" actually mean?

It means the player sees your game, not a generic template with your logo on top. Studios that do this well keep three things fixed and treat the rest as open.

Keep fixed	Open to change
{{ r.ak }} {{ r.av }}	{{ r.bk }} {{ r.bv }}

This matches what UA and creative leads tell us. Brand fidelity is non-negotiable. Gameplay is often borrowed from whatever is trending. The catch is policy. The UK ASA now treats even a genuine but non-core mechanic as potentially misleading, and Google Play bans ads that don't represent the app. The closer your playable is to what players actually get, the safer it is, and the better it tends to retain.

Why can't I just ship my game build as a playable?

Ad networks run playables inside their own ad containers, not as apps. Every playable has to fit this box:

SizeAbout 5 MB on every major network (AppLovin, Meta, Google, TikTok, Unity, Mintegral, Moloco, Snapchat). A WebGL export of a real game is usually far bigger. An empty Godot export is already over the limit.

FormatOne self-contained HTML file with every asset inlined (AppLovin, Unity, Moloco), or a ZIP with an index.html (Google, TikTok, Mintegral, Snapchat). Meta accepts both.

No external callsNothing loads from your servers at runtime.

Click-out callmraid.open() on AppLovin and Unity, FbPlayableAd.onCTAClick() on Meta and Moloco, ExitApi.exit() on Google, window.openAppStore() on TikTok, window.install() on Mintegral.

Behavior rulesMuted until the first interaction, no store redirect on the first tap, no auto-redirects, works in portrait and landscape.

The full list lives in our Playable ad specs by network (2026).

What are the three ways to turn my game into a playable?

	1. Engine export	2. HTML5 rebuild	3. Build from your APK or HTML and assets
{{ r.k }}	{{ r.a }}	{{ r.b }}	{{ r.c }}

Most studios mix routes: a hand-built hero playable for a launch, and faster tooling for the weekly variations that keep a campaign alive.

What buyers expect to pay and wait: roughly $3,000 to $8,000 and two to four weeks per new playable from an external studio, and roughly $600 to $2,000 in-house once the setup exists. Nobody has a clean source for these numbers, so treat them as market expectations. One mid-size studio we spoke to put it simply: the market norm is "two weeks onwards", and their target is under a week.

Your game is already the brief.

Upload your APK or HTML build. Genie Playable turns it into a playable ad with your own art, characters and live events, ready for 10 ad networks.

Try it free on your game

How do I make a playable ad from my game, step by step?

01\Pick the moment worth playing

Choose one mechanic a stranger understands in three seconds. Good sources: your core loop at its most satisfying, the live event running now, or a format already converting in your genre. If you can't explain the moment in one sentence, it's too complex. One mechanic beats two.

“Merge plus puzzle looks clever and almost always tests worse.”

Segwise

02\Script the arc

Almost every playable follows the same shape:

03\Prepare your assets

Export characters, backgrounds, UI and sounds from your game, then cut hard. Compress images, use sprite sheets, trim audio to short loops and subset fonts. Base64 inlining adds about a third to file size, so aim for 2 to 3 MB in total and treat 5 MB as the ceiling.

04\Build it

Whatever route you use, keep the logic in plain HTML and JavaScript, make the layout work in 9:16 and 16:9, and keep sound off until the first tap.

05\Package it for each network

This is where engineer time quietly disappears. "A unit that passes on AppLovin can bounce on Unity," as Segwise puts it. Each network needs the right file format, the right click-out call and any required hooks. Mintegral, for example, expects gameReady() and gameEnd() calls. Build once and generate network variants from one source. Hand-editing copies is how bugs creep in.

06\Test before you upload

Then test on a real low-end Android phone. A desktop browser hides most of the problems.

07\Launch and measure

Give each playable enough data before you judge it. A common rule of thumb is about 50 conversions in four days, or 100 in seven. Judge on D7 ROAS and retention, not CPI alone. If you can, add events inside the playable (start, first interaction, win or lose, CTA tap, drop-off point), because network dashboards only show the last step.

08\Plan the next version before this one fatigues

Top-grossing games ship dozens of new playables a month, and most of that volume is variations of a winner. Leading teams put roughly 70% of output into variations and 30% into new concepts. Decide now which single thing you'll change next: the hook, the fail state, the character, the event skin or the end card.

What should I keep from my game, and what can I borrow?

Keep everything that helps a player recognise your game after install: art, characters, UI language, sound. Borrow structure freely, whether that's a proven hook, a pacing pattern or a format working in another genre. Racing and narrative studios often tell us their game "isn't playable". The honest read: a playable doesn't need your full core loop. It needs one true moment from your world.

What are the most common mistakes?

Mistake	What happens	Fix
{{ m.a }}	{{ m.b }}	{{ m.c }}

The next version, not a new brief.

When a playable fatigues, Genie Playable lets your UA or creative team change the hook, the character or the event skin on the last version and ship it again. No dev queue.

Book a demo

Where does Genie fit?

Genie Playable takes the third route, and adds two more ways in. You can build from your game: your APK or HTML build together with the assets, narrative, mechanics and live events already in Genie. You can start from a format that's already working and rebuild it with your game inside. Or you can start from scratch and describe the playable to Genie Agent.

Genie Core keeps every version on brand, and Genie Playable exports network-ready variants for Meta, AppLovin, Unity Ads, Liftoff, Google Ads, TikTok Ads, Pangle and Mintegral from one source, with Moloco coming soon. No developer is needed at any step, and your team makes the final call on every version. Sign-up is free and covers your first two playables. Try it on your game.

Where to go next

First playable? Follow how to make a playable ad step by step, then run it through the playable ad best practices checklist. Once it's live, read how to know if your playable ad is working and how to A/B test playable ads.

What should you test in a playable ad?

Why test only one variable at a time?

How much data do you need before you call a winner?

A worked example in plain language

Planning the sample before launch

How do creative tests work on each network?

How should you name playable test variants?

How do you read playable test results?

What are the most common playable A/B testing mistakes?

Where does Genie Playable fit?

FAQ

Sources

Keep reading

How do you know a playable ad is working?

How many playables should you test a month?

Your playable is fatiguing: how to iterate a winning playable

Ready to ship playables from your game?

What does "a playable ad from your own game" actually mean?

Why can't I just ship my game build as a playable?

What are the three ways to turn my game into a playable?

How do I make a playable ad from my game, step by step?

01\Pick the moment worth playing

02\Script the arc

03\Prepare your assets

04\Build it

05\Package it for each network

06\Test before you upload

07\Launch and measure

08\Plan the next version before this one fatigues

What should I keep from my game, and what can I borrow?

What are the most common mistakes?

Where does Genie fit?

Where to go next

FAQ

Sources

{{ b.text }}

{{ b.text }}

FAQ

Sources

Keep reading

{{ r.title }}

Ready to ship playables from your game?