Putting ChatGPT and Bard to the Test in PPC Marketing

AI has been found useful for SEO, programming, content editing, and even meal planning and its applications continue to expand.

But is ChatGPT or Google Bard useful for PPC marketing?

To answer this question, my colleague Jake Wieczorek and I embarked on a month-long test, spanning all of our ad accounts and over 20 tasks, both big and small. 

In the following sections, I will be sharing our findings on 15 of these tasks (using Wayfair’s “All things outdoorsy” campaign for screenshot examples to protect client privacy).

Here are the 3 objectives of this test:

1. To see if ChatGPT and Bard can perform real, complex tasks that we perform (and test various prompts in a practical environment)
2. To see if ChatGPT and Bard can enhance our work and make it more effective and efficient
3. To see if ChatGPT and Bard can replace us altogether

Minimum viable test sample:

• 2 client accounts per task and all applicable prompt variants (from 2 to 5 depending on the task, not including refinement prompts)

We went into this with a bit of defiance (“We’re not getting replaced by some AI!”), but not without admiration for the things it’s been proven to do well and hope that it can streamline our processes. 

Achieve better ROI from your ads

Serious about growth? Let’s talk about getting better leads and higher ROI.

SCHEDULE A CALL

Scoring

Before we get into the results, a brief explanation of the scores.

‍To simplify scoring, we looked at just the two most important things for us:

• Usefulness (👍 Useful, 🤔 Potentially useful, 👎 Not useful). A ‘Useful’ score means we can leverage all or most of the results/answers while a ‘Potentially useful’ score means we can leverage only a portion of it, and ‘Not useful’ given to the results that required significant prompt refinement or result editing, defeating the purpose.
• Self-sufficiency (🤖 Self-sufficient, 🔧 Not self-sufficient). We defined self-sufficiency as the ability to produce results that required little to no editing or quality control on our part.

Summary of findings

Across the 15 tasks reviewed below, we got the following results:

Usefulness:

• 👍 Useful – 4 tasks
• 🤔 Potentially useful – 7 tasks
• 👎 Not useful – 4 tasks

Self-sufficiency:

• 🤖 Self-sufficient – 1 task
• 🔧 Not self-sufficient – 14 tasks

Takeaway: We emerged from this test a little underwhelmed, but also with a clearer understanding of the specific areas where ChatGPT and Bard can prove valuable.

In our assessment, AI’s best use cases for advertising are in copywriting and creative brief ideation, given their comparatively lower need for extensive refinement and quality control.

Unfortunately, when it comes to technical tasks ranging from keyword research to data analysis, both ChatGPT and Bard showed limitations, often providing only potentially useful information for practical use, meaning they should only be used as an additional step in the process, rather than replacing the process altogether.

Lastly, almost none of the results are self-sufficient and require diligent quality control to ensure accuracy.

Get the PPC insights we share with our clients—for free.

✅ Latest PPC news

✅ Our wins & lessons

✅ Creative inspiration

✅ Useful tools & tips

Keyword research

 🤔 Potentially useful

 🔧 Not self-sufficient

ChatGPT: Can be helpful for generating additional topic ideas for further research in Google Keyword Planner.

Note: we’ve found ChatGPT’s estimations of search traffic to be inaccurate across all of our clients and wouldn’t recommend basing assumptions on its estimates.

ChatGPT

vs Google Keyword Planner

Bard: The new topic ideas it provided were also useful, albeit tighter in nature, limiting expansion. 

Its search volume estimates, surprisingly, were completely off-base.

Negative keywords

👍 Useful

🔧 Not self-sufficient

ChatGPT: While not self-sufficient (or fully accurate as seen in the screenshot), we’ve found ChatGPT to be genuinely helpful for finding negative keywords for exclusions.

Worth noting, however, that it’s mostly useful for top-level exclusions and not for analyzing granular ad groups’ search terms – something we do on a weekly basis – as it doesn’t understand adjacent topics’ relation as well as a person would and applies overly restrictive exclusions.

Bard: The negative keywords were more specific and offered new options, but didn’t offer the broader “we-didn’t-consider-these” options.

Grouping keywords

🤔 Potentially useful

🔧 Not self-sufficient

ChatGPT: Can be helpful to improve the speed of ad group creation. 

However, it consistently leaves a number of keywords without a group, and mistakenly groups certain keywords together (such as “sofa sets” in the example screenshot, which deserves its own ad group as it would need a separate product focus on sets of furniture). 

It’s not applicable for granular keyword grouping – the way we set up campaigns.

Bard: Bard approaches it differently, providing new grouping ideas, but it also throws in its own keywords :O (not always bad, but it means you’ll have to review every keyword after grouping)

Ad copy

👍 Useful

🔧 Not self-sufficient

ChatGPT: Safe to say it struggles to stay within the character limits (and organize the copy into single-angle approaches) which is very important for most ad channels (Google and Microsoft Ads more so than others) and our own A/B testing efforts. 

Still, this is where we believe ChatGPT shines as, with appropriate guidance, it’s able to provide a large number of variations for different angles/personas/platforms (and neatly organize them into columns for easier analysis).

Having said that, the copy is rarely usable as-is and often needs additional refinement.

Bard: Same issues with staying within character limits, but also more uninspiring ad copy, in my opinion.

Not to say you can’t get more out of it with some prompt refinement 😉

 Pro Tip: Leverage “organize into tables, add X columns to the right and fill them out using different tones of voice” prompts to generate more copy at scale and in a more readable format.

ChatGPT ad copy script

👎 Not useful

🔧 Not self-sufficient

ChatGPT: We’ve tested the RSA-writing script that is currently circulating on LinkedIn. The script reviews search ads in Google Ads and suggests additional headlines where the ad doesn’t have a maximum number of headlines or descriptions.

Find the script here: https://searchengineland.com/google-ads-script-gpt-responsive-search-ads-395548

We’ve found that:

1. This approach doesn’t work with our approach of A/B testing different angles as the script ignores specific ad copy focuses and often produces generic ads
2. It often produces copy that’s over the limit (as already established previously), making the variant unusable as there’s no way to relay the same information in 30 characters.

We’re also planning on verifying the test results provided in the article by running A/B tests across a number of clients, as we haven’t noticed the number of headlines affecting results. In fact, Google often picks just 3-4 headlines and rotates through those.

Bonus: Analyzing reviews for ad copy

🤔 Potentially useful

🔧 Not self-sufficient

ChatGPT or Bard can help with analyzing own and competitor reviews in bulk to:

1. Identify common issues or complaints that should be addressed in the copy
2. Discover emerging trends or topics to be added as keywords

This, however, requires additional preparation as the data needs to be scraped and published or pasted (limited) into ChatGPT/Bard for reviewing.

Targeting research

🤔 Potentially useful

🔧 Not self-sufficient

ChatGPT: Targeting research with ChatGPT could be useful when doing top-level market research, as well as to uncover new targeting angles for audience targeting/ad copy/creatives/keywords. 

Further refinement is key to uncovering additional ideas. As is the case with all tasks so far, though, none of it can be used as-is and requires verification and further analysis to be practically applicable.

Similarly generic in nature, ChatGPT’s channel suggestions can help provide general ideas for ad networks worth considering (Houzz ads were a new one for me), but should not be acted on without proper research of network specifics (i.e. ChatGPT won’t tell you that at LinkedIn’s $5-10 CPCs, you’re not likely to make a profit promoting discounted outdoor furniture)

Bard: Top-level, but surprisingly detailed results from Bard.

Auditing / Optimization

 👎 Not useful

🔧 Not self-sufficient

There’s no way to connect ChatGPT or Bard directly to an ad account, so any kind of auditing would require feeding it considerable amounts of data first, which A – is limited due to its ability to process data and B – requires teaching it on what to look for exactly. 

We’d need to build a ChatGPT for ad audits.

Next unicorn startup idea?

Creative brief ideas

👍 Useful

🔧 Self-sufficient (well, kind of)

ChatGPT: Another useful application is coming up with new creative brief ideas (albeit a bit generic). The results are original and well-adjusted for brands, provided we give it enough context and specify the focus. The ad layout refinement prompts are also genuinely helpful for brief specifics.

Bonus: it can even generate storyboards for video ads, although we haven’t tested implementing this use case specifically.

Bard: Definitely less creative flair compared to ChatGPT. We couldn’t get it to produce specific ideas even with prompt refinement.