In a recent DBE vlog, Natalie Lorenzo and I talked about ChatGPT’s AI limitations from a “search” perspective. We discussed two primary caveats to using ChatGPT for searches:
- ChatGPT’s database stopped receiving inputs in September 2021 so the information is now almost two years old – a lifetime in the internet world.
- The free version of ChatGPT can not access the internet so it can’t look at websites or perform internet-related research.
Enter Bing Chat that leverages GPT-4 AI (not to be confused with ChatGPT), is built into Microsoft Edge, is integrated with Microsoft’s search engine, and can, therefore, perform web searches. And Bing Chat’s search “results” include detailed responses along with citations and clickable links to learn more.
Clickable links! That means trackable links in analytics, right? Alas, the answer is not exactly. Here’s the result of a Bing Chat search conversation that led me to click on DBE’s link citation to learn more:
At the same time as I did the search, I was monitoring DBE’s Real Time GA4 Google Analytics results and this is what came through:
The data was captured but it was sourced as “direct/none” – not “bing/search” – as I hoped it would. This means any conversion action taken on the DBE website would not be attributed directly or indirectly (as an assisted conversion) to the Bing Chat search.
As an SEO-er, visits and conversion data attributable to organic search are important success metrics used to determine ROI. The lack of visibility into this data also inhibits the ability to gauge the value of optimizing for conversational search engines (e.g., Bing Chat) in addition to the more traditional search engines.
I raised this question of “chattribution” during the recent SMX Advanced virtual conference. And while the question got a lot of “likes” from fellow attendees, no one seemed to have an answer for how or when this “chattribution” data gap will be filled. Will, of course, let you know of any developments.
PS – A separate note on Bing Chat’s reliability: Bing Chat’s response about DBE mentions “clients such as American Express, Disney, Johnson and Johnson, and more.” J&J was a DBE client for a number of years but Amex and Disney were not. Another example of a chatbot hallucination (or in this case “Fantasia”).
