Optimizing websites for AI

Is AIO (Artificial Intelligence Optimization) coming after SEO?

In this blog post, we take a look at what needs to be considered in future web development so that the content of websites can be read by artificial intelligence in the best possible way.

This article is not about how to optimize a website with the help of AI (artificial intelligence), but we have looked at how a website must be structured so that artificial intelligence can read, process and ultimately display the content of a website in the best possible way.

Since the number of users who use AI as a replacement for search engines such as Google Search is constantly growing, website optimization should no longer just focus on SEO, but also take AI into account - we have named it AIO (Artificial Intelligence Optimization). Especially because ChatGPT also includes the web search “SearchGPT”, which enables real-time searches on websites and is really cool.

SEO vs. AI optimization

In SEO, the website or the individual subpages are largely geared towards certain keywords or phrases that future customers enter into a search engine or that users are expected to enter. For some time now, especially since language assistants have been increasingly used, there has been a tendency for search engine users to search for entire sentences or questions rather than just specific keywords. For this reason, it has long been recommended to create FAQs on the website that cover possible questions that users could type into the search engine to improve search engine retrieval.

When optimizing AI, we need to think about what our target group could ask the AI so that we appear on the AI's screen at all.

Just as it has long been the case with voice assistants, where the first 3 positions in the search engine are interesting, as perhaps no more than 3 results are read out, so it is now with AI search. When we talk to ChatGPT and want a listing of something, such as website monitoring tools, the first three tools that ChatGPT thinks are interesting are listed. This means we have to think really carefully about how we build our website in order to create relevance for a topic from an AI perspective. 

Where we at Google at least have clues such as the 200 ranking factors for Google search, we at ChatGPT are rather in the dark as to exactly which factors are used here for the “ranking”. But we'll get to the bottom of this further down in the article and we'll tell you this much in advance - take a look at the guidelines for Bing webmasters ;)

Example based on SYSSY

We had a chat with ChatGPT's voice assistant to see how well the “ranking” is for the SYSSY website in the ChatGPT search. 

We asked the following questions:

What TYPO3 monitoring tools are available?
ChatGPT shows a list of several tools, SYSSY in 2nd place.

What features does SYSSY have?
ChatGPT lists all features, but adds a feature that does not exist, namely backups.

Does SYSSY also work with other content management systems?
ChatGPT correctly lists all CMS with which SYSSY can communicate.
The list of sources is long - apart from the TYPO3 website (typo3.org) via the WordPress website (wordpress.org), around 10 internal page visits to the SYSSY website are listed, including the homepage, manual, FAQ, features. The pages are accessed in both German and English and analyzed live.

Who developed SYSSY?
Unfortunately, ChatGPT provides a completely wrong answer here, namely a company from Norway. The information about SYSSY Online GmbH is only correct when asked again.
For questions about the company, ChatGPT retrieves data from Creditreform, Xing, FirmenABC, wirtschaft.at, Huddlex and many more.

What is the name of the managing director?
This question will be answered correctly. Xing then provides further information about the person and their professional career, all correct so far.

What are the price models for SYSSY?
The price models are read from the website in real time and we receive correct information. Thanks to the live analysis, even recent changes are taken into account.

The result of the search is surprisingly good with a few exceptions - hallucinating the additional feature and specifying the wrong developer company - and the information is provided in great detail. We wonder how exactly the ranking for the TYPO3 monitoring tools comes about and how the data gets into ChatGPT, how up-to-date it is and what happens in the background. If anyone needs to know (something), it's ChatGPT, so we asked ChatGPT directly and summarized the findings.

Does ChatGPT access current data for certain questions?

  1. Without web search::

    • ChatGPT draws on pre-trained knowledge. This knowledge comes from public information that was available up to the cut-off date of the data (October 2023).
    • In this case, the information may be outdated or incomplete, as no new data is retrieved without internet access.
  2. With web search:

    • If the answer to a question needs to be potentially up-to-date, e.g. for price models, tariffs or new functions, a web search is launched. This makes it possible to provide up-to-date and relevant information in real time.

How can you optimize a website for ChatGPT?

If you want your website content to be found quickly and effectively by models like ChatGPT (or similar AI systems), there are some proven optimization strategies that also support general SEO and data delivery practices. Here are the key steps:

1. Ensure search engine friendliness

AI systems obtain their data from search engines and indexed sources. The following important basics should be considered:

  • Ensure indexability: Ensure that your website can be crawled by search engines (no restrictions through robots.txt or noindex meta tags).
  • Provide a sitemap: Upload an XML sitemap to Google Search Console and Bing Webmaster Tools so that search engines can easily find all relevant content.
  • Fast loading times: Optimize the loading speed of your website. Tools such as Google PageSpeed Insights help with this.

2. Clear and structured content

AI models benefit from clearly structured, easy-to-understand information:

  • Clear headings and paragraphs: Use H1, H2, and H3 tags for a clear hierarchy of content.
  • Semantic markup in the text: Use the right HTML structure for quotes, lists, paragraphs, ...
  • Simple writing style: Use short sentences and avoid complex technical language if the target group does not expect it.
  • Position core information early: Important information belongs at the beginning (e.g. on the homepage or prominently in articles).

3. Use structured data (schema markup)

Structured data helps search engines (and therefore AI tools) to better understand what the page is about:

  • Implement Schema.org markup for relevant content (e.g. products, FAQs, articles).
    Example: Use the FAQPage schema for frequently asked questions that can be picked up directly by AI systems.

4. Regular updating of the content

Updated content is preferred by search engines and AI systems:

  • Specify update dates: Show the last update date (e.g. “Last updated on [date]”).
    Timely editing: Maintain content regularly to keep it up to date.

5. Unique URLs and metadata

  • Speaking URLs: Use clear, descriptive URLs (e.g. example.com/price-models-syssy).
  • Meta tags: Fill meta descriptions and title tags with relevant keywords.

6. Make information easily accessible

  • Content via APIs: Provide important content via APIs or easy-to-read HTML pages.
    Open FAQs: Important questions and answers can be integrated directly into the website.

7. Mobile optimization

Since many visits are made on mobile devices, mobile optimization is essential:

  • Use responsive design.
  • Avoid pop-ups that hinder access to content.

8. Focus on AI-optimized content

If you want to optimize specifically for AI systems:

  • Machine-readable content: Make sure text is available without lots of scripts or hard-to-access formats.
    No barriers: Content behind paywalls or login pages is difficult for AI systems to capture.

9. Testing and monitoring

Use tools to ensure that search engines (and AI) understand your website:

  • Google Search Console: Check indexing issues and crawling stats.
  • Schema Markup Testing Tools: Test structured data with tools like Google's Rich Results Test.

Conclusion

Optimizing for AI starts with an SEO-friendly website and continues with clearly structured, easily accessible data. By updating regularly, using structured data and providing high-quality content, you can ensure that AI systems like ChatGPT can easily find your website and extract relevant information efficiently.

ChatGPT and JavaScript

We have noticed that the visits of ChatGPT were not recorded by Matomo tracking. This means that the visits made by ChatGPT are not visible to us via the tracking. This is not a problem for visitor analysis, but it does raise the question of whether ChatGPT is able to execute JavaScript at all. In times of dynamic reloading of content, this is a relevant question. Google executes JavaScript and displays content that is reloaded, but does ChatGPT do this?

Is JavaScript executed on the website when ChatGPT crawls?

The answer is NO! ChatGPT does not execute JavaScript itself. When it accesses real-time information from the web (when using web search), it pulls the information from the HTML source code of the pages as provided by the search engine or web crawlers. The actual execution of JavaScript is not part of the process.

Restricted display of dynamic content: Content that is only generated by JavaScript (e.g. content that is only added after loading via AJAX) is generally not accessible for ChatGPT. If your website generates important content dynamically, you should provide a server-side rendering strategy or pre-rendered content.

AI models such as ChatGPT obtain a lot of information from search engine databases such as Google or Bing. These search engines can execute JavaScript and provide indexed content, i.e. content generated by JavaScript may be accessible to ChatGPT if the search engine has already indexed it.

To test the visibility for crawlers, you can use the Google Search Console or tools such as Screaming Frog to see which content is available without JavaScript. Alternatively, you can also deactivate JavaScript in the browser and take a look at the page.

Which services are used for the ChatGPT search?

The platform behind ChatGPT uses different technologies and services for web search, depending on the implementation and provider. Here is an overview of the possible and typically used services for web search:

1. Primary search services

  • Bing: One of the main providers of real-time web searches, often used by OpenAI and Microsoft-based platforms.
  • DuckDuckGo: A privacy-oriented search service that also compiles content from multiple sources.

2. Alternative search services

  • Yahoo Search: A service that works closely with Bing and delivers similar search results.
  • Brave Search: A newer provider that places value on data protection and uses its own search indices in some cases.
  • MetaGer: A meta search engine that combines search results from different sources.
  • Yandex: Often used in Russian-language contexts or for specific geographical regions.

3. APIs and metasearch services

  • SerpAPI:  A service that can aggregate results from various search engines (including Google, Bing and Yahoo) and provide APIs.
  • RapidAPI: A marketplace for various API services, including those for web searches.
  • Searx: An open source metasearch engine that combines results from multiple search services.

4. Industry or specific services

  • Wolfram Alpha: A computerized knowledge system specializing in structured data and computation.
  • Wayback Machine (Internet Archive): Access to archived websites can be used for older content or historical information.
  • LexisNexis: Available in some scenarios for legal or scientific information.

5. Targeted specialist search

  • Google Scholar: For scientific content or research papers, provided the platform implements access to them.
  • PubMed: For medical and life science topics.
  • ArXiv: For academic articles in the fields of physics, mathematics and computer science.

Which services are actually used?

The specific choice of search services depends on the context:

  • By default: The web search in ChatGPT implementations often uses Bing as a basis.
  • Flexibility: Depending on the platform or provider, other APIs or specialized search services can be integrated.

Is Bing the main source for the results in the ChatGPT search?

Bing is one of the main sources of real-time web search results when ChatGPT retrieves up-to-date information from the Internet. This is due to the partnership between Microsoft and OpenAI and the integration of Bing into the infrastructure of platforms using ChatGPT.

Why Bing as the main source?

  1. Microsoft partnership:

    • Microsoft is a major supporter of OpenAI and provides resources through Azure. Bing is part of this ecosystem.
    • Through this partnership, Bing is the preferred primary search engine.
  2. Reliable API:

    • Bing provides a powerful API for accessing real-time search results.
    • The API enables fast and scalable integration with systems such as ChatGPT.
  3. Data access and indexing:

    • Bing has comprehensive indexing of websites worldwide and provides high-quality, up-to-date data.
  4. Alternative to Google:

    • Google is a direct competitor and does not provide a direct API for real-time search to the general public (except through paid services like the Custom Search JSON API). Bing, on the other hand, is more available and more integrated.

Let us support you with website monitoring

SYSSY checks your website daily:

  • Check the CMS version
  • Check of all extensions/plugins
  • Monitoring of PHP and MySQL version
  • Performance check, SEO check & GDPR check of the homepage

Test now for free

Conclusion - Bing Up your Website!

While there is no specific “ranking” for ChatGPT, you can ensure that your website is well optimized through traditional SEO analysis and content audits. A combination of SEO best practices, structured data and open, accessible content will maximize the chances of AI models recognizing and correctly rendering your content.

It will be even more important in the future that websites have a semantic structure and good performance. Unnecessary bells and whistles should be avoided, especially when website data is read in real time.

However, it can be assumed that the Bing search will be used as the main source of information for the ranking in ChatGPT. Firstly, because Microsoft is a major investor in OpenAI and secondly, because ChatGPT also uses resources from the Azure infrastructure.

Furthermore, Bing has the advantage that there is an API connection to the search results, which is why search engines such as Ecosia and DuckDuckGo also use data from Bing search via the API.

We have been recommending for years to monitor websites not only via Google Search Console, but also via Bing Webmaster Tools, as Bing is becoming increasingly widespread.

The most important points summarized:

  • Qualitative content: Create relevant content for your target group in the form of precise product or service descriptions - as comprehensive and clear as possible
  • Semantic data structure: Structure the data well, i.e. make sure your website has a semantic structure
  • Good performance: Make sure that the website loads quickly and that no unnecessary data is loaded.
  • Machine-readable content: Make sure that texts are available without lots of scripts or formats that are difficult to access.
  • No barriers: Content behind paywalls or login pages can be difficult for AI systems to capture.
  • Use APIs: Make important content available via APIs or easy-to-read HTML pages.
  • Open FAQs: Important questions and answers can be integrated directly into the website. Clarify all questions about offers, features, prices and support and all other things that could be important and interesting for potential customers. A FAQ page is ideal for this.
  • Timely editing: Maintain content regularly to keep it up to date, as up-to-date data is preferred by search engines and AI systems.
  • Talk to ChatGPT about your own website. Ask questions about the product and check if the answers are correct. If answers are not correct, question the information provided on your website and revise it if necessary. ChatGPT will hallucinate if it does not find enough information, i.e. it is up to us to provide enough information to avoid hallucinating.

We support you with website optimization!!

We take a close look at your website:

  • SEO check
  • GDPR check
  • Performance check
  • Usability check
  • Accessibility check
  • Check for AI optimization

Inform now

[Translate to English:] Ingrid Stürmer

About the author
Ingrid Stürmer has been a web developer and TYPO3, WordPress and SEO specialist for many years.


This might also interest you

Is AIO (Artificial Intelligence Optimization) coming after SEO?

In this blog post, we take a look at what needs to be considered in future web…

Read more

We tried to create a website according to design specifications using only artificial intelligence (using ChatGPT). You can read how it worked and…

Read more
Why TYPO3 Monitoring is important

Regular TYPO3 monitoring provides a good overview and enables you to react quickly to problems and security gaps.

This article explains what the…

Read more