Applied AI, LLMs, AI Agents, and the Reality of German Mittelstand Adoption with Markus Böge

Episode Summary

In this episode of The Bhairav Show, Suhas Bhairav speaks with Markus Böge, a founder and CTO in Germany with more than 25 years of experience across software development, enterprise software products, startups, scaleups, SAP, and applied AI. The conversation is a grounded look at what is actually happening with LLMs, generative AI, and AI agents inside German companies, especially the Mittelstand. Markus begins by describing a career built around software product lifecycles. His experience spans successful enterprise products that lived for many years. This gives him a practical view of software as something that grows together with customers, teams, and business reality. A major theme of the episode is Markus's approach to product development: first who, then what, then how. Instead of starting with technology or jumping directly into solutions, he begins by understanding the customer, the target group, and the real patterns across multiple prospects. For enterprise software, he explains that five to six pre-sales conversations can often reveal what customers actually need, not just what they initially say they want. This customer-first thinking becomes especially important when companies ask for AI agents or generative AI solutions. Markus explains that AI adoption in Germany is not uniform. Some companies are still in the early stages of cloud adoption, with Microsoft 365 only recently introduced and many AI-related features deliberately switched off. Other organizations are testing their first AI-enabled software features but still need the ability to deactivate them because internal policy, compliance, or workers council agreements are not yet ready. At the other end of the spectrum, Markus describes mature enterprises with tens of thousands of employees already running internal knowledge systems, RAG-based platforms, and compliant ChatGPT-like tools hosted in Germany. The episode then moves into where real AI value lies. Markus is clear that generic chat with company content is a useful first step, but it is not the highest leverage. Many companies deploy systems that let employees ask questions over documents, databases, or internal knowledge, but users often do not know what to do with them. The deeper opportunity is to find business problems that can only be solved by AI, or can be solved many times better with LLMs. This is where AI stops being a bolt-on feature and becomes part of the product's core value. Markus also highlights the importance of custom GPTs, custom prompts, and internal AI champions. The people who create useful prompt templates, process definitions, or Excel templates inside companies are often the same people who can turn generic LLM access into practical workflows for their teams. This matters because chat interfaces create an empty-canvas problem. If employees are simply given a blank chatbot and told to use AI, most will struggle. Guided, narrow, specific AI assistants can create much more value because they ask the right questions and solve one focused problem at a time. A major part of the discussion focuses on GDPR, compliance, and European AI hosting. Markus explains the ideal case for German enterprises: a German-based provider that can host both the application and the LLM component without a US company relationship. But the technical reality is more complicated. Purely German or European providers may exist, but they often do not provide access to the flagship models from OpenAI, Anthropic, or other leading model providers. US hyperscalers can provide powerful models hosted in Germany, but the relationship to a US company may still be a concern or even a deal breaker for some customers. Performance and cost also enter the equation. Hosting top-tier models in Europe or Germany may create latency, GPU availability, and scaling tradeoffs. Some hyperscalers can offer Frankfurt hosting, but may still need the option to route traffic to other European regions such as Ireland during spikes or capacity constraints. Looking ahead, Markus does not believe that normal business users will run dozens of autonomous agents on their machines 24/7. That may be a trend among technical users, but enterprise employees want to do their jobs better, not manage swarms of agents. Instead, he sees two likely paths: internal AI platforms that become as broadly useful as Excel, and SaaS products that embed AI in more meaningful ways. The strongest products will not merely include an AI button for marketing. They will solve problems that do not make sense without AI. In the best case, switching off the AI feature should not be a normal discussion because the product's value is fundamentally tied to what AI enables.