<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>AI on Azure Readiness starts here...</title><link>https://www.pdtit.be/tags/ai/</link><description>Recent content in AI on Azure Readiness starts here...</description><generator>Hugo -- gohugo.io</generator><language>en-us</language><lastBuildDate>Sat, 14 Mar 2026 09:30:00 -0700</lastBuildDate><atom:link href="https://www.pdtit.be/tags/ai/index.xml" rel="self" type="application/rss+xml"/><item><title>Azure SRE Agent: Bringing Agentic AI to Site Reliability Engineering on Azure</title><link>https://www.pdtit.be/post/azure-sre-agent-intro/</link><pubDate>Sat, 14 Mar 2026 09:30:00 -0700</pubDate><guid>https://www.pdtit.be/post/azure-sre-agent-intro/</guid><description>&lt;p&gt;If you have been following me for a while, you know I&amp;rsquo;m a big fan of Azure reliability. It was the main topic I presented on several years ago (in the early days of Azure - if that sounds right?) and also mapped with a big part of my job as Azure Architect, consultant and trainer.&lt;/p&gt;
&lt;p&gt;I got amazed at the end of 2021 by &lt;strong&gt;&lt;a class="link" href="https://learn.microsoft.com/azure/chaos-studio/chaos-studio-overview" target="_blank" rel="noopener"
 &gt;Azure Chaos Studio&lt;/a&gt;&lt;/strong&gt;, a service that allows you to inject faults against your Azure workloads (preferably production!), to make them more stable, more reliable.&lt;/p&gt;
&lt;p&gt;But then came Generative AI, and Agentic InfraOps/DevOps. Welcome &lt;strong&gt;Azure SRE Agent&lt;/strong&gt;, which was in public preview for a few months, but went GA earlier this week. I played with it since its early inception, and thought the GA - with a lot of cool new updates - was a good time to dedicate a blog article to it.&lt;/p&gt;
&lt;h2 id="introduction-what-is-azure-sre-agent"&gt;Introduction: What Is Azure SRE Agent?
&lt;/h2&gt;&lt;p&gt;Modern cloud systems are increasingly distributed, dynamic, and failureâ€‘prone by design. While DevOps practices have optimized delivery velocity, operational reliability still demands significant human effort. Particularly during incident response, root cause analysis, and postâ€‘incident followâ€‘up. (Having been on the consultant side in physical and cloud environments since 1996, especially the outages and trying to fix issues is what got me into training - is what my wife says when you ask her why I love training so much, lol. She might be right&amp;hellip;)&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Azure SRE Agent&lt;/strong&gt; is an &lt;strong&gt;AIâ€‘powered reliability assistant&lt;/strong&gt; designed to automate and augment Site Reliability Engineering practices for Azure workloads. It continuously observes telemetry (metrics, logs, traces), understands Azure resource topology, correlates incidents with recent changes, and assists with, or ask human approval to execute, remediation steps.&lt;/p&gt;
&lt;p&gt;Unlike traditional monitoring or AI-Ops tools, Azure SRE Agent operates as an &lt;strong&gt;agentic system&lt;/strong&gt;:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;It reasons over multiple data sources simultaneously.&lt;/li&gt;
&lt;li&gt;It maintains contextual awareness of your Azure environment.&lt;/li&gt;
&lt;li&gt;It can take action via Azure CLI and REST APIs, subject to explicit approval.&lt;/li&gt;
&lt;li&gt;It integrates natively with incident management and developer workflows.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In effect, Azure SRE Agent acts as a &lt;strong&gt;virtual SRE teammate&lt;/strong&gt;, reducing operational toil and lowering mean time to resolution (MTTR) while preserving human oversight. (If we had agents in the early 2000&amp;rsquo;s, maybe I would still be a technical consultant instead of technical trainer, hmmmm)&lt;/p&gt;
&lt;h2 id="architecture-and-core-capabilities"&gt;Architecture and Core Capabilities
&lt;/h2&gt;&lt;p&gt;At a high level, Azure SRE Agent combines four capability pillars:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Continuous Observability Ingestion&lt;/strong&gt;&lt;br&gt;
The agent consumes signals from Azure Monitor, Log Analytics, Application Insights, and supported external observability systems to build a live understanding of system health and dependencies. The &lt;strong&gt;real benefit for me&lt;/strong&gt; here, is that organizations already have everything in place. So the adoption goes smooth. And the data the agent relies on, feels familiar.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Intelligent Diagnosis and Correlation&lt;/strong&gt;&lt;br&gt;
When an alert or anomaly occurs, the agent correlates telemetry with:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Recent deployments or configuration changes&lt;/li&gt;
&lt;li&gt;Resource topology and dependencies&lt;/li&gt;
&lt;li&gt;Historical incident patterns&lt;br&gt;
This enables accelerated root cause analysis without manual log spelunking. (does that exist as a word?)&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Automated and Approvalâ€‘Gated Remediation&lt;/strong&gt;&lt;br&gt;
Azure SRE Agent can execute operational actions. Think of scaling, restarting services, or reverting deployments. Or basically anything that relies on &lt;strong&gt;Azure CLI and REST APIs&lt;/strong&gt;. All write actions are gated by RBAC and explicit approval, ensuring governance and control. (If you don&amp;rsquo;t trust the commands it suggests, don&amp;rsquo;t approve the action&amp;hellip;)&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Workflow and Developer Tool Integration&lt;/strong&gt;&lt;br&gt;
The agent integrates with Azure Monitor alerts, GitHub, Azure DevOps, ServiceNow, and PagerDuty, allowing incidents to flow naturally into existing operational and engineering processes. (I have to be honest, I didn&amp;rsquo;t go that far yet to integrate with source control, probably another blog post in the near future)&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 id="setup-and-deployment"&gt;Setup and Deployment
&lt;/h2&gt;&lt;h3 id="prerequisites"&gt;Prerequisites
&lt;/h3&gt;&lt;p&gt;To deploy Azure SRE Agent, the following prerequisites must be met:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;An active Azure subscription&lt;/li&gt;
&lt;li&gt;Permissions to assign RBAC roles (&lt;code&gt;Microsoft.Authorization/roleAssignments/write&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;Network access to the &lt;code&gt;*.azuresre.ai&lt;/code&gt; domain&lt;/li&gt;
&lt;li&gt;Deployment in a supported region (Preview was available in EastUS2, SwedenCentral and AustraliaEast), you might check the docs for accurate updates&lt;/li&gt;
&lt;/ul&gt;

 &lt;blockquote&gt;
 &lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: I didn&amp;rsquo;t find any information on how to automate the deployment using bicep or az cli - have to come back to that at some point&lt;/p&gt;

 &lt;/blockquote&gt;
&lt;h3 id="creating-an-azure-sre-agent"&gt;Creating an Azure SRE Agent
&lt;/h3&gt;&lt;ol&gt;
&lt;li&gt;In the Azure Portal, search for &lt;strong&gt;Azure SRE Agent&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Select &lt;strong&gt;Create Agent&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Create or select a &lt;strong&gt;dedicated resource group&lt;/strong&gt; for the agent itself (I would recommend deploying this separate from application resources).&lt;/li&gt;
&lt;li&gt;Choose the region.&lt;/li&gt;
&lt;li&gt;Associate one or more &lt;strong&gt;resource groups to monitor&lt;/strong&gt;.&lt;br&gt;
The agent automatically gains visibility into all resources within those groups.&lt;/li&gt;
&lt;li&gt;Complete the deployment and wait for the agent to initialize.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Once deployed, the agent exposes a &lt;strong&gt;chatâ€‘based interface&lt;/strong&gt; in the Azure Portal, allowing engineers to interact using natural language to investigate and manage incidents.&lt;/p&gt;
&lt;h2 id="using-azure-sre-agent"&gt;Using Azure SRE Agent
&lt;/h2&gt;&lt;p&gt;After the baseline deployment of the agent, it&amp;rsquo;s nothing more than &lt;strong&gt;running prompts&lt;/strong&gt;. Using natural language, asking generic or more-specific questions, and off it goes :)&lt;/p&gt;
&lt;p&gt;To test this out, I deployed an Azure App Service, connecting to CosmosDB using Managed Identity. After testing the app, I removed the App Service Managed Identity to simulate the issue.&lt;/p&gt;
&lt;p&gt;I opened SRE Agent and asked:&lt;/p&gt;
&lt;p&gt;&lt;code&gt;can you investigate my app service outage&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;This is what it came back with:&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2026-03-14-090607.png"
 
 
 loading="lazy"
 
 alt="Investigating_App_Service"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;Followed by &lt;strong&gt;looking into the metrics&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2026-03-14-090726.png"
 
 
 loading="lazy"
 
 alt="Investigating_Metrics"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;To then provide &lt;strong&gt;a summary&lt;/strong&gt; of the findings and observations, &lt;strong&gt;INCLUDING CHART VIEWS&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2026-03-14-090957.png"
 
 
 loading="lazy"
 
 alt="Investigating_charts"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;Detailed Root Cause Analysis&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2026-03-14-091106.png"
 
 
 loading="lazy"
 
 alt="Investigating_summary"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;and detailed description of &lt;strong&gt;what happened&lt;/strong&gt; and &lt;strong&gt;Recommended actions&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2026-03-14-091159.png"
 
 
 loading="lazy"
 
 alt="Investigating_summary"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;It identified the root cause being an &lt;strong&gt;identity&lt;/strong&gt; problem, where the Web App could not connect to Cosmos DB.&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2026-03-14-091431.png"
 
 
 loading="lazy"
 
 alt="Investigating_summary"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;To wrap it up with a &lt;strong&gt;Diagnosis Complete - Data Unreachable Root Cause&lt;/strong&gt; report (in table format), including &lt;strong&gt;potential fix steps&lt;/strong&gt; (Isn&amp;rsquo;t that amazing?? I think it&amp;rsquo;s just brilliant&amp;hellip;!!!)&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2026-03-14-091606.png"
 
 
 loading="lazy"
 
 alt="Investigating_diagnose_complete"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;From there, it asked me if it was OK to move on and assist with fixing the problem. Using the same response I would tell when talking to a colleague, I said&lt;/p&gt;
&lt;p&gt;&lt;code&gt;Yes, go ahead and assist me with fixing this problem using the described steps&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;(I&amp;rsquo;m pretty sure just saying &amp;ldquo;yes&amp;rdquo;, or &amp;ldquo;sure&amp;rdquo; or &amp;ldquo;OK&amp;rdquo; or &amp;ldquo;YOUCANDOIT&amp;rdquo; might have worked too&amp;hellip;)&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2026-03-14-091835.png"
 
 
 loading="lazy"
 
 alt="acknowledge_fix"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;The above screenshot was taken after the process completed, but remember the SRE Agent can only perform actions when you as the &lt;strong&gt;human-in-the-loop&lt;/strong&gt; acknowledges the approval.&lt;/p&gt;
&lt;p&gt;Smoothly, it came back with &lt;strong&gt;Issue resolved&lt;/strong&gt;. Including a summary of the steps taken&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2026-03-14-092204.png"
 
 
 loading="lazy"
 
 alt="acknowledge_fix"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;Well done SRE Agent!!&lt;/p&gt;
&lt;h2 id="summary"&gt;Summary
&lt;/h2&gt;&lt;p&gt;Azure SRE Agent is - apart from GitHub Copilot - my next favorite use case for Generative AI. Having experienced the challenges of cloud workload outages myself for years, spending hours, sometimes days, digging in, gathering metrics and logs, pinpointing the root-cause,&amp;hellip; (which sort of was a lucrative business if I think back about it&amp;hellip;), I think this is an &lt;strong&gt;amazing&lt;/strong&gt; service to be added to your Azure environment. Even when you don&amp;rsquo;t trust it at first (actually, why not?) to take actions, having that AI assistant next to you to help you with the investigation, the outage analysis,&amp;hellip; will be a big time-saver. Which means, your workload will be back up-and-running faster too.&lt;/p&gt;
&lt;p&gt;And I didn&amp;rsquo;t talk about the source control integration with GitHub or Azure DevOps. I didn&amp;rsquo;t mention the notifications through Outlook or Teams. I didn&amp;rsquo;t explain the expansion to other data scenarios, third-party monitoring tools such as Grafana, DataDog,&amp;hellip; damn, there will be a lot of blog posts on Azure SRE Agent in the near-future I&amp;rsquo;m afraid.&lt;/p&gt;
&lt;p&gt;Also, if you want some inspiration to play with this, have a look at the &lt;a class="link" href="https://microsoftlearning.github.io/mslearn-devops/Instructions/agentic/03-optimize-azure-reliability-using-sre-agent.html" target="_blank" rel="noopener"
 &gt;Microsoft Learn lab - Optimize Azure Reliability using SRE Agent&lt;/a&gt; I published recently.&lt;/p&gt;
&lt;p&gt;If you deployed it and use it in your environment, please let me know. Happy to hear your stories!&lt;/p&gt;
&lt;p&gt;&lt;a class="link" href="https://www.buymeacoffee.com/pdtit" target="_blank" rel="noopener"
 &gt;&lt;img src="../images/screenshot-2026-03-14-17f576e7.png"
 
 
 loading="lazy"
 
 alt="BuyMeACoffee"
 
 
 
 &gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Cheers!!&lt;/p&gt;
&lt;p&gt;/Peter&lt;/p&gt;</description></item><item><title>How Spotify disappointed me with AI Artists</title><link>https://www.pdtit.be/post/spotify-disappointed-me-with-ai-artists/</link><pubDate>Sat, 29 Nov 2025 00:00:00 +0000</pubDate><guid>https://www.pdtit.be/post/spotify-disappointed-me-with-ai-artists/</guid><description>&lt;p&gt;I&amp;rsquo;m usually pretty excited about AI and more specifically about Generative AI, especially with &lt;strong&gt;&lt;a class="link" href="https://copilot.microsoft.com" target="_blank" rel="noopener"
 &gt;Microsoft Copilot&lt;/a&gt;&lt;/strong&gt;, &lt;strong&gt;&lt;a class="link" href="https://github.com/copilot" target="_blank" rel="noopener"
 &gt;GitHub Copilot&lt;/a&gt;&lt;/strong&gt; and &lt;strong&gt;&lt;a class="link" href="https://azure.microsoft.com/products/ai-foundry" target="_blank" rel="noopener"
 &gt;Microsoft Foundry&lt;/a&gt;&lt;/strong&gt;. I might be biased, but outside of my professional interactions with GenAI, I&amp;rsquo;m not into all the social media hypes around it (&lt;a class="link" href="https://www.forbes.com/sites/danidiplacido/2025/03/27/the-ai-generated-studio-ghibli-trend-explained/" target="_blank" rel="noopener"
 &gt;anyone remembers the Studio Gihbli hype from summer&lt;/a&gt; or any similar?)&lt;/p&gt;
&lt;p&gt;With a few days off for Thanksgiving here in the USA, I wanted to spend a bit more time on updating my Spotify playlists. While not always perfect, at least a few of the &amp;lsquo;recommended artists&amp;rsquo; are closely in line with the artists and songs I like.&lt;/p&gt;
&lt;h2 id="the-search-for-new-sounds"&gt;The search for new sounds
&lt;/h2&gt;&lt;p&gt;I always enjoyed listening to music, and exploring new artists. I remember as a kid going to the music store, buying LPs, and later on CDs. And honestly, I discovered several new great artists thanks to Spotify. Thereâ€™s a special thrill in discovering new music. For now, it started with a simple quest: I wanted to explore the &lt;strong&gt;country blues rock&lt;/strong&gt; corner. Think soul, whiskey bars, gritty guitar riffs, and that raw energy that bridges the rural blues tradition with rockâ€™s rebellious spirit. It also brings back great memories to &lt;strong&gt;&lt;a class="link" href="https://www.houseofblues.com" target="_blank" rel="noopener"
 &gt;House of Blues&lt;/a&gt;&lt;/strong&gt; during work travel trips.&lt;/p&gt;
&lt;p&gt;I actually discovered &lt;strong&gt;&lt;a class="link" href="https://www.sonsoflegion.com" target="_blank" rel="noopener"
 &gt;Sons of Legion&lt;/a&gt;&lt;/strong&gt; recently, which &lt;em&gt;offers a unique mix of authentic soul, folk and rock music&lt;/em&gt;, as they describe it themselves.&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-11-29-5d726e9a.png"
 
 
 loading="lazy"
 
 alt="Sons of Legion - verified artist"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;So I dove in. I let Spotifyâ€™s recommendation engine guide me, clicking â€œlikeâ€ on songs that resonated, building a playlist over the course of a week. The algorithm seemed to understand me well. The tracks had the right vibe: swampy slide guitar, stomping rhythms, smoky vocals. I was hooked.&lt;/p&gt;
&lt;h2 id="the-nasty-suprise-of-discovery"&gt;The nasty suprise of discovery
&lt;/h2&gt;&lt;p&gt;&lt;strong&gt;But then came the surprise&lt;/strong&gt;. As I looked closer at the artists behind these songs, something I usually do to learn more about what got them into the music scene, where they are from, what other albums do they have,&amp;hellip; I realized something interesting: One of the artists was an AI-generated artist: &lt;strong&gt;&lt;a class="link" href="https://en.wikipedia.org/wiki/Breaking_Rust" target="_blank" rel="noopener"
 &gt;Breaking Rust&lt;/a&gt;&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-11-29-21aafd83.png"
 
 
 loading="lazy"
 
 alt="Breaking Rust - verified artist"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;Yet, Spotify flagged them as &lt;strong&gt;verified artist&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;At first, I thought I had stumbled upon obscure musicians from small towns or indie labels. Their names sounded authentic (), their album covers, while only showing drawings, they looked convincing, and the songs fitting perfectly in the genre. But maybe a bit too perfect? After I couldn&amp;rsquo;t find much information on the artist or the band. These â€œartistsâ€ had no social media presence, no live performances, no interviews. Instead, they were products of AI music generation platforms, uploaded to Spotify (and basically any other music streaming platform&amp;hellip;) under artificial identities.&lt;/p&gt;
&lt;h2 id="my-genai-dilemma"&gt;My GenAI dilemma
&lt;/h2&gt;&lt;p&gt;Hereâ€™s the paradox: I genuinely enjoyed the songs. They had groove, soul, and typical blues/bluesrock themes. Yet knowing they were AIâ€‘made changed my experience. I felt betrayed.&lt;/p&gt;
&lt;p&gt;Questions coming to mind:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Was I connecting with art, or just with a clever simulation?&lt;/li&gt;
&lt;li&gt;Does it matter if the music moves me, even if no human created it?&lt;/li&gt;
&lt;li&gt;What happens to real musicians struggling to get noticed when &lt;a class="link" href="https://www.msn.com/en-us/music/news/the-no-1-christian-artist-topping-the-charts-has-no-soul-thanks-to-ai/ar-AA1R9jvX?ocid=BingNewsSerp" target="_blank" rel="noopener"
 &gt;AI and AI artists flood the market&lt;/a&gt; with endless tracks?&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;And the most important question: would I continue listening? And from there, would I need to do a thourough research on any new artist I get recommended or discover on my own, to check if they are real?&lt;/p&gt;
&lt;h2 id="my-personal-reflection"&gt;My Personal Reflection
&lt;/h2&gt;&lt;p&gt;After the initial shock, I couldn&amp;rsquo;t keep listening; I removed the full playlist - and &lt;strong&gt;flagged to no longer play the artist&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-11-29-e0cd547f.png"
 
 
 loading="lazy"
 
 alt="Don’t play this artist"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;Some AI tracks still made it into my playlists. But I also made a conscious effort to seek out real musicians â€” artists with biographies, live shows (and buying tickets for some!!), and human voices.&lt;/p&gt;
&lt;p&gt;I realized that part of the joy of music discovery is not just the sound, but the story behind it. Which - for me - adds depth and meaning to the listening experience.&lt;/p&gt;
&lt;p&gt;AI can mimic the sound, but it canâ€™t replicate the story. And my interest for GenAI got a little punch today :/&lt;/p&gt;
&lt;p&gt;Btw, if you know any &lt;strong&gt;real, raw, blues/bluesrock artists&lt;/strong&gt;, please let me know! I am still in search of some new groups to listen to&amp;hellip;&lt;/p&gt;
&lt;p&gt;&lt;a class="link" href="https://www.buymeacoffee.com/pdtit" target="_blank" rel="noopener"
 &gt;&lt;img src="../images/screenshot-2025-11-29-17f576e7.png"
 
 
 loading="lazy"
 
 alt="BuyMeACoffee"
 
 
 
 &gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Cheers!!&lt;/p&gt;
&lt;p&gt;/Peter&lt;/p&gt;</description></item><item><title>Using GitHub Copilot Agent Mode to vibe code a Python shooting game</title><link>https://www.pdtit.be/post/using-github-copilot-agent-mode-to-vibe-code-a-python-shooting-game/</link><pubDate>Fri, 28 Nov 2025 00:00:00 +0000</pubDate><guid>https://www.pdtit.be/post/using-github-copilot-agent-mode-to-vibe-code-a-python-shooting-game/</guid><description>&lt;p&gt;For about a year now, I&amp;rsquo;ve been teaching a lot on &lt;a class="link" href="https://github.com/copilot" target="_blank" rel="noopener"
 &gt;GitHub Copilot&lt;/a&gt; as part of my Microsoft role. Our program offers 2 different learning paths, one created by the Microsoft Content developers, &lt;a class="link" href="https://learn.microsoft.com/en-us/training/paths/accelerate-app-development-using-github-copilot/" target="_blank" rel="noopener"
 &gt;AZ-2007&lt;/a&gt;, and the other one is managed by GitHub Content team, known as &lt;a class="link" href="https://learn.microsoft.com/en-us/training/paths/copilot/" target="_blank" rel="noopener"
 &gt;GH-300&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;If you know my approach to teaching tech a bit, which a learner in my class lately called &lt;strong&gt;inspiring through technology&lt;/strong&gt;, it means I&amp;rsquo;m trying to explain as much as possible through compelling, live demos. After walking learners through different GitHub Copilot features such as documenting/explaining code, generate application code (on different development frameworks), but also Azure CLI, CI/CD pipeline, Dockerfile, YAML, JSON and alike, I usually close with &lt;a class="link" href="https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent" target="_blank" rel="noopener"
 &gt;Agent Mode&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Having showed the first time in April when GitHub Copilot was still in preview, I usually show a demo where the Agent Mode builds me an ASP.NET webapp, modifies the Welcome home page, creates some sample employee data in a json-file, which then gets displayed in a table view in the webapp. If time allows, I also ask to then migrate everything into a SQLite setup, which brings in more complexity such as Entity Framework, SQL data migration steps and interaction with Azure KeyVault, since I specify I want to run this in Azure SQL, but not allowing connection strings in my appsettings.json.&lt;/p&gt;
&lt;p&gt;(Now that I think about it, it might be another great blog post to write on in the near future&amp;hellip;)&lt;/p&gt;
&lt;p&gt;Earlier this week however, I came up with a new scenario, asking Agent Mode to develop a &lt;em&gt;shooter game&lt;/em&gt; using Python code, bringing me back to my youth in the mid-80&amp;rsquo;s when I was playing such games on my first 486-PC.&lt;/p&gt;
&lt;h2 id="agent-mode-prompt"&gt;Agent Mode Prompt
&lt;/h2&gt;&lt;p&gt;The prompt I used was this:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;div class="chroma"&gt;
&lt;table class="lntable"&gt;&lt;tr&gt;&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code&gt;&lt;span class="lnt"&gt;1
&lt;/span&gt;&lt;span class="lnt"&gt;2
&lt;/span&gt;&lt;span class="lnt"&gt;3
&lt;/span&gt;&lt;span class="lnt"&gt;4
&lt;/span&gt;&lt;span class="lnt"&gt;5
&lt;/span&gt;&lt;span class="lnt"&gt;6
&lt;/span&gt;&lt;span class="lnt"&gt;7
&lt;/span&gt;&lt;span class="lnt"&gt;8
&lt;/span&gt;&lt;span class="lnt"&gt;9
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;
&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;*&lt;span class="s2"&gt;&amp;#34;as a kid, I played arcade games a lot. I want to build a Python app, which tests me on my shooting reflexes. Help me developing a game which does the following:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="s2"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="s2"&gt;1. aks for player name input
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="s2"&gt;2. bottom middle of the game screen shows a shooter 
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="s2"&gt;3. anywhere random on screen appears a target
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="s2"&gt;4. player uses the space bar to simulate a shoot
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="s2"&gt;5. calculate the time between the target appearing and player pressing the space bar to shoot
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="s2"&gt;6. if that time is less than 0.3 sec, player wins, otherwise computer wins
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="s2"&gt;7. show a &amp;#34;&lt;/span&gt;YOU WIN&lt;span class="s2"&gt;&amp;#34; or &amp;#34;&lt;/span&gt;YOU ARE TOO SLOW&lt;span class="s2"&gt;&amp;#34; dpeending on the outcome&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;
&lt;/div&gt;
&lt;/div&gt;&lt;p&gt;&lt;img src="../images/screenshot-2025-11-28-018f3ae5.png"
 
 
 loading="lazy"
 
 alt="Agent Mode Prompt Used"
 
 
 
 &gt;&lt;/p&gt;
&lt;h2 id="agent-mode-processing"&gt;Agent Mode Processing
&lt;/h2&gt;&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;From here, the &lt;strong&gt;Agent started rolling&amp;hellip;&lt;/strong&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Confirming with some sort of understanding what I asked for, followed by &lt;em&gt;creating 3 todos&lt;/em&gt;:&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;ul&gt;
&lt;li&gt;Set Up Python Environment&lt;/li&gt;
&lt;li&gt;Creating the Reflex Shooting Game&lt;/li&gt;
&lt;li&gt;Testing the Game&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-11-28-88f73764.png"
 
 
 loading="lazy"
 
 alt="Agent Mode Starting"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;This process took less than &lt;strong&gt;1 minute&lt;/strong&gt;, can you imagine? From there, it continued with providing detailed instructions on how to the game works.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-11-28-f533f735.png"
 
 
 loading="lazy"
 
 alt="Game Playing Instructions"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Next, it also had a list of features included:&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-11-28-dc0b721b.png"
 
 
 loading="lazy"
 
 alt="Game Features Included"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Time to start the game!!&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-11-28-c3d6fa9b.png"
 
 
 loading="lazy"
 
 alt="Start Game"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Which allowed me to play exactly as I asked for. When I was too slow, it would tell me&amp;hellip;&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-11-28-df129c5d.png"
 
 
 loading="lazy"
 
 alt="Too Slow"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;And several attempts later, I finally managed to win a game!!&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-11-28-8d0ea57f.png"
 
 
 loading="lazy"
 
 alt="You Win"
 
 
 
 &gt;&lt;/p&gt;
&lt;h2 id="summary"&gt;Summary
&lt;/h2&gt;&lt;p&gt;I thought after using GitHub Copilot for training our customers and showing capabilities using live demos for about 5 hours per class, as well as using it for a lot of &amp;ldquo;coding&amp;rdquo; tasks as part of my role as trainer, mainly creating more demo scenarios (see &lt;a class="link" href="https://aka.ms/trainer-demo-deploy" target="_blank" rel="noopener"
 &gt;Trainer-Demo-Deploy&lt;/a&gt; to get an idea what that means&amp;hellip;), I thought I&amp;rsquo;d seen it all.&lt;/p&gt;
&lt;p&gt;Yet, &lt;strong&gt;it keeps suprising me every single day when I try something new.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;If I had access to this technology in the mid 80&amp;rsquo;s, I guess I would have spent more time learning about coding than playing games&amp;hellip; although this game is actually pretty addicting already. Time to wrap up this post and go play a bit more! And feel 12 years old again.&lt;/p&gt;
&lt;p&gt;If you want to see some similar version of this templategame in action, head over to &lt;a class="link" href="https://github.com/petender/vibeshooter" target="_blank" rel="noopener"
 &gt;my github repo&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;&lt;a class="link" href="https://www.buymeacoffee.com/pdtit" target="_blank" rel="noopener"
 &gt;&lt;img src="../images/screenshot-2025-11-28-17f576e7.png"
 
 
 loading="lazy"
 
 alt="BuyMeACoffee"
 
 
 
 &gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Cheers!!&lt;/p&gt;
&lt;p&gt;/Peter&lt;/p&gt;</description></item><item><title>Generate Azure JWT Token in Copilot Studio</title><link>https://www.pdtit.be/post/generate-azure-jwt-token-in-copilot-studio/</link><pubDate>Sat, 09 Aug 2025 00:00:00 +0000</pubDate><guid>https://www.pdtit.be/post/generate-azure-jwt-token-in-copilot-studio/</guid><description>&lt;p&gt;Out of my role as a Lead Technical Trainer at Microsoft, the portfolio of trainings I&amp;rsquo;m covering has heavily shifted to Azure AI and Copilot over the last few months. Still doing Azure Architecture and Developing courses as well, but not as frequent anymore. This confirms the interest we see at customers in adopting Generative AI solutions. Apart from &lt;a class="link" href="https://www.microsoft.com/microsoft-365/copilot" target="_blank" rel="noopener"
 &gt;Copilot in M365&lt;/a&gt;, or using &lt;a class="link" href="https://azure.microsoft.com/en-us/products/ai-foundry" target="_blank" rel="noopener"
 &gt;Azure AI Foundry&lt;/a&gt;, I also started digging into &lt;a class="link" href="https://azure.microsoft.com/en-us/products/copilot-studio" target="_blank" rel="noopener"
 &gt;Copilot Studio&lt;/a&gt; a lot more. Having a good background in Azure LogicApps and a bit of PowerPlatform, Copilot Studio feels quite comfortable to me.&lt;/p&gt;
&lt;p&gt;I&amp;rsquo;ve been working on a few Agent scenarios in Copilot Studio, which I will blog more about in the near future. One of the newer features that got my attention, is the &lt;strong&gt;advanced feature&lt;/strong&gt; of HTTP Requests, which opens the door to using &lt;strong&gt;REST API calls&lt;/strong&gt; to other platforms, for example &lt;strong&gt;Azure&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;As you probably know, any action against Azure requires authentication, whether an interactive admin user logon, or a Service Principal application logon. Which means that - before I can trigger any actions against Azure, I first need to get the Copilot Studio Agent authenticating to Azure, using a JWT Bearer token (Azure Entra ID OAuth 2.0 Token).&lt;/p&gt;
&lt;p&gt;This article walks you through the different steps and setup of the Copilot Studio Agent, to allow it to authenticate to Azure, and from there taking a possible next step against the platform.&lt;/p&gt;
&lt;h2 id="what-this-article-covers"&gt;What this article covers
&lt;/h2&gt;&lt;p&gt;âœ… How to create App Registration for Copilot Studio Agent in Entra ID
âœ… How to generate JWT Bearer token in Copilot Studio for API authentication
âœ… How to set up Microsoft Graph API authentication with Azure Entra ID OAuth 2.0 token&lt;/p&gt;
&lt;h2 id="create-entra-id-app-registration-for-copilot-studio-agent"&gt;Create Entra ID App Registration for Copilot Studio Agent
&lt;/h2&gt;&lt;p&gt;Any Service / Application level interaction with Azure, starts from an App Registration. This generates a Service Principal, think of it as a Service Account, which then gets linked/reused by a 3rd party application, as like in our case, Copilot Studio.&lt;/p&gt;
&lt;p&gt;Apart from creating the Service Principal entity object, it also needs corresponding API permissions to interact with Microsoft Graph.&lt;/p&gt;
&lt;p&gt;These are the steps to set all this up:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;From Entra ID, navigate to &lt;strong&gt;App Registration&lt;/strong&gt;, and select &lt;strong&gt;New Registration&lt;/strong&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-eea9c540.png"
 
 
 loading="lazy"
 
 alt="App Registration"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="2"&gt;
&lt;li&gt;Provide a name for the App Registration &lt;strong&gt;e.g. Copilot Studio Agent Demo&lt;/strong&gt;, and leave all other default settings as-is. Click &lt;strong&gt;Register&lt;/strong&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-26cbbfbf.png"
 
 
 loading="lazy"
 
 alt="App Registration"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="3"&gt;
&lt;li&gt;This generates the Service Principal, with some specific IDs you need to copy aside. The Client ID, reflecting the unique GUID of the Service Principal as well as the Tenant ID which corresponds to your Entra ID Tenant GUID.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-3d5a737f.png"
 
 
 loading="lazy"
 
 alt="Client Credentials"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="4"&gt;
&lt;li&gt;Next, the Service Principal also needs the necessary credential to authenticate, which can be set up from the &lt;strong&gt;Subscription or Resource Group Access Control IAM (RBAC)&lt;/strong&gt; permissions. For my example, I specify &amp;ldquo;Read&amp;rdquo; permissions on Subscription level, as I only want to run that as a validation of my workflow actually running successfully. You could alter to any permissions your scenario requires.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-ed33ad68.png"
 
 
 loading="lazy"
 
 alt="RBAC Permissions"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="5"&gt;
&lt;li&gt;And to request an authentication JWT Token, we also need to pass the Client ID password, which can be generated from the Entra ID App Registration page for the newly created App Registration. Copy this secret aside, as you will need it in a later step.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-2563fe19.png"
 
 
 loading="lazy"
 
 alt="App Registration Client Secret"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="6"&gt;
&lt;li&gt;With all this out of the way, we have all ID information, credentials and RBAC permissions to bring into Copilot Studio. But there are a few other pieces of information we are gathering first, the &lt;strong&gt;Authentication API Endpoint&lt;/strong&gt;, which is the Microsoft Login URL for our Tenant. this should look like this:&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;div class="chroma"&gt;
&lt;table class="lntable"&gt;&lt;tr&gt;&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code&gt;&lt;span class="lnt"&gt;1
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;
&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-fallback" data-lang="fallback"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;https://login.microsoftonline.com/&amp;lt;yourtenantID&amp;gt;/oauth2/v.2.0/token
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;
&lt;/div&gt;
&lt;/div&gt;&lt;p&gt;and we also need a REST API Header and Body, which are the additional parameters Copilot Studio Agent REST API action requires:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;div class="chroma"&gt;
&lt;table class="lntable"&gt;&lt;tr&gt;&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code&gt;&lt;span class="lnt"&gt;1
&lt;/span&gt;&lt;span class="lnt"&gt;2
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;
&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-fallback" data-lang="fallback"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;- Header: Content-Type: application/x-www-form-urlencoded
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;- Body: client-id=&amp;lt;yourclientid&amp;gt;&amp;amp;client_secret=&amp;lt;clientsecret&amp;gt;&amp;amp;grant_type=client_credentials&amp;amp;scope=https://graph.microsoft.com/.default
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;
&lt;/div&gt;
&lt;/div&gt;&lt;p&gt;where you insert the actual value of the clientid and clientsecret you copied earlier for the placeholders.&lt;/p&gt;
&lt;h2 id="configure-the-http-request-action-in-copilot-studio"&gt;Configure the HTTP Request Action in Copilot Studio
&lt;/h2&gt;&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Navigate to &lt;a class="link" href="https://copilotstudio.microsoft.com" target="_blank" rel="noopener"
 &gt;Copilot Studio&lt;/a&gt; and open the workflow setup of your Agent.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Navigate to &lt;strong&gt;Topics&lt;/strong&gt; and &lt;strong&gt;Add a new topic&lt;/strong&gt;. Select &amp;ldquo;From Blank&amp;rdquo;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-e3c7cb65.png"
 
 
 loading="lazy"
 
 alt="New Copilot Studio Agent"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="3"&gt;
&lt;li&gt;Click the &lt;strong&gt;+ Sign&lt;/strong&gt; below the Trigger step, and select &lt;strong&gt;Advanced / HTTP Request&lt;/strong&gt; from the option menu.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-66d30fbe.png"
 
 
 loading="lazy"
 
 alt="HTTP Request"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="4"&gt;
&lt;li&gt;Complete the following fields of the HTTP Request per below overview:&lt;/li&gt;
&lt;/ol&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;URL&lt;/strong&gt;: the login-URL specified earlier: &lt;a class="link" href="https://login.microsoftonline.com/" target="_blank" rel="noopener"
 &gt;https://login.microsoftonline.com/&lt;/a&gt;&lt;tenantID&gt;/oauth2/v.2.0/token, where the &lt;tenantid&gt; placeholder gets replaced with the actual GUID of your tenant, something like this (redacted)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-919da3b8.png"
 
 
 loading="lazy"
 
 alt="HTTP Request URL"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;&lt;a class="link" href="https://login.microsoftonline.com/1c5e3b03-f225-4622-b785-abcdefghi/oauth2/token" target="_blank" rel="noopener"
 &gt;https://login.microsoftonline.com/1c5e3b03-f225-4622-b785-abcdefghi/oauth2/token&lt;/a&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Method&lt;/strong&gt;: POST&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Headers and Body&lt;/strong&gt;:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;em&gt;Headers / Key&lt;/em&gt;: Content-Type&lt;/li&gt;
&lt;li&gt;&lt;em&gt;Headers / Value&lt;/em&gt;: application/x-www-form-urlencoded&lt;/li&gt;
&lt;li&gt;&lt;em&gt;Body&lt;/em&gt;: Raw Content&lt;/li&gt;
&lt;li&gt;&lt;em&gt;Content Type&lt;/em&gt;: application/x-www-form-urlencoded&lt;/li&gt;
&lt;li&gt;&lt;em&gt;Content&lt;/em&gt;: client-id=c92c3f9f-7ba3-4e5b-1234-abcdefghi&amp;amp;client_secret=nzE8Q~q-tDIrvlLkBGe2IwWH.abcdefghij_&amp;amp;grant_type=client_credentials&amp;amp;scope=https://graph.microsoft.com/.default&lt;/li&gt;
&lt;li&gt;&lt;em&gt;Response headers&lt;/em&gt;: Create new &lt;strong&gt;Global&lt;/strong&gt; variable to store the value in, e.g. HTTPResponseVar&lt;/li&gt;
&lt;li&gt;&lt;em&gt;Response data type&lt;/em&gt;: Record + select Edit Schema, and add the following schema structure:
&lt;div class="highlight"&gt;&lt;div class="chroma"&gt;
&lt;table class="lntable"&gt;&lt;tr&gt;&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code&gt;&lt;span class="lnt"&gt;1
&lt;/span&gt;&lt;span class="lnt"&gt;2
&lt;/span&gt;&lt;span class="lnt"&gt;3
&lt;/span&gt;&lt;span class="lnt"&gt;4
&lt;/span&gt;&lt;span class="lnt"&gt;5
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;
&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-fallback" data-lang="fallback"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;kind: Record
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;properties: 
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; access_token: String
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; expires_in: Number
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; token_type: String
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;
&lt;/div&gt;
&lt;/div&gt;&lt;/li&gt;
&lt;li&gt;&lt;em&gt;Save Response as&lt;/em&gt;: select the Global.HTTPResponseVar again&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;ol start="5"&gt;
&lt;li&gt;&lt;strong&gt;Save&lt;/strong&gt; the changes.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-0114f195.png"
 
 
 loading="lazy"
 
 alt="HTTP Request Headers"
 
 
 
 &gt;
Request Header details&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-255a4fbb.png"
 
 
 loading="lazy"
 
 alt="HTTP Request Body Content"
 
 
 
 &gt;
Request Body details&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-beda8599.png"
 
 
 loading="lazy"
 
 alt="Global Variable details"
 
 
 
 &gt;
Global Variable details&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-d9e11cea.png"
 
 
 loading="lazy"
 
 alt="Record Schema details"
 
 
 
 &gt;
Record Schema details&lt;/p&gt;
&lt;ol start="6"&gt;
&lt;li&gt;
&lt;p&gt;While this flow should work fine now, you won&amp;rsquo;t get any output from it. We need to update the flow with a follow-up message, in which we read/present the output from the HTTPResponseVar variable. &lt;strong&gt;Click&lt;/strong&gt; the &lt;strong&gt;+&lt;/strong&gt; sign below the HTTP Request step in the workflow, and select &lt;strong&gt;Send a Message&lt;/strong&gt; from the context menu.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Enter an informative text, e.g. &amp;ldquo;Here is the Azure Token String&amp;rdquo;, and add the HTTPResponseVar variable into the text box, by selecting the &lt;em&gt;insert variable {X}&lt;/em&gt; option and selecting the variable from the list.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-4bb46368.png"
 
 
 loading="lazy"
 
 alt="Response Message"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="8"&gt;
&lt;li&gt;&lt;strong&gt;Save&lt;/strong&gt; the changes. Next, from the &lt;strong&gt;Test your Agent&lt;/strong&gt; pane, trigger the Agent flow by sending a short chat message, like &amp;ldquo;get my token&amp;rdquo;. This should result in the chat response, showing your message &amp;ldquo;Here is the Azure Token String&amp;rdquo;, and the actual JWT token with all necessary information in it.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-591adf22.png"
 
 
 loading="lazy"
 
 alt="Response JWT Token"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="9"&gt;
&lt;li&gt;
&lt;p&gt;Cool, this works as expected! While we&amp;rsquo;re close, we&amp;rsquo;re not 100% done yet, as the value of this variable is not immediately reusable as an authentication token, as not all information in the response is part of the actually authentication token (e.g. String{&amp;ldquo;access_token&amp;rdquo;}, &amp;ldquo;expires_in&amp;rdquo;, &amp;ldquo;token_type&amp;rdquo;). We can fix this by running a &lt;strong&gt;concatenate&lt;/strong&gt; formula, and splitting the received information in a new variable which only stores the actual Token information we need to authenticate. After the last message step in the flow, &lt;strong&gt;click the + sign&lt;/strong&gt; again, and once more, select &lt;em&gt;Send a Message&lt;/em&gt;. Provide a new informative message, something like &amp;ldquo;And this is the cleaned up version of the Bearer token, just what you need&amp;hellip;&amp;rdquo;, and add a new &lt;strong&gt;PowerFx Expression&lt;/strong&gt; by clicking the &lt;strong&gt;{fX}&lt;/strong&gt; button. Enter the following formula:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;div class="chroma"&gt;
&lt;table class="lntable"&gt;&lt;tr&gt;&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code&gt;&lt;span class="lnt"&gt;1
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;
&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-fallback" data-lang="fallback"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Concatenate(Topic.HTTPResponseVar.token_type,&amp;#34; &amp;#34;,Topic.HTTPResponseVar.access_token)
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;
&lt;/div&gt;
&lt;/div&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-7792f764.png"
 
 
 loading="lazy"
 
 alt="Concat Response JWT Token"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="10"&gt;
&lt;li&gt;Which would transform the response into a valid Bearer token text string &amp;ldquo;Bearer ey&amp;hellip;&amp;rdquo; which you can use for any Azure HTTP REST API in a different Topic. To do that, it&amp;rsquo;s best to save the concat result in a new Global Variable.&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 id="reuse-the-jwt-authenticator-topic-in-copilot-studio-agent"&gt;Reuse the JWT Authenticator Topic in Copilot Studio Agent
&lt;/h2&gt;&lt;p&gt;While I want to keep this article on the actual JWT Token authentication process, I wanted to add a little teaser for a follow-up article, in which I create a Copilot Studio Agent to interact with Azure, relying on the Bearer Token from this Topic we just created. In any Copilot Studio flow you have, you can now refer to the Auth2Azure Authentication request Topic like this:&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-08-09-624f9a1b.png"
 
 
 loading="lazy"
 
 alt="Reuse Bearer Token Topic"
 
 
 
 &gt;&lt;/p&gt;
&lt;h2 id="summary"&gt;Summary
&lt;/h2&gt;&lt;p&gt;In this article, I wanted to document the necessary steps on how to use the Copilot Studio Agent - HTTP Request task, to get a Bearer Token to authenticate to Azure (or any similar HTTP REST API for that matter).&lt;/p&gt;
&lt;p&gt;&lt;a class="link" href="https://www.buymeacoffee.com/pdtit" target="_blank" rel="noopener"
 &gt;&lt;img src="../images/screenshot-2025-08-09-17f576e7.png"
 
 
 loading="lazy"
 
 alt="BuyMeACoffee"
 
 
 
 &gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Cheers!!&lt;/p&gt;
&lt;p&gt;/Peter&lt;/p&gt;</description></item><item><title>Using GitHub Copilot Agent Mode to transform ARM templates to BICEP</title><link>https://www.pdtit.be/post/using-github-copilot-agent-mode-to-transform-arm-templates-to-bicep/</link><pubDate>Sat, 12 Jul 2025 00:00:00 +0000</pubDate><guid>https://www.pdtit.be/post/using-github-copilot-agent-mode-to-transform-arm-templates-to-bicep/</guid><description>&lt;p&gt;Over the last few months, I&amp;rsquo;ve been working on an exciting project for our Microsoft Technical Trainer team, known as &lt;a class="link" href="https://aka.ms/trainer-demo-deploy" target="_blank" rel="noopener"
 &gt;&amp;ldquo;Trainer-Demo-Deploy&amp;rdquo;&lt;/a&gt;, a catalog of Azure end-to-end demo scenarios, available as an Open-Source project.&lt;/p&gt;
&lt;p&gt;While we managed to get about 50 templates live, there can never be enough scenarios to integrate into your Azure classes or POC activities if you ask me. One of the challenging tasks in the project is not only coming up with demo ideas, but also creating the actual artifacts, such as Azure templates with Bicep, sample apps and sample data.&lt;/p&gt;
&lt;p&gt;I had an &lt;a class="link" href="https://azure.microsoft.com/en-us/products/site-recovery/" target="_blank" rel="noopener"
 &gt;Azure Site Recovery Services&lt;/a&gt; scenario from a few years ago, written in modular ARM templates. With Bicep providing a great way to &lt;a class="link" href="https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/decompile?tabs=azure-cli" target="_blank" rel="noopener"
 &gt;transform your ARM to Bicep&lt;/a&gt;, I could have gone through each template file and convert them. Have done several of those over the last few months.&lt;/p&gt;
&lt;p&gt;But out of teaching &lt;a class="link" href="https://learn.microsoft.com/en-us/training/paths/accelerate-app-development-using-github-copilot/" target="_blank" rel="noopener"
 &gt;AZ-2007 Accelerate app development by using GitHub Copilot&lt;/a&gt;, where I integrate a - what I think amazing demo on how to use Agent mode to deploy a sample web app - I started thinking about testing if &lt;strong&gt;Agent Mode&lt;/strong&gt; could help me with this transformation project.&lt;/p&gt;
&lt;p&gt;Fact I&amp;rsquo;m dedicating a blog post to it, is mainly to confirm it worked amazingly well, as well as sharing my excitement and some steps of what the process looked like. Hopefully this post inspires you to start embracing &lt;a class="link" href="https://github.blog/ai-and-ml/github-copilot/agent-mode-101-all-about-github-copilots-powerful-mode/" target="_blank" rel="noopener"
 &gt;&lt;strong&gt;GitHub Copilot Agent Mode&lt;/strong&gt;&lt;/a&gt; into your own tasks.&lt;/p&gt;
&lt;h2 id="my-starting-templates"&gt;My starting templates
&lt;/h2&gt;&lt;ol&gt;
&lt;li&gt;My original setup was pretty straightforward, having a folder &amp;ldquo;templates&amp;rdquo;, in which I have modular templates for each part of my Azure Site Recovery Vault deployment.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Each templates hold a snippet of ARM / JSON structured code to deploy one or more Azure Resources.&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-677ae48c.png"
 
 
 loading="lazy"
 
 alt="Current ARM Templates"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="2"&gt;
&lt;li&gt;I opened up the folder structure in my Visual Studio Code, and opened GitHub Copilot, selecting &lt;strong&gt;Agent Mode&lt;/strong&gt;. I &lt;em&gt;clearly described in a prompt&lt;/em&gt;, what I wanted the Agent to perform as tasks. I didn&amp;rsquo;t provide much detail to be honest, as initially, I was merily experimenting to try and find out if Agent could actually help with this, or how far it would go in the process.&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 id="agent-mode-prompt"&gt;Agent Mode Prompt
&lt;/h2&gt;&lt;p&gt;The prompt I used was this:
&lt;em&gt;for each file in the templates folder, convert to azure bicep. create a new bicep file for each, keeping the same name as the original json file. the azuredeploy should be transformed to main.bicep. validate all pointers to all new bicep files to be correct&amp;quot;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-0e6d3c1f.png"
 
 
 loading="lazy"
 
 alt="Agent Mode Prompt Used"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="3"&gt;
&lt;li&gt;
&lt;p&gt;From here, the &lt;strong&gt;Agent started rolling&amp;hellip;&lt;/strong&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Informing me about the different steps it would take to handle this task, starting with &lt;em&gt;exploring the templates folder to see all the files that need to be converted&lt;/em&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-9563c155.png"
 
 
 loading="lazy"
 
 alt="Agent Mode Starting"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="5"&gt;
&lt;li&gt;
&lt;p&gt;Followed by going more in-depth into each and every template JSON file&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Followed by &lt;em&gt;starting the conversion process to Bicep files&lt;/em&gt;. Before doing that, it also highlighted it would check the Azure deployment best practices (although I didn&amp;rsquo;t explicitly asked to do that, nice one! )&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-23d6f25c.png"
 
 
 loading="lazy"
 
 alt="Agent Mode inspecting files"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="7"&gt;
&lt;li&gt;It felt like it &lt;em&gt;learned&lt;/em&gt; from the best practices, by starting with the azuredeploy.json conversion to main.bicep first. It could also be that it started with this, because I mentioned this in the prompt itself.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-048d528f.png"
 
 
 loading="lazy"
 
 alt="Agent Mode creating main.bicep"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="8"&gt;
&lt;li&gt;As the main.bicep conversion took a bit longer than normal - although it was only running over it for about a minute, it &lt;em&gt;prompted&lt;/em&gt;, asking if it was ok to continue. Obviously, I confirmed to &lt;strong&gt;continue&lt;/strong&gt;.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-9f313718.png"
 
 
 loading="lazy"
 
 alt="Agent Mode pending task"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="9"&gt;
&lt;li&gt;From here, it nicely continued looping through all smaller json-files, and transforming them into corresponding bicep-files. Since each file typically had only 1 or 2 resource references, the conversion went really smooth.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-cc406a5d.png"
 
 
 loading="lazy"
 
 alt="Agent Mode conversion"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="10"&gt;
&lt;li&gt;After a bit, it had finished the transformation of each ARM to Bicep, and &lt;em&gt;started on updating the template links&lt;/em&gt;, as I asked for in my prompt, to also validate the references to all the deployment files.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-3a52cde6.png"
 
 
 loading="lazy"
 
 alt="Agent Mode updating file links"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="11"&gt;
&lt;li&gt;With all references updated, it continued with its own &lt;em&gt;error checking&lt;/em&gt;, and validating the different templates for any possible errors. Even more interesting, &lt;strong&gt;without me specifying&lt;/strong&gt;, it detected an error in the &lt;strong&gt;azure.yaml&lt;/strong&gt;, which I had in my project folder, from a baseline started AZD-template we use to create all our Trainer-Demo-Deploy scenarios.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-c58c585e.png"
 
 
 loading="lazy"
 
 alt="Agent Mode create azure.yaml"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="12"&gt;
&lt;li&gt;Last, it also created a &lt;em&gt;main.parameters.json&lt;/em&gt;, to capture any specific Parameters for the deployment.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-52ae46b8.png"
 
 
 loading="lazy"
 
 alt="Agent Mode create main.parameters.json"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="13"&gt;
&lt;li&gt;From here, it went back to &lt;em&gt;validating&lt;/em&gt; the Bicep templates again, where it detected a few different issues. (I didn&amp;rsquo;t check in detail what got identified as issue, as it didn&amp;rsquo;t prompt me to validate anything on my end&amp;hellip;); based on the &lt;em&gt;next informational message&lt;/em&gt;, it struggled with missing an output for &lt;strong&gt;myWorkspaceKey&lt;/strong&gt;, in the deploy-infra.bicep file.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images2025-07-04-bicep-error.png"
 
 
 loading="lazy"
 
 alt="Agent Mode validating infra"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="14"&gt;
&lt;li&gt;&lt;em&gt;Chewing&lt;/em&gt; a bit on the myWorkspaceKey problem, it managed to find its own work-around to solve the problem. It even provided a clear explaining on why, identifying the dependency on the parent template.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-7c9e3eae.png"
 
 
 loading="lazy"
 
 alt="Agent Mode detecting an issue"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="15"&gt;
&lt;li&gt;Feeling we were close to the end of the process, it &lt;strong&gt;continued amazing&lt;/strong&gt; me, as it now also created its &lt;strong&gt;own documentation&lt;/strong&gt; in a BICEP_CONVERSION_SUMMARY.md Markdown file, in which it listed up what conversations it did.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-233210a1.png"
 
 
 loading="lazy"
 
 alt="Agent Mode creating conversion documentation"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="16"&gt;
&lt;li&gt;With all that out of the way, it ran another &lt;strong&gt;final validation&lt;/strong&gt; to conclude there were no more issues, closing the task with creating another README-BICEP.md file, describing how to run the actual deployment, using AZD.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-ea367bd4.png"
 
 
 loading="lazy"
 
 alt="Agent Mode creating readme"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="17"&gt;
&lt;li&gt;Finally, the Agent &lt;strong&gt;provided a description within the Chat Agent window&lt;/strong&gt;, clearly describing all the tasks accomplished with the necessary file references included:&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-12b1b132.png"
 
 
 loading="lazy"
 
 alt="Agent Mode Chat description"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="18"&gt;
&lt;li&gt;As well as adding additional details on the &lt;strong&gt;task validation&lt;/strong&gt;.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-8a5606f6.png"
 
 
 loading="lazy"
 
 alt="Agent Mode creating task validation"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="19"&gt;
&lt;li&gt;Finishing with describing different ways on how to run the actual deployment, using &lt;strong&gt;azd&lt;/strong&gt;, &lt;strong&gt;Azure CLI&lt;/strong&gt; and &lt;strong&gt;Azure PowerShell&lt;/strong&gt;.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-1a21a2ce.png"
 
 
 loading="lazy"
 
 alt="Agent Mode explaining how to run the deployment"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="20"&gt;
&lt;li&gt;Last step was running the deployment, and this worked without any hiccups!&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2025-07-12-b01b0623.png"
 
 
 loading="lazy"
 
 alt="Deployment completed"
 
 
 
 &gt;&lt;/p&gt;
&lt;h2 id="summary"&gt;Summary
&lt;/h2&gt;&lt;p&gt;As mentioned earlier, I didn&amp;rsquo;t intend to go through this process as part of writing a blog post. Yet, the fact that the &lt;strong&gt;GitHub Copilot Agent Mode&lt;/strong&gt; happily suprised me once more, I wanted to share my joy and excitement about this.&lt;/p&gt;
&lt;p&gt;Starting from a &lt;em&gt;somewhat complex&lt;/em&gt; JSON ARM template folder with about 10 modular arm-json files, it managed to nicely convert all of them into the new Bicep template language, with only a few minor issues throughout the process. Without asking assistance or halting the process, it ran its own troubleshooting and issue resolution, resulting in a 100% successful transformation.&lt;/p&gt;
&lt;p&gt;Apart from the technical success of the task, what surprised me even more, is &lt;strong&gt;it only took the agent barely 5 minutes&lt;/strong&gt; and only had to &lt;strong&gt;prompt me twice&lt;/strong&gt; during the whole process!!&lt;/p&gt;
&lt;p&gt;If you want to see this template in action, head over to &lt;a class="link" href="https://github.com/petender/azd-asrdemo" target="_blank" rel="noopener"
 &gt;my github repo&lt;/a&gt; and continue your Azure learning journey with more demo scenarios at &lt;a class="link" href="https://aka.ms/trainer-demo-deploy" target="_blank" rel="noopener"
 &gt;Trainer-Demo-Deploy&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;&lt;a class="link" href="https://www.buymeacoffee.com/pdtit" target="_blank" rel="noopener"
 &gt;&lt;img src="../images/screenshot-2025-07-12-17f576e7.png"
 
 
 loading="lazy"
 
 alt="BuyMeACoffee"
 
 
 
 &gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Cheers!!&lt;/p&gt;
&lt;p&gt;/Peter&lt;/p&gt;</description></item><item><title>Introduction to Semantic Kernel</title><link>https://www.pdtit.be/post/introduction-to-semantic-kernel/</link><pubDate>Sun, 15 Sep 2024 00:00:00 +0000</pubDate><guid>https://www.pdtit.be/post/introduction-to-semantic-kernel/</guid><description>&lt;h2 id="introduction-to-developing-azure-ai-solutions"&gt;Introduction to Developing Azure AI Solutions
&lt;/h2&gt;&lt;p&gt;In today&amp;rsquo;s rapidly evolving tech landscape, Artificial Intelligence (AI) has become a cornerstone for innovation. Azure AI offers a robust suite of tools and services that empower developers to build intelligent applications. From natural language processing (NLP) to computer vision, &lt;a class="link" href="https://azure.microsoft.com/en-us/solutions/ai" target="_blank" rel="noopener"
 &gt;Azure AI&lt;/a&gt; provides the building blocks to create solutions that can understand, interpret, and respond to human inputs in a meaningful way. While Microsoft has several Copilot offerings for different use cases, ranging from an &lt;a class="link" href="https://azure.microsoft.com/en-us/products/copilot" target="_blank" rel="noopener"
 &gt;AI assistant in Azure&lt;/a&gt;, &lt;a class="link" href="https://www.microsoft.com/en-us/microsoft-365/copilot" target="_blank" rel="noopener"
 &gt;Copilot in M365&lt;/a&gt; or &lt;a class="link" href="https://copilot.microsoft.com" target="_blank" rel="noopener"
 &gt;web and mobile&lt;/a&gt;, there are still valid use cases for developing your own custom Copilot. One of the key components in this ecosystem is the &lt;strong&gt;Semantic Kernel&lt;/strong&gt;, a powerful tool that enhances the capabilities of AI models by providing semantic understanding.&lt;/p&gt;
&lt;p&gt;The Kernel is the central component of Semantic Kernel. In its easiest format, the Kernel is a Dependency Injection objects, which manages all of the services and plugins necessary to run your AI application. If you provide all of your services and plugins to the kernel, they will then be seamlessly used by the AI as needed.&lt;/p&gt;
&lt;h2 id="what-is-semantic-kernel"&gt;What is Semantic Kernel?
&lt;/h2&gt;&lt;p&gt;Semantic Kernel is a framework designed to bridge the gap between raw data and meaningful insights. It leverages advanced machine learning algorithms to understand the context and semantics of the data, enabling more accurate and relevant responses. Unlike traditional keyword-based approaches, Semantic Kernel focuses on the meaning behind the words, making it a valuable asset for applications that require a deep understanding of language.&lt;/p&gt;
&lt;h2 id="difference-between-using-semantic-kernel-and-other-solutions-such-as-promptflow"&gt;Difference Between Using Semantic Kernel and Other Solutions Such as PromptFlow
&lt;/h2&gt;&lt;p&gt;While both &lt;a class="link" href="https://learn.microsoft.com/en-us/semantic-kernel/" target="_blank" rel="noopener"
 &gt;Semantic Kernel&lt;/a&gt; and &lt;a class="link" href="https://github.com/microsoft/promptflow" target="_blank" rel="noopener"
 &gt;PromptFlow&lt;/a&gt; are designed to enhance AI capabilities, they serve different purposes and offer unique advantages. &lt;strong&gt;PromptFlow&lt;/strong&gt; is a tool that helps in designing and managing prompts for AI models, ensuring that the inputs are structured in a way that maximizes the model&amp;rsquo;s performance. On the other hand, &lt;strong&gt;Semantic Kernel&lt;/strong&gt; goes a step further by interpreting the meaning behind the inputs, providing a more nuanced and context-aware response.&lt;/p&gt;
&lt;h3 id="key-differences"&gt;Key Differences:
&lt;/h3&gt;&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Focus&lt;/strong&gt;: PromptFlow is primarily concerned with the structure and format of prompts, while Semantic Kernel focuses on understanding the semantics and context.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Use Cases&lt;/strong&gt;: PromptFlow is ideal for scenarios where the input needs to be carefully crafted to elicit the desired response from the AI model. Semantic Kernel is better suited for applications that require a deep understanding of language and context.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Complexity&lt;/strong&gt;: Semantic Kernel involves more complex algorithms and models to interpret the data, whereas PromptFlow is more straightforward in its approach.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="sample-code-scenarios-of-using-semantic-kernel-using-c-net-code"&gt;Sample Code Scenarios of Using Semantic Kernel, Using C# .NET Code
&lt;/h2&gt;&lt;h3 id="scenario-1-text-classification"&gt;Scenario 1: Text Classification
&lt;/h3&gt;&lt;div class="highlight"&gt;&lt;div class="chroma"&gt;
&lt;table class="lntable"&gt;&lt;tr&gt;&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code&gt;&lt;span class="lnt"&gt; 1
&lt;/span&gt;&lt;span class="lnt"&gt; 2
&lt;/span&gt;&lt;span class="lnt"&gt; 3
&lt;/span&gt;&lt;span class="lnt"&gt; 4
&lt;/span&gt;&lt;span class="lnt"&gt; 5
&lt;/span&gt;&lt;span class="lnt"&gt; 6
&lt;/span&gt;&lt;span class="lnt"&gt; 7
&lt;/span&gt;&lt;span class="lnt"&gt; 8
&lt;/span&gt;&lt;span class="lnt"&gt; 9
&lt;/span&gt;&lt;span class="lnt"&gt;10
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;
&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-csharp" data-lang="csharp"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;using&lt;/span&gt; &lt;span class="nn"&gt;Microsoft.SemanticKernel&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;using&lt;/span&gt; &lt;span class="nn"&gt;Microsoft.SemanticKernel.Models&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;kernel&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="n"&gt;SemanticKernel&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;model&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;kernel&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;LoadModel&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;&amp;#34;text-classification-model&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;inputText&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;&amp;#34;Azure AI is transforming the tech industry.&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;classification&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Classify&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;inputText&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;Console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;WriteLine&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;$&amp;#34;Classification: {classification}&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;
&lt;/div&gt;
&lt;/div&gt;&lt;h3 id="scenario-2-sentiment-analysis"&gt;Scenario 2: Sentiment Analysis
&lt;/h3&gt;&lt;div class="highlight"&gt;&lt;div class="chroma"&gt;
&lt;table class="lntable"&gt;&lt;tr&gt;&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code&gt;&lt;span class="lnt"&gt; 1
&lt;/span&gt;&lt;span class="lnt"&gt; 2
&lt;/span&gt;&lt;span class="lnt"&gt; 3
&lt;/span&gt;&lt;span class="lnt"&gt; 4
&lt;/span&gt;&lt;span class="lnt"&gt; 5
&lt;/span&gt;&lt;span class="lnt"&gt; 6
&lt;/span&gt;&lt;span class="lnt"&gt; 7
&lt;/span&gt;&lt;span class="lnt"&gt; 8
&lt;/span&gt;&lt;span class="lnt"&gt; 9
&lt;/span&gt;&lt;span class="lnt"&gt;10
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;
&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-csharp" data-lang="csharp"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;using&lt;/span&gt; &lt;span class="nn"&gt;Microsoft.SemanticKernel&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;using&lt;/span&gt; &lt;span class="nn"&gt;Microsoft.SemanticKernel.Models&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;kernel&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="n"&gt;SemanticKernel&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;model&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;kernel&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;LoadModel&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;&amp;#34;sentiment-analysis-model&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;inputText&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;&amp;#34;I love using Azure AI services!&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;sentiment&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;AnalyzeSentiment&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;inputText&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;Console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;WriteLine&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;$&amp;#34;Sentiment: {sentiment}&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;
&lt;/div&gt;
&lt;/div&gt;&lt;h3 id="scenario-3-named-entity-recognition-ner"&gt;Scenario 3: Named Entity Recognition (NER)
&lt;/h3&gt;&lt;div class="highlight"&gt;&lt;div class="chroma"&gt;
&lt;table class="lntable"&gt;&lt;tr&gt;&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code&gt;&lt;span class="lnt"&gt; 1
&lt;/span&gt;&lt;span class="lnt"&gt; 2
&lt;/span&gt;&lt;span class="lnt"&gt; 3
&lt;/span&gt;&lt;span class="lnt"&gt; 4
&lt;/span&gt;&lt;span class="lnt"&gt; 5
&lt;/span&gt;&lt;span class="lnt"&gt; 6
&lt;/span&gt;&lt;span class="lnt"&gt; 7
&lt;/span&gt;&lt;span class="lnt"&gt; 8
&lt;/span&gt;&lt;span class="lnt"&gt; 9
&lt;/span&gt;&lt;span class="lnt"&gt;10
&lt;/span&gt;&lt;span class="lnt"&gt;11
&lt;/span&gt;&lt;span class="lnt"&gt;12
&lt;/span&gt;&lt;span class="lnt"&gt;13
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;
&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-csharp" data-lang="csharp"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;using&lt;/span&gt; &lt;span class="nn"&gt;Microsoft.SemanticKernel&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;using&lt;/span&gt; &lt;span class="nn"&gt;Microsoft.SemanticKernel.Models&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;kernel&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="n"&gt;SemanticKernel&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;model&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;kernel&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;LoadModel&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;&amp;#34;ner-model&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;inputText&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;&amp;#34;Microsoft was founded by Bill Gates and Paul Allen.&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;entities&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;RecognizeEntities&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;inputText&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;foreach&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;entity&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;entities&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;Console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;WriteLine&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;$&amp;#34;Entity: {entity.Name}, Type: {entity.Type}&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;
&lt;/div&gt;
&lt;/div&gt;&lt;h3 id="scenario-4-question-answering"&gt;Scenario 4: Question Answering
&lt;/h3&gt;&lt;div class="highlight"&gt;&lt;div class="chroma"&gt;
&lt;table class="lntable"&gt;&lt;tr&gt;&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code&gt;&lt;span class="lnt"&gt; 1
&lt;/span&gt;&lt;span class="lnt"&gt; 2
&lt;/span&gt;&lt;span class="lnt"&gt; 3
&lt;/span&gt;&lt;span class="lnt"&gt; 4
&lt;/span&gt;&lt;span class="lnt"&gt; 5
&lt;/span&gt;&lt;span class="lnt"&gt; 6
&lt;/span&gt;&lt;span class="lnt"&gt; 7
&lt;/span&gt;&lt;span class="lnt"&gt; 8
&lt;/span&gt;&lt;span class="lnt"&gt; 9
&lt;/span&gt;&lt;span class="lnt"&gt;10
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;
&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-csharp" data-lang="csharp"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;using&lt;/span&gt; &lt;span class="nn"&gt;Microsoft.SemanticKernel&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;using&lt;/span&gt; &lt;span class="nn"&gt;Microsoft.SemanticKernel.Models&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;kernel&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="n"&gt;SemanticKernel&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;model&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;kernel&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;LoadModel&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;&amp;#34;question-answering-model&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;question&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;&amp;#34;What is Azure AI?&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;answer&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;AnswerQuestion&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;question&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;Console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;WriteLine&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;$&amp;#34;Answer: {answer}&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;
&lt;/div&gt;
&lt;/div&gt;&lt;h2 id="conclusion"&gt;Conclusion
&lt;/h2&gt;&lt;p&gt;Semantic Kernel is a powerful tool that enhances the capabilities of AI models by providing a deeper understanding of language and context. By leveraging Semantic Kernel, developers can build more intelligent and responsive applications that go beyond simple keyword matching. Whether you&amp;rsquo;re working on text classification, sentiment analysis, named entity recognition, or question answering, Semantic Kernel offers the tools and frameworks needed to create sophisticated AI solutions. As AI continues to evolve, tools like Semantic Kernel will play a crucial role in shaping the future of intelligent applications.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;I hope this article provides you with a comprehensive overview of Semantic Kernel and its applications. If you have any questions or need further details, feel free to ask! In a next blog post, we&amp;rsquo;ll go over several use cases with many more code snippets, to give you enough examples to start building/developing your own Copilots. Stay tuned!&lt;/p&gt;
&lt;p&gt;&lt;a class="link" href="https://www.buymeacoffee.com/pdtit" target="_blank" rel="noopener"
 &gt;&lt;img src="../images/screenshot-2024-09-15-17f576e7.png"
 
 
 loading="lazy"
 
 alt="BuyMeACoffee"
 
 
 
 &gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Cheers!!&lt;/p&gt;
&lt;p&gt;/Peter&lt;/p&gt;</description></item><item><title>Festive Tech Calendar - Azure AI - OCR on Steroids</title><link>https://www.pdtit.be/post/festive-2023-ocr-on-steroids/</link><pubDate>Sat, 23 Dec 2023 00:00:00 +0000</pubDate><guid>https://www.pdtit.be/post/festive-2023-ocr-on-steroids/</guid><description>&lt;p&gt;&lt;strong&gt;&lt;a class="link" href="https://www.gohugo.io" target="_blank" rel="noopener"
 &gt;Hugo&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;a class="link" href="https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blob-static-website-how-to?tabs=azure-portal" target="_blank" rel="noopener"
 &gt;Azure Static Storage Sites&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-4f833e5c.jpg"
 
 
 loading="lazy"
 
 alt="Festive Tech Calendar 2023 - OCR on Steroids image"
 
 
 
 &gt;&lt;/p&gt;
&lt;h1 id="welcome-to-this-years-festive-tech-calendar"&gt;Welcome to this year&amp;rsquo;s Festive Tech Calendar!!
&lt;/h1&gt;&lt;p&gt;Hi everyone, welcome to my contribution to this year&amp;rsquo;s Festive Tech Calendar once more. This will be the fourth year, and I still love the concept of bringing some (Azure) joy to you/your family this season. If you ask me what the biggest news in tech was this year, especially within the Microsoft ecosystem, it&amp;rsquo;s &lt;strong&gt;Azure AI&lt;/strong&gt;. Shouldn&amp;rsquo;t be surprising to most of you who know me and my role within Microsoft as Technical Trainer - providing Azure workshops every week to our top customers and partners across the globe - we also integrated (A lot :)) of AI focus early in the year. And it is only becoming more imported.&lt;/p&gt;
&lt;p&gt;Therefore, I decided to bring you an &lt;strong&gt;Azure AI-inspired topic&lt;/strong&gt; for this year&amp;rsquo;s Festive Tech Calendar session, using &lt;strong&gt;Computer Vision - Document Intelligence&lt;/strong&gt;, or what I describe as &lt;strong&gt;OCR on steroids&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;In the late 1920s and into the 1930s, &lt;a class="link" href="https://en.wikipedia.org/wiki/Optical_character_recognition" target="_blank" rel="noopener"
 &gt;Emanuel Goldberg&lt;/a&gt; developed what he called a &amp;ldquo;Statistical Machine&amp;rdquo; for searching microfilm archives using an optical code recognition system, which evolved into an IBM OCR solution.&lt;/p&gt;
&lt;p&gt;about 100 years after, OCR is transitioning into powerful document and text analysis capabilities, thanks to Azure AI Document Intelligence APIs.&lt;/p&gt;
&lt;p&gt;By &lt;strong&gt;reading through this post&lt;/strong&gt;, and &lt;strong&gt;following the demo-steps&lt;/strong&gt;, you can build your own &amp;ldquo;Statistical Machine&amp;rdquo; in no-time. And from there, learn about Azure AI Document Intelligence APIs, to take it even further&amp;hellip; let&amp;rsquo;s go!&lt;/p&gt;
&lt;p&gt;To embrace our Azure AI and Microsoft Copilot even more myself, I actually used it to create (parts) of this blog post. What a wonderful world we live in today!&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-2a410c72.png"
 
 
 loading="lazy"
 
 alt="Office 365 Copilot"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;About 2 weeks ago, I presented a session on the same topic for the &lt;strong&gt;&lt;a class="link" href="https://globalai.community/conference" target="_blank" rel="noopener"
 &gt;GlobalAI Community Conference&lt;/a&gt;&lt;/strong&gt;, led by &lt;strong&gt;&lt;a class="link" href="https://twitter.com/SjoukjeZaal" target="_blank" rel="noopener"
 &gt;Sjoukje Zaal&lt;/a&gt;, &lt;a class="link" href="https://twitter.com/AmyKateNicho" target="_blank" rel="noopener"
 &gt;Amy Kate Boyd&lt;/a&gt; and &lt;a class="link" href="https://twitter.com/hboelman" target="_blank" rel="noopener"
 &gt;Henk Boelman&lt;/a&gt;&lt;/strong&gt;, for which the video is available &lt;a class="link" href="https://www.youtube.com/watch?v=G860AXhO9lg" target="_blank" rel="noopener"
 &gt;On Youtube&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;So instead of creating a similar video, I worked with Azure AI, Microsoft Copilot and my own notes, to produce this article. Let me know if you liked it&amp;hellip;&lt;/p&gt;
&lt;p&gt;In this article, I will use the following flow:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Iâ€™ll start with setting the scene on Azure AI, using Computer Vision for OCR&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Followed by the more advanced scenario, using Intelligent Document Processing or IDP&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Last, Iâ€™ll show you how you can train the IDP using your own custom models&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;And I hope to do all this using several demosâ€¦ which you can go through in your own Azure subscriptions&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h1 id="setting-the-scene-on-azure-ai-using-computer-vision-for-ocr"&gt;setting the scene on Azure AI, using Computer Vision for OCR
&lt;/h1&gt;&lt;p&gt;Computer Vision allows for different use cases, of which the most important ones are:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Image Analysis&lt;/strong&gt; â€“ typically used to detect objects or items&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Spatial Analysis&lt;/strong&gt; â€“ is what you would use to detect people, like video cameras in a store&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;OCR or Optical Character Recognition&lt;/strong&gt; â€“ allows you to recognize text, both printed and handwritten&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Facial Recognition&lt;/strong&gt; â€“ recognize human identity, without exposing privacy details&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-0d183516.png"
 
 
 loading="lazy"
 
 alt="Computer Vision"
 
 
 
 &gt;&lt;/p&gt;
&lt;h2 id="deploying-azure-ai---computer-vision"&gt;Deploying Azure AI - Computer Vision
&lt;/h2&gt;&lt;p&gt;If you donâ€™t already have one in your subscription, youâ€™ll need to provision an Azure AI Services resource. If you don&amp;rsquo;t have an Azure subscription yet, you can sign up for a &lt;strong&gt;[Free subscription]&lt;/strong&gt;(&lt;a class="link" href="https://azure.microsoft.com/en-us/free%29" target="_blank" rel="noopener"
 &gt;https://azure.microsoft.com/en-us/free)&lt;/a&gt;.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Open the Azure portal at &lt;a class="link" href="https://portal.azure.com" target="_blank" rel="noopener"
 &gt;https://portal.azure.com&lt;/a&gt;, and sign in using the Microsoft account associated with your Azure subscription.&lt;/li&gt;
&lt;li&gt;In the top search bar, search for &lt;strong&gt;Azure AI services&lt;/strong&gt;, select Azure AI Services, and create an Azure AI services &lt;strong&gt;multi-service account resource&lt;/strong&gt; with the following settings:&lt;/li&gt;
&lt;/ol&gt;
&lt;ul&gt;
&lt;li&gt;Subscription: Your Azure subscription&lt;/li&gt;
&lt;li&gt;Resource group: Choose or create a resource group (if you are using a restricted subscription, you may not have permission to create a new resource group - use the one provided)&lt;/li&gt;
&lt;li&gt;Region: Choose from East US, France Central, Korea Central, North Europe, Southeast Asia, West Europe, West US, or East Asia*&lt;/li&gt;
&lt;li&gt;Name: Enter a unique name&lt;/li&gt;
&lt;li&gt;Pricing tier: Standard S0
*Azure AI Vision 4.0 features are currently only available in these regions.&lt;/li&gt;
&lt;/ul&gt;
&lt;ol start="3"&gt;
&lt;li&gt;Select the required checkboxes and &lt;strong&gt;create the resource&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Wait for deployment to complete, and then view the &lt;strong&gt;deployment details&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;When the resource has been deployed, go to it and view its &lt;strong&gt;Keys and Endpoint&lt;/strong&gt; page. This is what you would need from a development perspective in your application source code. I will show you an example on how to use a ComputerVision Docker Container later on which also needs those parameters to run successfully&amp;hellip;&lt;/li&gt;
&lt;li&gt;From the &lt;strong&gt;ComputerVision&lt;/strong&gt; tab within &lt;strong&gt;Azure AI Services&lt;/strong&gt;, create a new Computer Vision Resource, keeping most default settings as-is.&lt;/li&gt;
&lt;li&gt;Wait for deployment to complete. From the &lt;strong&gt;Overview&lt;/strong&gt; section of Computer Vision, notice the &lt;strong&gt;Vision Studio&lt;/strong&gt; button.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-cdcd7414.png"
 
 
 loading="lazy"
 
 alt="Azure AI Computer Vision"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="8"&gt;
&lt;li&gt;&lt;strong&gt;Click&lt;/strong&gt; the &lt;strong&gt;Open Vision Studio&lt;/strong&gt; button to navigate to Azure AI Compute Vision Studio.&lt;/li&gt;
&lt;li&gt;From here, select &lt;strong&gt;Optical Character Recognition&lt;/strong&gt; , and select &lt;strong&gt;Extract text from images&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;Here, you can test the functionality of how text is getting recognized, using the sample images provided, or you can upload your own images as well. I would recommend you to try with handwritten notes as well, especially when your handwriting skills are as great as mine&amp;hellip;&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-9fa0ac56.png"
 
 
 loading="lazy"
 
 alt="Azure AI Computer Vision Recognizing handwriting"
 
 
 
 &gt;&lt;/p&gt;
&lt;h2 id="how-does-ocr-recognition-actually-work"&gt;How does OCR recognition actually work?
&lt;/h2&gt;&lt;p&gt;When we talk about Azure AI, it means using APIs, which allow you to bring in a source into the AI engine, from there processes a given scenario â€“ like read text in the case of OCR, and from there we call the result&lt;/p&gt;
&lt;p&gt;Looking at this example, and Iâ€™ll show you in a quick demo, each item of text gets moved into a box/a boundary, which gets translated into understandable characters -&amp;gt; forming words&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Step 1&lt;/strong&gt; involves &lt;strong&gt;creating a requestID&lt;/strong&gt;
&lt;strong&gt;Step 2&lt;/strong&gt; means &lt;strong&gt;reading out the results, for the specific requestID&lt;/strong&gt;&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Going back to the &lt;strong&gt;Vision Studio&lt;/strong&gt; in the Azure Portal, where you uploaded or selected an image, and saw the outcome of how text gets recognized, select &lt;strong&gt;JSON&lt;/strong&gt;.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-ff48c684.png"
 
 
 loading="lazy"
 
 alt="Azure AI Computer Vision Boundbox"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="2"&gt;
&lt;li&gt;For each character or set of characters recognized as text, different JSON properties and values are getting created. These identify the boundaries of the text on the image.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-10521604.png"
 
 
 loading="lazy"
 
 alt="Azure AI Computer Vision Boundbox"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="3"&gt;
&lt;li&gt;
&lt;p&gt;This is the core work of the &lt;strong&gt;Read API&lt;/strong&gt;. So let&amp;rsquo;s have a more detailed look into that one for a second. The easiest I found to show this, is spinning up a &lt;strong&gt;Azure Cognitive Service Vision&lt;/strong&gt; Docker Container.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Assuming you know a bit about Docker containers, and you have Docker Engine running on your local machine, execute the following command:&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;div class="chroma"&gt;
&lt;table class="lntable"&gt;&lt;tr&gt;&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code&gt;&lt;span class="lnt"&gt;1
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;
&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-fallback" data-lang="fallback"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;docker run --rm -it -p 5050:5000 --memory 4g --cpus 1 mcr.microsoft.com/azure-cognitive-services/vision/read:3.2-model-2022-04-30 Eula=accept Billing=https://yourcomputevisionresource.cognitiveservices.azure.com/ ApiKey=yourcomputevisionapikey
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;
&lt;/div&gt;
&lt;/div&gt;&lt;p&gt;Replacing the &lt;strong&gt;Billing&lt;/strong&gt; and &lt;strong&gt;ApiKey&lt;/strong&gt; with the correct values from the Computer Vision Keys picked up earlier.&lt;/p&gt;
&lt;ol start="5"&gt;
&lt;li&gt;With the Docker container running, open the browser to &lt;strong&gt;http://localhost:5050/status&lt;/strong&gt;, which confirms the ReadAPI service is ready, and your API Key is valid&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-b672b551.png"
 
 
 loading="lazy"
 
 alt="Azure AI Computer Vision Docker Ready"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="6"&gt;
&lt;li&gt;Next, connecto to &lt;strong&gt;http://localhost:5050/swagger&lt;/strong&gt; to interact with the different API endpoints of the Language Service running within the container&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-da29ca8d.png"
 
 
 loading="lazy"
 
 alt="Azure AI Computer Vision Read API Swagger"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="7"&gt;
&lt;li&gt;From the swagger API interface, select &lt;strong&gt;POST&lt;/strong&gt; in the &lt;strong&gt;Analyze&lt;/strong&gt; section, and click &lt;strong&gt;Try it out&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Scroll down a bit, and in the &lt;strong&gt;Request body&lt;/strong&gt; section, provide a URL to an actual image. For example, you can use the sample image below, which is the same one available in the Vision Studio portal, showing the nutrition facts about some food item.&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;div class="chroma"&gt;
&lt;table class="lntable"&gt;&lt;tr&gt;&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code&gt;&lt;span class="lnt"&gt;1
&lt;/span&gt;&lt;span class="lnt"&gt;2
&lt;/span&gt;&lt;span class="lnt"&gt;3
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;
&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-fallback" data-lang="fallback"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;{
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &amp;#34;url&amp;#34;: &amp;#34;https://raw.githubusercontent.com/Azure-Samples/cognitive-services-sample-data-files/master/ComputerVision/Images/printed_text.jpg&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;}
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;
&lt;/div&gt;
&lt;/div&gt;&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-a92ee081.png"
 
 
 loading="lazy"
 
 alt="Azure AI Computer Vision Read API Request Body"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="9"&gt;
&lt;li&gt;Click the &lt;strong&gt;Execute&lt;/strong&gt; button.&lt;/li&gt;
&lt;li&gt;This sends an API request and returns an analyze request ID from this URL: http://localhost:5000/vision/v3.2/read/analyze&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-feee6d2d.png"
 
 
 loading="lazy"
 
 alt="Azure AI Computer Vision Read API Response"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="11"&gt;
&lt;li&gt;
&lt;p&gt;Copy the request id, and open it in the browser, e.g. http://localhost:5050/vision/v3.2/read/analyzeResults/339da9a7-aa6b-4c81-a5d0-5840448fdfaf&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;This brings up the full JSON structure of the analyzed text, including text snippets, bounding boxes,â€¦&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-5d8280b5.png"
 
 
 loading="lazy"
 
 alt="Azure AI Computer Vision Read API JSON Response"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;While quite impressive if you ask me, we had OCR technology doing almost the same, for the last 50-60 years already. Anyone remembers copiers and scanners, saving to a PDF document? Basically based on the sameâ€¦&lt;/p&gt;
&lt;p&gt;So letâ€™s focus a bit more on the next level of OCR, using Intelligent Document Processing&lt;/p&gt;
&lt;h1 id="using-intelligent-document-processing-or-idp"&gt;Using Intelligent Document Processing or IDP
&lt;/h1&gt;&lt;p&gt;Similar to the Computer Vision Read API and Vision Studio, Azure AI also provides the &lt;strong&gt;Form Recognizer Service&lt;/strong&gt; , which got recently renamed to &lt;strong&gt;Intelligent Document Processing&lt;/strong&gt; or IDP.&lt;/p&gt;
&lt;p&gt;Document Intelligence Read Optical Character Recognition (OCR) model runs at a higher resolution than Azure AI Vision Read and extracts print and handwritten text from PDF documents and scanned images. It also includes support for extracting text from Microsoft Word, Excel, PowerPoint, and HTML documents. It detects paragraphs, text lines, words, locations, and languages. The Read model is the underlying OCR engine for other Document Intelligence prebuilt models like Layout, General Document, Invoice, Receipt, Identity (ID) document, Health insurance card, W2 in addition to custom models.&lt;/p&gt;
&lt;p&gt;Optical Character Recognition (OCR) for documents is optimized for large text-heavy documents in multiple file formats and global languages. It includes features like higher-resolution scanning of document images for better handling of smaller and dense text; paragraph detection; and fillable form management. OCR capabilities also include advanced scenarios like single character boxes and accurate extraction of key fields commonly found in invoices, receipts, and other prebuilt scenarios.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;From the Azure AI Portal, navigate to &lt;strong&gt;Document Intelligence&lt;/strong&gt;, and create a new resource within. Default settings should be ok as-is.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-c3396348.png"
 
 
 loading="lazy"
 
 alt="Azure AI Document Intelligence"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="2"&gt;
&lt;li&gt;Once deployed, from the &lt;strong&gt;Overview&lt;/strong&gt; section, notice the &lt;strong&gt;Document Intelligence Studio&lt;/strong&gt; option, and open it.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-6c02cf38.png"
 
 
 loading="lazy"
 
 alt="Azure AI Document Intelligence Studio"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;In the previous examples, the content was coming from an image-file type (jpeg,â€¦). Where sometimes, we have more specific data types, such as forms, receipts, invoices, passport,â€¦&lt;/p&gt;
&lt;p&gt;This is where the compute vision text analyzer is not finetuned enough. Thatâ€™s where we will use the form recognizer service, now known as document intelligence service.&lt;/p&gt;
&lt;p&gt;Azure AI Document Intelligence is a cloud-basedÂ Azure AI serviceÂ that enables you to build intelligent document processing solutions. Massive amounts of data, spanning a wide variety of data types, are stored in forms and documents. Document Intelligence enables you to effectively manage the velocity at which data is collected and processed and is key to improved operations, informed data-driven decisions, and enlightened innovation.&lt;/p&gt;
&lt;p&gt;Document Intelligence recognizes 3 different models:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Document Analysis&lt;/strong&gt; - enable text extraction from forms and documents and return structured business-ready content ready for your organization&amp;rsquo;s action, use, or progress.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-ec0dc870.png"
 
 
 loading="lazy"
 
 alt="Azure AI Document Intelligence Studio"
 
 
 
 &gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Prebuilt models&lt;/strong&gt; - Prebuilt models enable you to add intelligent document processing to your apps and flows without having to train and build your own models.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-3a6aafe7.png"
 
 
 loading="lazy"
 
 alt="Azure AI Document Prebuilt Models"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;From the Document Intelligence Studio, select a prebuilt model type of choice. I&amp;rsquo;ll use &lt;strong&gt;Invoice&lt;/strong&gt; but the approach is the same for the other ones.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-4daf4b79.png"
 
 
 loading="lazy"
 
 alt="Azure AI Document Prebuilt Invoice Models"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="2"&gt;
&lt;li&gt;Click &lt;strong&gt;Run Analysis&lt;/strong&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-8b5e58e2.png"
 
 
 loading="lazy"
 
 alt="Azure AI Document Prebuilt Invoice Analysis"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="3"&gt;
&lt;li&gt;As you can see, the different text items from the document (Invoice) are &lt;strong&gt;getting identified&lt;/strong&gt; and &lt;strong&gt;tagged with a label&lt;/strong&gt;. Similar to before, the technical output is stored in a &lt;strong&gt;JSON file&lt;/strong&gt;.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-f50cd78f.png"
 
 
 loading="lazy"
 
 alt="Azure AI Document Prebuilt Invoice Analysis JSON output"
 
 
 
 &gt;&lt;/p&gt;
&lt;h1 id="how-to-train-the-idp-using-your-own-custom-models"&gt;How to train the IDP using your own custom models
&lt;/h1&gt;&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Custom models&lt;/strong&gt; - Custom models are trained using your labeled datasets to extract distinct data from forms and documents, specific to your use cases. Standalone custom models can be combined to create composed models.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Document Intelligence uses advanced machine learning technology to identify documents, detect and extract information from forms and documents, and return the extracted data in a structured JSON output. With Document Intelligence, you can use document analysis models, pre-built/pre-trained, or your trained standalone custom models.&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-9adabb58.png"
 
 
 loading="lazy"
 
 alt="Azure AI Document Intelligence - Custom Model"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;Custom models now include &lt;strong&gt;custom classification models&lt;/strong&gt; for scenarios where you need to identify the document type prior to invoking the extraction model. Classifier models are available starting with the 2023-07-31 (GA) API. A classification model can be paired with a custom extraction model to analyze and extract fields from forms and documents specific to your business to create a document processing solution. Standalone custom extraction models can be combined to create composed models.&lt;/p&gt;
&lt;p&gt;Custom document models can be one of two types, custom template or custom form and custom neural or custom document models. The labeling and training process for both models is identical, but the models differ as follows:&lt;/p&gt;
&lt;p&gt;Custom extraction models
To create a custom extraction model, label a dataset of documents with the values you want extracted and train the model on the labeled dataset. You only need five examples of the same form or document type to get started.&lt;/p&gt;
&lt;p&gt;Custom neural model
The custom neural (custom document) model uses deep learning models and base model trained on a large collection of documents. This model is then fine-tuned or adapted to your data when you train the model with a labeled dataset. Custom neural models support structured, semi-structured, and unstructured documents to extract fields. Custom neural models currently support English-language documents. When you&amp;rsquo;re choosing between the two model types, start with a neural model to determine if it meets your functional needs. See neural models to learn more about custom document models.&lt;/p&gt;
&lt;p&gt;The custom template or custom form model relies on a consistent visual template to extract the labeled data. Variances in the visual structure of your documents affect the accuracy of your model. Structured forms such as questionnaires or applications are examples of consistent visual templates.&lt;/p&gt;
&lt;p&gt;Your training set consists of structured documents where the formatting and layout are static and constant from one document instance to the next. Custom template models support key-value pairs, selection marks, tables, signature fields, and regions. Template models and can be trained on documents in any of the supported languages. For more information, see custom template models.&lt;/p&gt;
&lt;p&gt;If the language of your documents and extraction scenarios supports custom neural models, we recommend that you use custom neural models over template models for higher accuracy.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;From &lt;strong&gt;Document Intelligence Studio&lt;/strong&gt;, scroll down and select &lt;strong&gt;Custom Models&lt;/strong&gt;. Next, select &lt;strong&gt;Custom Extraction Model&lt;/strong&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;From here, create &lt;strong&gt;a new project&lt;/strong&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-a21a0a69.png"
 
 
 loading="lazy"
 
 alt="Azure AI Document Custom Model"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="3"&gt;
&lt;li&gt;
&lt;p&gt;You need to specify where your document model sources (e.g. I am using my electricity bills, but could be receipts, delivery notes, business forms,&amp;hellip;) can be found in Azure Blob Storage, but apart from that, all settings should be clear I hope.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Open the &lt;strong&gt;project&lt;/strong&gt; you created, from where it allows you to &lt;strong&gt;upload sample files&lt;/strong&gt;. &lt;strong&gt;Upload 5-10 identical documents, as that is what is required to train the model&lt;/strong&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-03c382bd.png"
 
 
 loading="lazy"
 
 alt="Azure AI Document Custom Label Data"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="5"&gt;
&lt;li&gt;
&lt;p&gt;You can choose to use the &lt;strong&gt;Auto Label&lt;/strong&gt; feature, or &lt;strong&gt;Draw Region&lt;/strong&gt; to establish the different labels for the different text sections of your documents.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Once done for several identical documents, click &lt;strong&gt;Train&lt;/strong&gt;. Which allows you to create an AI Data Model.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Last, you can run a &lt;strong&gt;test&lt;/strong&gt; to validate how a new custom document gets analyzed and all items in the document will get recognized.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h1 id="summary"&gt;Summary
&lt;/h1&gt;&lt;p&gt;In this article, I wanted to introduce you to the exciting world of Azure AI, and more specifically how &lt;strong&gt;Computer Vision&lt;/strong&gt; can be used to recognize text in images, known as &lt;strong&gt;OCR&lt;/strong&gt;, but even more so, how Azure AI &lt;strong&gt;Document Intelligence API&lt;/strong&gt; allows to bring in more advanced document template recognition into your applications.&lt;/p&gt;
&lt;p&gt;Thanks to the Azure Festive Tech Calendar team for having me! Take care and Happy Holidays!&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-12-23-591067f1.png"
 
 
 loading="lazy"
 
 alt="Thank you "
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;&lt;a class="link" href="https://www.buymeacoffee.com/pdtit" target="_blank" rel="noopener"
 &gt;&lt;img src="../images/screenshot-2023-12-23-17f576e7.png"
 
 
 loading="lazy"
 
 alt="BuyMeACoffee"
 
 
 
 &gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Cheers!!&lt;/p&gt;
&lt;p&gt;/Peter&lt;/p&gt;</description></item><item><title>Build an Azure AI chatbot using your own data from blob storage</title><link>https://www.pdtit.be/post/build-an-azure-ai-chatbot-using-your-own-data-in-blob-storage/</link><pubDate>Mon, 17 Jul 2023 00:00:00 +0000</pubDate><guid>https://www.pdtit.be/post/build-an-azure-ai-chatbot-using-your-own-data-in-blob-storage/</guid><description>&lt;p&gt;You must have lived under a rock if you didn&amp;rsquo;t hear about how important &lt;a class="link" href="https://azure.microsoft.com/en-us/solutions/ai/" target="_blank" rel="noopener"
 &gt;Azure AI&lt;/a&gt; is for Microsoft and its partner and customer ecosystem. Thanks to Artificial Intelligence (AI), companies will be more innovative, employees will be more productive. While I honestly was a bit hesitant at first myself - knowing a big part of my job is providing training, and seeing what AI can do here in terms of content creation, video creation and alike, yes the trainer role will definitely (has to) change over the next coming months - once I started digging into its capabilities more, I started to see the AI engine potential.&lt;/p&gt;
&lt;p&gt;To me, AI capabilities are here to support us, think of it as a facilitator, a coach, someone who&amp;rsquo;s walking the path with you. But you&amp;rsquo;re still in control and deciding which way to go, when and where.&lt;/p&gt;
&lt;p&gt;Over the last 18 months, I&amp;rsquo;ve been developing an app for your internal Microsoft Trainer team, using a combination of Azure DevOps Pipelines, Blazor .NET and Azure Blob Storage for storing the guidelines and documentation, allowing them to quickly deploy Azure demo scenarios using a self-service portal. One of the missing features in the app, was a decent search capability. Allowing trainers to search for Azure Resources keywords or demo scenarios.&lt;/p&gt;
&lt;p&gt;With Azure AI Studio being available, promising a smooth experience for building AI-integrated solutions, I saw this as the perfect candidate for my missing search feature. What if I could provide a chat bot, allowing trainers to ask natural language-based questions, where the answer would be a summary of demo steps to showcase, or pulling up the full demoguide document? Sounds amazing, right?&lt;/p&gt;
&lt;p&gt;I gotta say, it&amp;rsquo;s amazing. Especially because it took me &lt;strong&gt;less than 30 minutes&lt;/strong&gt; (including making mistakes or missing steps, so technically you can do this in &lt;strong&gt;less than 15min&lt;/strong&gt; now :) )&lt;/p&gt;
&lt;h1 id="what-you-will-build"&gt;What you will build
&lt;/h1&gt;&lt;p&gt;In this article, you will learn how to use Azure AI and Azure AI Studio, to deploy a chat bot which connects to Azure Blob Storage and uses your own markdown files as input for providing answers to questions.&lt;/p&gt;
&lt;h2 id="what-you-need"&gt;What you need
&lt;/h2&gt;&lt;p&gt;In order to follow below steps and succeed in getting your first Azure AI Chat bot up-and-running, you need to meet these prerequisites:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;An Azure Subscription (you can use the &lt;a class="link" href="https://azure.microsoft.com/free/cognitive-services" target="_blank" rel="noopener"
 &gt;Azure Free Subscription&lt;/a&gt; link if you don&amp;rsquo;t have one yet)&lt;/li&gt;
&lt;li&gt;Access to &lt;strong&gt;Azure OpenAI&lt;/strong&gt; in your Azure Subscription. Complete the request form &lt;a class="link" href="https://aka.ms/oai/access" target="_blank" rel="noopener"
 &gt;here&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;**Note: access to Azure OpenAI requires company details, it doesn&amp;rsquo;t work for private/personal accounts.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Having &lt;a class="link" href="https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/role-based-access-control#cognitive-services-contributor" target="_blank" rel="noopener"
 &gt;Cognitive Services Contributor&lt;/a&gt; or higher permissions in your Azure Subscription.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;An &lt;strong&gt;Azure Blob Storage account&lt;/strong&gt; with at least 1 container. The sample files will be uploaded as part of the data source selection steps later on.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;With the prerequisites validated, you are ready to move on with the base setup of Azure AI Chat Playground using the following steps.&lt;/p&gt;
&lt;h2 id="deploying-azure-ai-chat-playground-using-azure-ai-studio"&gt;Deploying Azure AI Chat Playground using Azure AI Studio
&lt;/h2&gt;&lt;ol&gt;
&lt;li&gt;Open &lt;a class="link" href="https://oai.azure.com/" target="_blank" rel="noopener"
 &gt;Azure AI Studio&lt;/a&gt; from your browser, using your Azure admin account credentials.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-ddfb8eee.png"
 
 
 loading="lazy"
 
 alt="Azure OpenAI Studio"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="2"&gt;
&lt;li&gt;
&lt;p&gt;Select your &lt;strong&gt;Azure Subscription&lt;/strong&gt;, and click &lt;strong&gt;Create Resource&lt;/strong&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;This opens the &lt;strong&gt;Create Azure OpenAI Resource&lt;/strong&gt; blade. Here, complete the necessary parameters:&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;ul&gt;
&lt;li&gt;Azure Subscription&lt;/li&gt;
&lt;li&gt;Existing or New Azure Resource Group&lt;/li&gt;
&lt;li&gt;Azure Region of choice&lt;/li&gt;
&lt;li&gt;Unique Name for the Azure OpenAI Resource&lt;/li&gt;
&lt;li&gt;Pricing Tier: &lt;strong&gt;Select Standard S0&lt;/strong&gt; (The Free Basic won&amp;rsquo;t allow us to use the necessary Cognitive Search later on)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-9a216e8d.png"
 
 
 loading="lazy"
 
 alt="Azure OpenAI Resource"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="4"&gt;
&lt;li&gt;Wait for the Azure OpenAI Resource to get deployed, and navigate to the resource once its ready.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-9435dc38.png"
 
 
 loading="lazy"
 
 alt="Azure OpenAI Resource"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="5"&gt;
&lt;li&gt;Navigate back to &lt;a class="link" href="https://oai.azure.com/" target="_blank" rel="noopener"
 &gt;Azure OpenAI Studio&lt;/a&gt;, which will allow you to select your Azure AI Resource created earlier.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-23c291bb.png"
 
 
 loading="lazy"
 
 alt="Azure OpenAI Studio"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="6"&gt;
&lt;li&gt;From Azure OpenAI Studio, select &lt;strong&gt;Chat Playground&lt;/strong&gt; from the &lt;strong&gt;Get Started&lt;/strong&gt; options.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-6337eff8.png"
 
 
 loading="lazy"
 
 alt="Azure OpenAI Chat Playground"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="7"&gt;
&lt;li&gt;Within the &lt;strong&gt;Chat Playground&lt;/strong&gt;, click the &lt;strong&gt;Create new deployment&lt;/strong&gt; button to set up a new Chat Plaground.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-dca9553e.png"
 
 
 loading="lazy"
 
 alt="Azure OpenAI Chat Playground"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="8"&gt;
&lt;li&gt;In the next step, you will create the AI model needed for the Cognitive Search later on. Complete the necessary settings:&lt;/li&gt;
&lt;/ol&gt;
&lt;ul&gt;
&lt;li&gt;Model: &lt;strong&gt;gpt-35-turbo&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;Model Version: &lt;strong&gt;Auto-update-to-default&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;Deployment Name: descriptive name of what the model is about&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-e194d150.png"
 
 
 loading="lazy"
 
 alt="Deploy Model"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;This completes the first part of the steps, where you deployed Azure AI Chat Playground using Azure AI Studio. In the next step, you add the data source which will be used for the chat content.&lt;/p&gt;
&lt;h2 id="adding-your-own-data-sources-blob-storage-to-the-azure-ai-chat-playground"&gt;Adding your own data sources (Blob Storage) to the Azure AI Chat Playground
&lt;/h2&gt;&lt;ol&gt;
&lt;li&gt;With the model created, we can move on to the next step, adding data. From the &lt;strong&gt;Chat Playground&lt;/strong&gt;, select &lt;strong&gt;Add your data (preview)&lt;/strong&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-7d1a86c6.png"
 
 
 loading="lazy"
 
 alt="Chat Playground add your data"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="2"&gt;
&lt;li&gt;From the &lt;strong&gt;Add Data&lt;/strong&gt; blade, complete the necessary settings and parameters:&lt;/li&gt;
&lt;/ol&gt;
&lt;ul&gt;
&lt;li&gt;Azure Subscription&lt;/li&gt;
&lt;li&gt;Azure Blob Storage Account&lt;/li&gt;
&lt;li&gt;Azure Blob Storage Account container&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-01070d5a.png"
 
 
 loading="lazy"
 
 alt="Chat Playground add your data"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="3"&gt;
&lt;li&gt;
&lt;p&gt;Note how it asks for an Azure Cognitive Search resource as well; this is to be able to read the actual content of the blobs, such as Word documents, PDF files, etc&amp;hellip;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Click the &lt;strong&gt;Create a new Azure Cognitive Search Resource&lt;/strong&gt; link to get this resource created.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;From the &lt;strong&gt;Create a search service&lt;/strong&gt; blade, enter the necessary parameters for the resource creation:&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;ul&gt;
&lt;li&gt;Azure Subscription&lt;/li&gt;
&lt;li&gt;Azure Resource Group&lt;/li&gt;
&lt;li&gt;Location (make sure this matches the previous settings for the Azure OpenAI resource)&lt;/li&gt;
&lt;li&gt;Service Name: unique name for the search service&lt;/li&gt;
&lt;li&gt;Pricing Tier: &lt;strong&gt;Basic&lt;/strong&gt; (since the free service won&amp;rsquo;t be recognized by Chat Playground)&lt;/li&gt;
&lt;li&gt;Scale / Replica: 1/1&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-d179c90c.png"
 
 
 loading="lazy"
 
 alt="Create Cognitive search resource"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="6"&gt;
&lt;li&gt;Once the Search Service got created, navigate back to the Chat Playground, and repeat the steps to &lt;strong&gt;add your own data&lt;/strong&gt;. This time, the Cognitive Search will be recognized as service in the &lt;strong&gt;Add Data Source&lt;/strong&gt; step.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-6a33f151.png"
 
 
 loading="lazy"
 
 alt="Chat Playground add your own data"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="7"&gt;
&lt;li&gt;Click &lt;strong&gt;Next&lt;/strong&gt;, and upload a few sample files in the &lt;strong&gt;Upload Files&lt;/strong&gt; step.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-8b05f521.png"
 
 
 loading="lazy"
 
 alt="Chat Playground upload files data"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="8"&gt;
&lt;li&gt;Continue the &lt;strong&gt;add data source&lt;/strong&gt; wizard steps, and completing with clicking &lt;strong&gt;Save and Close&lt;/strong&gt;.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-d64210cb.png"
 
 
 loading="lazy"
 
 alt="Chat Playground upload files data"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="9"&gt;
&lt;li&gt;From the Chat Playground window, notice how &lt;strong&gt;your data is getting added&lt;/strong&gt;. This process should only take a few minutes.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;This completes this part of the steps, where you specified the data source to be used by the Chat Playground. In the next and last step, you will use the Chat Session to validate the functionality of the chat bot, and testing the accuracy of the responses.&lt;/p&gt;
&lt;h2 id="using-chat-session-to-validate-data-content-and-response"&gt;Using Chat Session to validate data content and response
&lt;/h2&gt;&lt;ol&gt;
&lt;li&gt;From the &lt;strong&gt;Chat Playground&lt;/strong&gt; blade, navigate to &lt;strong&gt;Chat session&lt;/strong&gt;. Enter a basic question in the &lt;strong&gt;your message&lt;/strong&gt; field.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Note: in my scenario, I was using demoguides, so wanted to check back if the chat bot could find a demo scenario containing Cosmos DB as resource.&lt;/p&gt;
&lt;p&gt;Based on the question, the chat bot responded with an &lt;strong&gt;accurate answer&lt;/strong&gt;, providing a brief description of the actual demo steps from the guide, as well as a link to the actual source markdown-file in Blob Storage.&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-37f398b9.png"
 
 
 loading="lazy"
 
 alt="Chat Playground chat session"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="2"&gt;
&lt;li&gt;As I only have 1 single guide with CosmosDB, let&amp;rsquo;s test how it handles the question, if there are more results possible. I asked a somewhat broader question, using &lt;strong&gt;retail application&lt;/strong&gt; as keyword (Note: our demo scenarios involve a retail application as example, which exists in a Virtual Machine architecture, a Platform as a Service, Container Instance, Kubernetes Service and Azure Container Apps architecture). So based on that, the expectation is to get &lt;strong&gt;multiple results&lt;/strong&gt; back.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Woohoo!! The Chat bot found the &lt;strong&gt;different sources&lt;/strong&gt; and provided a &lt;strong&gt;nice summary overview&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-37f398b9.png"
 
 
 loading="lazy"
 
 alt="Chat Playground chat session"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;This was more than convincing to me how powerful Azure AI is. While the answers might not have been 100% accurate - close to 95% I guess :), know we just deployed the model without any fine-tuning, nor wait a long time to actually build up an accurate index of the blob source content.&lt;/p&gt;
&lt;ol start="3"&gt;
&lt;li&gt;Next, &lt;strong&gt;optional though&lt;/strong&gt;, you can publish the Chat Bot to an Azure App Service. This would allow a developer to integrate the bot in a broader web app scenario using iframe or similar HTML/CSS code.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;From the &lt;strong&gt;Chat Playground&lt;/strong&gt; blade, navigate to &lt;strong&gt;Deploy to&lt;/strong&gt; in the upper right corner. and select &lt;strong&gt;Azure App Service&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-29de58ea.png"
 
 
 loading="lazy"
 
 alt="Deploy Chat Bot to Azure App Service"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="4"&gt;
&lt;li&gt;Complete the necessary parameters for the &lt;strong&gt;Azure App Service&lt;/strong&gt; to be created:&lt;/li&gt;
&lt;/ol&gt;
&lt;ul&gt;
&lt;li&gt;App Service Name&lt;/li&gt;
&lt;li&gt;Azure Subscription&lt;/li&gt;
&lt;li&gt;Azure Resource Group&lt;/li&gt;
&lt;li&gt;Location&lt;/li&gt;
&lt;li&gt;Pricing Plan (S0 or S1 would be OK)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-afa0d73e.png"
 
 
 loading="lazy"
 
 alt="Deploy Chat Bot to Azure App Service"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="5"&gt;
&lt;li&gt;
&lt;p&gt;After waiting about 5 minutes (although the wizard set it could take up to 10min&amp;hellip;), it asked me for my &lt;strong&gt;Azure AD credentials&lt;/strong&gt; to authenticate. (**Note: out of the publishing wizard, a new App Registration and Service Principal gets created, granting only the admin user access. In a real-life scenario, you would need to update the Authentication settings on the App Registration to allow for a broader Identity scope)&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;After successful authentication, the Chat Bot is ready to be used:&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-7219ed8d.png"
 
 
 loading="lazy"
 
 alt="Deployed Chat Bot as Azure App Service"
 
 
 
 &gt;&lt;/p&gt;
&lt;ol start="7"&gt;
&lt;li&gt;As a final test, I wanted to see how the Chat Bot responded if it didn&amp;rsquo;t find the correct answer. (This was partly to rule out the ChatGPT experience I had before, where the AI engine invents its own answers, but still explaining it in such a way it feels as the correct answer)&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="../images/screenshot-2023-07-17-669866e4.png"
 
 
 loading="lazy"
 
 alt="Deployed Chat Bot as Azure App Service"
 
 
 
 &gt;&lt;/p&gt;
&lt;p&gt;As you can see, Azure AI handles it a bit more &amp;lsquo;honest&amp;rsquo;, and admitting it couldn&amp;rsquo;t help providing an accurate answer.&lt;/p&gt;
&lt;h1 id="summary"&gt;Summary
&lt;/h1&gt;&lt;p&gt;Azure AI is an amazing cloud service, with unseen capabilities. With this article, I wanted to inform you as a reader on how easy it can be to set up Azure AI Cognitive Search, and using it with a Chat Bot functionality, based on your own data from Azure Blob Storage.&lt;/p&gt;
&lt;p&gt;I&amp;rsquo;m confident Azure AI will become a big part of our day-to-day skillset. So expect more similar blog posts in the near future how I&amp;rsquo;m continuing my journey of learning about AI and get ready for the future.&lt;/p&gt;
&lt;p&gt;&lt;a class="link" href="https://www.buymeacoffee.com/pdtit" target="_blank" rel="noopener"
 &gt;&lt;img src="../images/screenshot-2023-07-17-17f576e7.png"
 
 
 loading="lazy"
 
 alt="BuyMeACoffee"
 
 
 
 &gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Cheers!!&lt;/p&gt;
&lt;p&gt;/Peter&lt;/p&gt;</description></item></channel></rss>