Hello, AI World and Streaming Content Between ASP.NET APIs and Angular

tl;dr - just show me the code (and the readme): https://github.com/dahlsailrunner/hello-ai-world

AI technologies and tooling are evolving more rapidly than most technology in recent past and are not slowing down. Getting started in this space can be intimidating. In this post I intend to make this as easy as possible, as well as show how to provide streamed content from an ASP.NET API and then consume it (as streamed content) in an Angular application.

The End Result

The final result of this code is an Angular app has a page that makes an API call to an ASP.NET API, which asks a hard-coded question to Azure OpenAI, provides a streaming response, and that response is rendered as it comes in nice HTML based on the markdown that it came as.

Shown here:

Setting UP an Azure OpenAI Instance

The code uses an Azure OpenAI instance which needs to exist. For small little experimentation purposes like I'm describing here, there is little to no cost, but you do need to have your own instance.

You'll need to have an Azure OpenAI instance and a model deployed as a "Chat Deployment". The simple instructions to do that are in the readme tied to this post.

When you're done, you should be able to identify three things:

• Azure OpenAI Endpoint: The URL for your Azure OpenAI instance, something like https://YOUR-NAME.openai.azure.com
• API Key: This is a key for authenticating to your Azure OpenAI instance
• Deployment Model: This will be a short string like YOUR-VALUE-gpt-4o-mini (as an example)

Using OpenAI in an ASP.NET API

An easy way to use OpenAI capabilities (including both Azure OpenAI and OpenAI) in an ASP.NET application is to reference the Aspire.Azure.AI.OpenAI.

Once you've got the package referenced, you need a single line in Program.cs:

1builder.AddAzureOpenAIClient("azureOpenAi");

That registers an OpenAIClient with the dependency injection provider that can be injected where you need it.

The string "azureOpenAi" is a connection string from your configuration. I recommend using user secrets, which in Visual Studio is as simple as right-clicking on the API project and choosing "Manage User Secrets".

The JSON for the connection string looks like this and should use values from the Azure OpenAI instance mentioned above:

1{
2  "ConnectionStrings": {
3    "azureOpenAi": "Endpoint=https://YOUR-VALUE.openai.azure.com/;Key=YOUR-KEY"
4  }
5}

Then in a constructor for an API controller (or wherever), you can simply inject an OpenAIClient and you should be able to use it!

Providing a Streamed Response in ASP.NET

Consider the following method as the simplest possible example of using the OpenAIClient:

 1public async Task<ActionResult<string>> GetHello(CancellationToken cxl)
 2{
 3    var chatClient = aiClient.GetChatClient(_chatDeploymentModel); // set to your deployment model from above
 4
 5    ChatCompletion completion = await chatClient.CompleteChatAsync(
 6    [
 7        new UserChatMessage("What are the main steps to set up an Azure OpenAI instance?")
 8    ], cancellationToken: cxl);
 9    return completion.Content[0].Text;
10}

The method will work just fine - but it waits until the complete response is obtained before returning anything at all, and returns the entire response at once.

A better experience is to return the response in chunks so that the user can start reading as soon as the earliest information is available.

A new method that does that is here:

 1public async IAsyncEnumerable<string> StreamChat(
 2        [EnumeratorCancellation] CancellationToken cxl)
 3{
 4    var chatClient = aiClient.GetChatClient(_chatDeploymentModel);
 5
 6    var response = chatClient.CompleteChatStreamingAsync(
 7    [new UserChatMessage("What are the main steps to set up an Azure OpenAI instance?")],
 8            cancellationToken: cxl);
 9
10    await foreach (var message in response)
11    {
12        foreach (var contentPart in message.ContentUpdate)
13        {
14            yield return contentPart.Text;
15        }
16    }
17}

Note first the return type of the method: IAsyncEnumerable<string>. Also note that cancellation is handled with different types here, too.

We're calling the CompleteChatStreamingAsync method instead of CompleteChatAsync in this example (and not awaiting the method call), and then there's an awaited foreach loop and the yield return of the text from each response part.

If you keep in mind the different return type, cancellation handling, and the yield return, then you should be able to provide streamed API responses to callers.

1[
2  "",
3  "Setting",
4  " up",
5  " an",
6  " Azure",
7  " Open",
8  "AI",
9  ... MORE FOLLOWS IN SINGLE ARRAY

Receiving Streamed Responses in Angular

Gettting a streamed response in Angular to display as it comes in can be done pretty easily, but requires the use of the JavaScript Fetch API which isn't enabled by default on the Angular HTTP client.

Enabling the Fetch API for HttpClient

In the app.config.ts of your Angular app, you can enable the Fetch API with the withFetch() call. All of your existing calls should still work, but now you have the option of getting "progress reports" from streaming APIs.

So my app.config.ts contains this code (note provideHttpClient):

 1export const appConfig: ApplicationConfig = {
 2  providers: [
 3    provideZoneChangeDetection({ eventCoalescing: true }), 
 4    provideRouter(routes), 
 5    provideAnimationsAsync(),
 6    provideHttpClient(
 7      withInterceptors([csrfInterceptor]),
 8      withFetch() // <-- this is the important line
 9    ),
10  ]
11};

The Angular Service

The service code in Angular is as follows for my example:

 1export class ChatService {
 2
 3  constructor(private http: HttpClient) { }
 4  private apiUrl = '/api/v1/openai/stream-chat';
 5
 6  streamChat() {
 7    return this.http.get(this.apiUrl, {
 8      observe: 'events',
 9      responseType: 'text',
10      reportProgress: true,
11    });
12  }
13}

I'm using the Duende Backend For Frontend (BFF) security pattern here to avoid making a remote API call with an exposed access token, but the concept above should be pretty clear.

An HTTP call will be made to the endpoint that provides a streaming response, and with the Fetch API I want to make sure we can observe the progress events from this call.

The Angular Component - with a Signal

The Angular component code is as follows (notes below the code):

 1export class ChatComponent implements OnInit {
 2  streamedMessages = signal('');
 3
 4  constructor(
 5    private chatService: ChatService,
 6    private sanitizer: DomSanitizer,
 7  ) {}
 8
 9  ngOnInit(): void {
10
11    this.chatService.streamChat().subscribe((event) => {      
12      if (event.type === HttpEventType.DownloadProgress) {
13        // getting streamed content
14        const partial = (event as HttpDownloadProgressEvent).partialText!;
15        const partialAsString = this.convertToString(partial!);
16        // The response in this example is a markdown string, 
17        // so we need to convert it to HTML
18        const updatedMessage = this.sanitizer.bypassSecurityTrustHtml(
19          marked(partialAsString) as string
20        );
21        this.streamedMessages.set(updatedMessage as string);
22      }
23      // else if (event.type === HttpEventType.Response) { 
24      //   Do stuff when the streaming is done
25      //   console.log('Streaming done');
26      //}
27    });
28  }
29
30  convertToString(responseContent: string): string {
31    // the response might or might not be a valid JSON array, 
32    // but it always starts with [, so we need to conditionallyadd the ]
33    if (responseContent.slice(-1) !== ']') {
34      responseContent += ']';
35    }
36    return JSON.parse(responseContent).join('');
37  }
38}

Two things are injected: the ChatService that we just described above, and a DomSanitizer - so that we can make the markdown-formatted content returned by the streaming API look nice when displayed.

The ngOnInit() method has the logic that subscribes to the events from the streamChat() method -- when it gets a DownloadProgress event it adds the partial content received to what is displayed.

The partial content is received as a JSON array, but it might not be a closed-array (with the final ]), so that is what the convertToString() method handles.

1[ "hi", ", there", ". This",

The final string is parsed into HTML by the marked library, and the signal (the streamedMessages variable) is updated.

Also note that if you wanted to perform some logic when the final content is received, there is a commented-out conditional block that shows where you could do that.

The Component HTML

The HTML is really simple - and just references the streamedMessages signal:

1<h2 class="heading-lg">Chat responses below</h2>
2<h2 class="heading-md">What are the main steps to set up an Azure OpenAI instance?</h2>
3
4<div [innerHtml]="streamedMessages()"></div>

Where to Go Next

This repo and post are meant as "stepping stones" into the world of AI and its capabilities.

A great place to go next is to start exploring the use of "tools" in chat.

Or do the same thing as above with SemanticKernel.

Happy coding!