Python: Multiple results per prompt (incl. streaming)

[awharrison-28]

Motivation and Context

To take advantage of APIs offered by most LLMs and to be in sync with .NET SK, this PR introduces the ability to generate multiple text completions or chat completions from a single prompt.

Description

• Return type hint for complete_async and complete_chat_async changed from str -> Union[str, List[str]]. The use of Union is the proper way to indicate multiple return types prior to Python 3.10+. 3.10 supports the use of the | symbol, but since the Python SK is supported on 3.8 and 3.9, I did not adopt the newer standard.
• complete_async, complete_stream_async, complete_chat_async, and complete_chat_stream_async now support settings field number_of_responses greater than 1. Previously only a value of 1 was supported.
• Note: hf_text_completion does not support streaming multiple responses due to a limitation of TextIteratorStreamer. This feature requires the ability to parse distinct responses from TextIteratorStreamer.
• Fixed a bug where complete_async was streaming single responses as 1D arrays, content now is simply a string.

Example Usage

Setup

    kernel = sk.Kernel()

    # Configure OpenAI service
    api_key, org_id = sk.openai_settings_from_dot_env()
    oai_text_service = OpenAITextCompletion("text-davinci-003", api_key, org_id)
    oai_chat_service = OpenAIChatCompletion("gpt-3.5-turbo", api_key, org_id)

    # Configure Hugging Face service
    hf_text_service = HuggingFaceTextCompletion("gpt2", task="text-generation")
  
    # Configure Prompt
    prompt = "what is the purpose of a rubber duck?"

    # Configure Request Settings
    text_request_settings_multi = CompleteRequestSettings(
        max_tokens=100,
        temperature=0.7,
        top_p=1,
        frequency_penalty=0.5,
        presence_penalty=0.8,
        number_of_responses=4
    )

    chat_request_settings_multi = ChatRequestSettings(
        max_tokens=100,
        temperature=0.7,
        top_p=1,
        frequency_penalty=0.5,
        presence_penalty=0.8,
        number_of_responses=4
    )

Text Completion (Standard)

    texts = await oai_text_service.complete_async(prompt, text_request_settings_multi)
    i = 0
    for text in texts:
        print("Option " + str(i) + ": " + text)
        i += 1

Streaming Text Completion

    multi_stream = oai_text_service.complete_stream_async(prompt, text_request_settings_multi)
    texts = [''] * text_request_settings_multi.number_of_responses
    async for text in multi_stream:
        i = 0
        os.system('cls' if os.name == 'nt' else 'clear') # clear the screen for a better experience
        print("PROMPT: " + prompt)
        for option in text:
            texts[i] = texts[i] + option
            print("{0}: {1}".format(i, texts[i]))
            i += 1

Chat Completion (Standard)

    texts = await oai_chat_service.complete_chat_async([("user",prompt)], chat_request_settings_multi)
    i = 0
    for text in texts:
        print("Option " + str(i) + ": " + text)
        i += 1

Streaming Chat Completion

    multi_stream = oai_chat_service.complete_chat_stream_async([("user",prompt)], chat_request_settings_multi)
    texts = [''] * chat_request_settings_multi.number_of_responses
    async for text in multi_stream:
        i = 0
        os.system('cls' if os.name == 'nt' else 'clear') # clear the screen for a better experience
        print("PROMPT: " + prompt)
        for option in text:
            texts[i] = texts[i] + option
            print("{0}: {1}".format(i, texts[i]))
            i += 1

HuggingFace Standard Completion

    texts = await hf_text_service.complete_async(prompt, request_settings_multi)
    i = 0
    for text in texts:
        print("-----------------------------------")
        print("Option " + str(i) + ": " + text)
        i += 1

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows SK Contribution Guidelines (https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
• The code follows the .NET coding conventions (https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions) verified with dotnet format
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄