Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.Net - Add support for agent tools: code-interpreter and retrieval tool #4546

Merged
merged 14 commits into from
Jan 10, 2024

Conversation

crickman
Copy link
Contributor

@crickman crickman commented Jan 10, 2024

Motivation and Context

Enable deeper alignment with Open AI features: code-interpreter and retrieval

Description

  • Add support to enable code-interpreter and retrieval
  • Add support to associated uploaded files with agent
  • Files must be uploaded outside of the Agent API and then referenced by identifier
  • Added support for message citations

Known issue where Open AI API text includes citation reference and no corresponding annocations:

{
  "id": "msg_jCIdC8gdO5yPylbP9JMGCNO0",
  "object": "thread.message",
  "created_at": 1704872920,
  "thread_id": "thread_BQLiJQFVFxfNH2blMROZYDQ2",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": {
        "value": "Sam Marks went on a trip from Seattle to Raleigh/Durham. According to the itinerary, Sam departed from Seattle-Tacoma International Airport (SEA) on Monday, September 11, 2023, at 10:05 AM on Alaska Airlines Flight AS 572 and arrived in Raleigh (RDU) at 06:15 PM【6†source】. Sam stayed for two nights【4†source】 and returned on Wednesday, September 13, 2023, departing from Raleigh (RDU) at 07:15 PM on Alaska Airlines Flight AS 491 and arrived back in Seattle (SEA) at 09:59 PM【7†source】.",
        "annotations": []
      }
    }
  ],
  "file_ids": [],
  "assistant_id": "asst_2zdSRjQJCyAKdXvhqw9dNrie",
  "run_id": "run_Rn5nJPncHY41nAr5VgOvPeyC",
  "metadata": {}
}

Contribution Checklist

Sorry, something went wrong.

@crickman crickman added PR: ready for review All feedback addressed, ready for reviews .NET Issue or Pull requests regarding .NET code feature branch PR targeting a feature branch experimental Associated with an experimental feature enhancement labels Jan 10, 2024
@crickman crickman self-assigned this Jan 10, 2024
@crickman crickman requested a review from a team as a code owner January 10, 2024 07:16
@crickman crickman linked an issue Jan 10, 2024 that may be closed by this pull request
crickman and others added 3 commits January 9, 2024 23:19

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
crickman and others added 6 commits January 10, 2024 12:37

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Co-authored-by: Gil LaHaye <gillahaye@microsoft.com>

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Co-authored-by: Gil LaHaye <gillahaye@microsoft.com>

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Co-authored-by: Gil LaHaye <gillahaye@microsoft.com>
@crickman crickman requested a review from glahaye January 10, 2024 20:48
@crickman crickman added this pull request to the merge queue Jan 10, 2024
Merged via the queue into main with commit 019dc1a Jan 10, 2024
@crickman crickman deleted the update_agent_tools branch January 10, 2024 22:16
Bryan-Roe pushed a commit to Bryan-Roe-ai/semantic-kernel that referenced this pull request Oct 6, 2024
…ol (microsoft#4546)

### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Enable deeper alignment with Open AI features: code-interpreter and
retrieval

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

- Add support to enable code-interpreter and retrieval
- Add support to associated uploaded files with agent
- Files must be uploaded outside of the Agent API and then referenced by
identifier
- Added support for message citations

Known issue where Open AI API text includes citation reference and no
corresponding annocations:

```
{
  "id": "msg_jCIdC8gdO5yPylbP9JMGCNO0",
  "object": "thread.message",
  "created_at": 1704872920,
  "thread_id": "thread_BQLiJQFVFxfNH2blMROZYDQ2",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": {
        "value": "Sam Marks went on a trip from Seattle to Raleigh/Durham. According to the itinerary, Sam departed from Seattle-Tacoma International Airport (SEA) on Monday, September 11, 2023, at 10:05 AM on Alaska Airlines Flight AS 572 and arrived in Raleigh (RDU) at 06:15 PM【6†source】. Sam stayed for two nights【4†source】 and returned on Wednesday, September 13, 2023, departing from Raleigh (RDU) at 07:15 PM on Alaska Airlines Flight AS 491 and arrived back in Seattle (SEA) at 09:59 PM【7†source】.",
        "annotations": []
      }
    }
  ],
  "file_ids": [],
  "assistant_id": "asst_2zdSRjQJCyAKdXvhqw9dNrie",
  "run_id": "run_Rn5nJPncHY41nAr5VgOvPeyC",
  "metadata": {}
}
```

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Gil LaHaye <gillahaye@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement experimental Associated with an experimental feature feature branch PR targeting a feature branch .NET Issue or Pull requests regarding .NET code PR: ready for review All feedback addressed, ready for reviews
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

.Net: OpenAI code_interpreter support in SK Agents
3 participants