Copilot Enterprise - Feedback

[arfon]

• Who: Customers using Copilot Enterprise.
• What: We want to hear from you! Provide feedback on the Copilot Enterprise in this Discussion

---

---

As of 27th February 2024, GitHub Copilot Enterprise is now generally available 🚀. Read more about Copilot Enterprise on our recent blog post.

This thread is just for product feedback.

We are responding to general Questions & Answers related to Copilot Enterprise in another discussion: Copilot Enterprise Q & A

[mattbrandman]

Would love to be able to bulk index repositories as it seems to hang while indexing is happening. It is also unclear if a chat works without indexing the repo first as well.

[arfon]

Thanks for the feedback @mattbrandman. Could you say more about what is 'hanging' when indexing?

It is also unclear if a chat works without indexing the repo first as well.

Chat will still work, it's just the scenarios that connect natural language questions like 'where does authentication happen' to underlying code won't work.

[arfon]

/ cc @colinwm

[RealEstateSaga]

How are you? If I delete this, will my site crash, or is this just for search engines?

/next-sitemap.config.js

Sorry, I am just in a little bit of bind, Google My business does not like something on it, and they emailed me about fixing it.

Thanks!

[syossan27]

When I type Japanese in Copilot chat, the message is sent when I press the enter key after conversion.
I am having trouble typing sentences in Japanese if this is not corrected.
Please check the attached video.

feedback.mov

[arfon]

Thanks @syossan27 – we've noticed this issue too and are working on a fix.

[arfon]

This should be fixed now, would love to hear back if you're still facing issues?

[ChrisAveniq]

I have noticed two issues:

• When using chat in our indexed repository, it does not seem to know about directories that are clearly there. It says it does not exist.
  It says: I'm sorry, but I couldn't find any content in the "ingress" folder in the "xxx/reponame" repository. It's possible that the folder doesn't exist, is empty, or is not publicly accessible. Please verify the folder name and its accessibility.

But the repo has a folder named "ingress" in the root. The repo is internal, however.

• and this is also weird:
  

[arfon]

Got it. Thanks. And just confirming that the repository is showing up as indexed when you open up a chat session?

[ChrisAveniq]

@arfon Yes: Indexed for improved understanding and accuracy.

[OneCyrus]

@arfon it seems to be really hard to get a response at all which is related to the index.

[arfon]

I think you've identified a good gap in Copilot's capabilities here. Currently Copilot knows how to do the following four things:

• show-symbol-definition: To retrieve the lines of code that define a code symbol from a specified repository's checked in git files. This function is used when we have an exact symbol name to query with.
• pathsearch: To retrieve a file from a repository's git checked in files that matches the filename or file path provided. This function is used when the exact filename or file path to a file checked in to the repository is provided.
• bing-search: To search the web using the Bing search engine. This function is used when the user's query pertains to recent events, information that is frequently updated, new developments, trends, technologies, or is extremely specific or detailed.
  codesearch: To search code within a specified repository's git checked in files. This function is used when a user asks questions related to the repository's code, such as where or how certain functionality has been implemented.

(you can ask Copilot about this too)

What you're asking Copilot to do here is related to the various code search scenarios, but isn't quite right (as you say, you want a count/list from the index).

It's certainly possible to teach Copilot how to do this kind of thing and we'll put it on our backlog for sure. Can you say more about what you're wanting from Copilot in this kind of scenario? Is it a better understanding of what the 'shape' of a repository is, or were you looking for particular content?

[kokuyouwind]

It's certainly possible to teach Copilot how to do this kind of thing and we'll put it on our backlog for sure. Can you say more about what you're wanting from Copilot in this kind of scenario? Is it a better understanding of what the 'shape' of a repository is, or were you looking for particular content?

In my case, I encountered both.

The former use is a mono-repo asking for a description under a specific directory or asking for a use under a specific directory.
For example, the question "What is under the server directory?" will receive the answer "I'm sorry, but I couldn't find any information about a 'server' directory in the 'xxx' repository."
If I changed the question to "What framework is used under the server directory?", I could get answers such as Rails or Next.js. However, what I wanted with the original question was something a bit more abstract, such as "This is a client SPA for an API in the server directory."

The latter is a case where you want to find code under a specific directory or get the diff of multiple files that are close in usage.
For example, the questions "Is there a Google client under the server/app/gateways directory?" or "What are the differences in specifications for each client under the server/app/gateways directory?" will receive the answer "I'm sorry, but I couldn't find specific files under the server/app/ gateways directory in the xxx repository."

[ChrisAveniq]

I think what is missing is, similar to other copilots and chats, a stop button to stop a response. Sometimes it goes on for quite a long time and I already know I need to change my prompt.

[arfon]

Great suggestion, I agree this is a missing feature. We've added this to the backlog ✅

[PaulDMendoza]

For the browser chat tool:

My code base is 99% of C# but when I asked it to generate a unit text example it generated it in python. I think I need to provide some base level instructions to all the chats with some basic information. For example, I have created a custom Chat in ChatGPT that has the following instruction.

"My code base uses C# and we use Postgresql for our database server. We use Entity Framework for our simple data access. We use dependency injection. To get a connection to the database use ITeamDBConnection and to get an EntityFramework context use ITeamDBContextFactory and then call GetAsync()..."

This would really help the chat tool to improve the initial responses.

[arfon]

Hi @PaulDMendoza – this is great feedback, thank you.

My code base is 99% of C# but when I asked it to generate a unit text example it generated it in python. I think I need to provide some base level instructions to all the chats with some basic information. For example, I have created a custom Chat in ChatGPT that has the following instruction.

Yeah, this is a great idea, and we already do something similar with Copilot, but crucially we don't currently tell Copilot what programming languages are used in a repository. Doing this seems like a 'no-brainer' and I'll put this on the backlog!

[tristanbes]

There's few question I would like to have answers (right now the AI says it can't help):

Team / developpers related

• who are the top contributors of the past XX days ?
• what are the team working on right now (expecting results of a list of recent active PR with a short summary)
• what PR should require my attention ?

Code / Data / Analysis

• what are the most recurring fixes ?
• find the most updated file in the project
• how many unit tests there is in the project ?
• how many cypress scenarios we have ?
• can you give the % of failed github actions for the past 10 days ?

Lack of exhaustivity

• Find all events we're sending to our CDP -> while having 100+ events, the AI only gave 5 events (and said it wasn't exhaustive)

Screenshot

[timrogers]

Thanks for sharing these examples @tristanbes! They are super useful.

To be completely transparent, these are not the kinds of questions that Copilot Chat is able to answer right now - but we are working to fill these gaps and make Copilot better. Your examples are really valuable to help us prioritise.

We'll also look at how we can help you and other developers to understand what Copilot is good at and what it isn't good at - we don't do that very well right now.

[ghost]

I have transitioned to the enterprise version so that the Copilot chat will understand the context of the entire project. Thus, I won't need to explain that this file does X and that file does Y, or that I need to paste the code for it to understand which file does what. I had assumed that with the enterprise version, it would scan the repositories, and when I talk about a specific file, it would understand what I am referring to. However, it does not. I have configured the policies and created a knowledge base, but it did not change anything.

Bottom line:
I want the copilot to scan the entire repository be aware of all the files and the code inside and answer the questions in the context of the entire project.
here is example
I had issues with CSS that share the same name inside the project I want the copilot be aware of it and let me know that I need to change the names or when it recommend code or css to write it in the context of a project and not just as an example

[timrogers]

Hi Eran! I'm sorry to hear that you're having a frustrating experience. A couple of questions:

1. Are you using Copilot Chat on GitHub.com, or in your IDE? If so, which IDE are you using?
2. Can you share some examples of the kinds of questions you're asking? If you're not comfortable sharing here, feel free to email me at the @github.com email on my profile.

[emartin-monterey]

We are testing out Git Hub Enterprise & Copilot Enterprise for some of the same reasons and have similar questions. Local Visual Studio Pro Copilot Enterprise subscription Chat & Completion extensions do not seem to be aware if the indexed repos. Is there some other aspect to configure, etc. where this works locally?

For example, a Copilot Web chat prompt of "Use the Collector repo to find examples of Jobs and then create a Job that reads all the files in a given directory" provides a reasonable response of "..., class ReadDirectoryFilesJob : AbstractJob ..." with appropriate use of the AbstractJob class including the class method entry point which is the kind of functionality we are looking for.

On the other hand, locally the Visual Studio Pro Copilot Enterprise subscription chat for the same prompt returns "I'm sorry for the confusion, but as an AI, I don't have the ability to access or search through repositories, including the Collector repo". In addition, Copilot Enterprise subscription Completions VS Pro extension can not comprehend this directive without a number of related repo files open locally.

Note that we have not added any information to the repo's knowledge base. I will also note that the local Copilot extension does not have the ability to use the Bing search tool.

[arfon]

Thanks for the detailed feedback @emartin-monterey. It's possible that the (GitHub) Copilot Extension in Visual Studio Pro needs some configuration/setup here. I'm checking with the team to find out what might be the issue.

[emartin-monterey]

I posed this same question to GitHub Enterprise support and their response was:

"I was just informed by engineering that this feature is not available yet outside of GitHub.com. They are working to have this feature available for Visual Studio soon. I recommend you keep an eye on the GitHub Blog and our social media feeds for the latest announcements about new features and fixes.
You can also check out our Public roadmap to see what we're currently working on."

Looking at the public roadmap linked and even the backlog, Copilot is not mentioned and nothing related seems to be slotted for this year so far, which tells me this capability is not really on the radar. Paying $39 a user for the Copilot "chat with your code" feature, on top of the regular GitHub Enterprise subscription fee, is certainly less of a value-add than what we envisioned when upgrading to the Enterprise subscription.

[ghost]

Hey Tim,

I'm absolutely enamored with GitHub Copilot Chat. It writes nearly all of my code and consistently helps me debug and brainstorm new ideas. It's truly the most powerful tool I've ever used; I love it.

The only issue is that you always have to paste the code into it so it can grasp the context of your question. Even if I open the file and it appears in the editor, it seems to "read" it as if the chat is open on the left and the code is open on the right, and it still doesn't fully understand the context of the question.

I was hoping that the enterprise edition would grant the tool access to the entire repository, eliminating the need to copy the code for the initial question on a specific file. Unfortunately, it appears that for now, it's limited. I can inquire about a specific file if I provide the path, but this isn't conducive to seamless back-and-forth conversations.

I want to be able to chat with it and have it precisely understand what I'm talking about in the context of the repository.
I hope it helps.
Eran

[arfon]

I was hoping that the enterprise edition would grant the tool access to the entire repository, eliminating the need to copy the code for the initial question on a specific file. Unfortunately, it appears that for now, it's limited. I can inquire about a specific file if I provide the path, but this isn't conducive to seamless back-and-forth conversations.

@erangross – are you specifically looking for Copilot to have this additional 'repository-wide context' in the IDE chat sessions?

[stvnwrgs]

@arfon This would be huge! Biggest missing feature IMO!

[ghost]

From: Arfon Smith ***@***.***> Sent: Friday, March 1, 2024 2:03 PM To: community/community‏ ***@***.***> Cc: Eran Gross‏ ***@***.***>; Mention‏ ***@***.***> Subject: Re: [community/community] Copilot Enterprise - Feedback (Discussion #110314) I was hoping that the enterprise edition would grant the tool access to the entire repository, eliminating the need to copy the code for the initial question on a specific file. Unfortunately, it appears that for now, it's limited. I can inquire about a specific file if I provide the path, but this isn't conducive to seamless back-and-forth conversations. @erangross<https://github.com/erangross> – are you specifically looking for Copilot to have this additional 'repository-wide context' in the IDE chat sessions? Exactly. 'repository-wide context' in the IDE chat sessions I could not phrase it better myself. Eran — Reply to this email directly, view it on GitHub<#110314 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AMRPK2CQVSL5EJXBFXYAISLYWBU5ZAVCNFSM6AAAAABD4MYPTKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DMNBSGM2DS>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[arfon]

@erangross – are you familiar with the @workspace command in VS Code: https://code.visualstudio.com/updates/v1_84#_workspace

[neo-cheung]

The knowledge base can support indexing issues, wikis, discussions, and PRs. It would be nice, since my organization uses issues and discussions for design documents and specifications.

[arfon]

Great suggestions, thanks @neo-cheung .

[YusukeKokubo]

I would be very happy for my organization if this is supported 🙏

[kokuyouwind]

I would like to have the option to specify the output language when generating the Pull Request Summary.
In Copilot Chat, inquiries in Japanese are answered in Japanese, but the Pull Request Summary is always generated in English, so a separate translation is required.

[arfon]

Great suggestion, thanks @kokuyouwind. Out of interest, would this be a preference for you (i.e., is it only you that speaks Japanese, or your team, or your whole organization)?

[kokuyouwind]

@arfon
The whole organization. Since I am Japanese working for a Japanese company and almost all of my colleagues are Japanese, I always write the Description of Pull Requests in my organization in Japanese.

[arfon]

Got it. Thanks @kokuyouwind - that makes a lot of sense!

[seanemmer]

Nice work! Very excited about this.

It appears that the VSCode version of Copilot Chat does not have access to the index that is generated for the github.com Copilot chat.

I have attempted to use the @workspace directive but don't get the same results (it doesn't generate references to the correct files).

[seanemmer]

Thanks for the quick reply and additional info!

I've done some further testing and both github.com and VSCode IDE @workspace seem to be behaving similarly.

The issue is they both have trouble referencing my schema.prisma file. They seem to prefer to referencing .ts files.

e.g. the prompt @workspace what fields are in the account model in schema.prisma

only returns references to .ts files, not schema.prisma. This is for both versions of Copilot.

[timrogers]

@seanemmer That's interesting! My guess is that this is something to do with the file being a .prisma file. On Copilot Chat on GitHub.com, can you confirm what question you're asking? Do you get any little indicators like this in the response, or nothing?

[seanemmer]

Even more interesting. Here's a sample prompt:

But the reference is not to our schema.prisma, it's to the schema.prisma in a public repo:

https://github.com/Airbotics/uptane-server/blob/3ffe8ffb58e4ea19ed0bb724a07ea798dc0348ff/src/prisma/schema.prisma#L164-L278

[timrogers]

Even stranger! Just to confirm, did you get that answer without navigating to that public repo?

Did you take that screenshot immediately after getting the answer, or was there any kind of page refresh in between?

It is possible that Copilot could run a search including files outside of your repository - and that search could include public repositories - but I'm surprised not to see an indicator showing the search that was run, like in my screenshot above.

[seanemmer]

No page refresh, I'd never been to that repo.

It's no longer replicating today. Now both github.com and vscode correctly references my schema.prisma file! Not sure if you changed anything on your end.

Github.com correctly refers to all of the fields on the Account model. Vscode still fails in answering the question despite having a reference to schema.prisma:

The Account model in your schema.prisma file is not directly provided in the excerpts you've shared. However, there are references to an Account model in the relations of other models. For example, in the[redacted] model, there's a relation to Account:

This suggests that there is an Account model in your schema, but without the direct excerpt of the Account model, I can't provide the exact fields in it.

[PaulDMendoza]

Are there release notes somewhere that we can see what has been updated with Copilot Enterprise Chat or somewhere we can track the improvements?

[timrogers]

Great question! We'll update the GitHub Changelog (which shows on your GitHub homepage) when we ship new improvements to Copilot 😊

[yanskun]

Depending on the GitHub color theme, the code is buried in the code block background color, so you can't see anything, which is very disappointing.

Can I change the color of codeblock?

[arfon]

Thanks for reporting this @yanskun - I've filed a bug with the team to look into this!

[arfon]

Hi @yanskun – we've been trying to replicated this issue but have so far failed. Can you share with me a little more detail:

• What browser are you using?
• Do you have an extensions enabled? If so, do you still see this issue with them disabled?

[yanskun]

@arfon
Thank you
When I was turning on/off various extensions, I realized that they were not compatible with Sourcegraph..!

[arfon]

Ah, so just to confirm, disabling the Sourcegraph extension fixed the issue for you?

[yanskun]

@arfon
Sorry,
Yes, I disabled it and it was resolved.

[ouweiya]

Is it currently not supported to use the knowledge base in VSCode? I saw in the YouTube video demonstration that the @docs directive is supported?

https://youtu.be/AAT4zCfzsHI?t=2287

[arfon]

Hi @ouweiya – I'm afraid this isn't shipped just yet, but it is being actively worked!

[phillmv]

GitHub Chat window in dotcom keeps stealing focus to scroll while delivering an answer. I.e. I type something in, and it starts answering and scrolling the text down, but I want to scroll up to keep reading the text that it has already generated.

Also, I can't get it to do an ls on the repository. It can do a straight search but I just wanted it to tell me which file managed dependencies 🤷‍♀️.

[timrogers]

GitHub Chat window in dotcom keeps stealing focus to scroll while delivering an answer. I.e. I type something in, and it starts answering and scrolling the text down, but I want to scroll up to keep reading the text that it has already generated.

Thanks for raising this! ❤️ Frankly, it's something that I've found a bit frustrating too. We should have a think about how we can make this UX smoother.

Also, I can't get it to do an ls on the repository. It can do a straight search but I just wanted it to tell me which file managed dependencies 🤷‍♀️.

At the moment, Copilot can't list a directory, but I do think this would be a sensible skill for us to add. Can you give an example of the kind of question you want to be able to ask Copilot that would use this capability?

[ouweiya]

I added two source code repositories, facebook/react and reactjs/react.dev, to a knowledge base, but there seems to be only one when editing, as if some kind of bug has occurred?

chrome_cLrCZm03e8.mp4

[timrogers]

@ouweiya Definitely looks like a bug to me! Thanks for flagging it. Would you be able to share the URL from the address bar when viewing that knowledge base? It's safe to share here, or you can email me using the @github.com email on my profile.

[ouweiya]

https://github.com/organizations/ouweiya-pro/settings/copilot/chat_settings/40c23b46-60f8-4b8f-bd1f-3318d0a6f7e0/edit

[timrogers]

@ouweiya Thanks! I've flagged this internally, and will let you know once we've figured out what's going wrong.

[timrogers]

@ouweiya This should be fixed now. Can you take another look?

[ouweiya]

Yes, this bug is fixed.

[joostvanaarle]

Hi, In browser chat I was hoping to get help on a refactor we are working on in our code. But it was not able to see the changes we work on in the other branch. "Unfortunately, I currently don't have the capability to access specific branches of a repository other than the default branch." Same goes for commits. Also when chatting it has no context of the history of the chat in that sessie. Which does not help for more complex questions.

[timrogers]

@joostvanaarle Thanks for reaching out!

I was hoping to get help on a refactor we are working on in our code. But it was not able to see the changes we work on in the other branch. "Unfortunately, I currently don't have the capability to access specific branches of a repository other than the default branch." Same goes for commits.

You're right that Copilot Chat can't see other branches or historic commits at the moment.

We're working to add more skills to allow Copilot to do things like this.

I'd love to understand what you wanted Copilot to help you with so we can build the right skills.

Can you tell me more about the refactor you were trying to do, and how you wanted Copilot to help you?

Also when chatting it has no context of the history of the chat in that sessie. Which does not help for more complex questions.

I'm sorry to hear this - Copilot should remember your context, but sometimes it does go wrong. Do you have any examples of prompts that lead to this problem?

[joostvanaarle]

Hi Tim,
thanks for the fast response. Great to hear you are working on improving. We are working on a refactor that is effecting a lot of functions in our code. So we made a separate branch, disabled some functionalities and functions and then step by step refactor the code. I was hoping copilot would be able to understand the difference between branch X and branch Y. Maybe by tracking the changes in the commits, and then together with an explanation on what the goal and the strategy of the refactoring is help out with the remaining functions that need to be updated. As copilot index the repository I was hoping it was able to understand the code on that level.

As for the remembering of the context. I was basically referring to an earlier question and answer asked in that session. As I noticed it did not knew it happend I asked: "Can you look back at previous questions and answers within a chat session to follow the context of the session?" it then replied "As an AI assistant, I handle each question individually and do not have access to the history of previous questions in the session. The term "context following" was misleading and I apologize if that caused any confusion. I may respond to the information provided in each individual question, but I do not store information from one question to another. I hope this is clearer and I appreciate your patience."
I then checked with gtp-3.5 and it gave me the exact same answer. Also checked in gtp-4 which responded with "Yes, I can follow the context of our chat session by looking back at previous questions and answers. This allows me to respond to your questions or comments in a more relevant and coherent manner. If you reference something we've talked about before, I can use that information to improve my answers.". As im used working with gtp-4 model and I was expecting copilot to be more similar and context aware. Which unfortunately was not the case when using. I read somewhere copilot switches between the models as they are used in the backend, but its very unclear when.

[stvnwrgs]

Hey GitHub Team!

First and foremost, I love the recent updates! The new features have significantly streamlined our workflows, marking a substantial improvement in our daily operations.

However, I believe there are several areas where further enhancements would be great. Below are some suggestions:

IDE Plugin Support for JetBrains IDEs
We utilize JetBrains IDEs extensively and have observed that several GitHub Copilot features, available on the GitHub platform, are currently missing in these IDEs:

• Querying Indexed Repositories: The ability to ask questions directly related to indexed repositories within the IDE is unavailable.
• Integration of Knowledge Bases with Chats: There's no functionality to attach Knowledge Bases to chats in the IDE.
• Enhanced File Creation: When generating a new file, it would be immensely helpful if the IDE could not only generate the code but also automatically create the file, naming it based on similar existing files in the directory or according to a specified class name.

Repository Indexing
An organizational overview of indexed repositories would be a valuable addition. Ideally, this could be represented as a table within the organization settings, featuring actions for easy management. Here’s a conceptual structure:

Repository	Status	Action
owner1/repo1	Last Indexed (13:44 01.03.2024)	Remove, Reindex
owner2/repo1	Indexing	Remove

Including a functionality to easily add repositories to the indexing queue would greatly enhance usability.

Knowledge Bases
Introducing knowledge bases as automatically compiled wikis accessible via a navigation point within the organization settings could streamline information sharing and collaboration.

Guidance for Design Patterns
The diversity in design patterns across different companies is vast. We suggest creating a README.md file in each src folder to outline the specifications for items like models in an MVC framework, including recommended properties, functions, and annotations.

• A comprehensive guide on drafting documents to teach GitHub Copilot the nuances of our engineering practices would be invaluable. This could ensure Copilot acts as an integral member of our engineering team, tailored to our specific development methodologies.

Code Style Guides
While numerous tools for code styling already exist (e.g., Prettier, PHP CS Fixer, gofmt), integrating these styles directly with generated code could streamline the development process further.

• Implementing a secondary model that understands and applies code formatting guidelines to the generated code would make the integration of generated code more seamless and efficient.

Merge queue link in navigation
Last but not least, I know this is not a copilot topic, but maybe you can dm someone. We use merge queues and accessing them is super inconvenient. An optional navigation element at the top bar inside a repo, maybe with an indicator how many PR's are currently queued, would be great!

[arfon]

Apologies for the slow reply here @stvnwrgs – this is all great feedback!

IDE Plugin Support for JetBrains IDEs
We utilize JetBrains IDEs extensively and have observed that several GitHub Copilot features, available on the GitHub platform, are currently missing in these IDEs:

/ cc @Jenverse

Guidance for Design Patterns
The diversity in design patterns across different companies is vast. We suggest creating a README.md file in each src folder to outline the specifications for items like models in an MVC framework, including recommended properties, functions, and annotations.

Love this idea, and very directionally-aligned with our thinking too. Is this behavior you'd want to see in something like an AI/Copilot assisted code review or during the code editing/creation process?

[felickz]

Install the Copilot App

Consider adding a vanity url to immersive copilot so that when you Install the App in browser you can go directly to the chat view and have a copilot "installed" on your machine! Today, even if you install the app when on https://github.com/copilot/ - it will subsequently open the app to https://github.com/. This requires you to click the open copilot icon, ... open take conversation to immersive view, then choose general purpose chat

[IDrumsey]

It would be awesome to be able to chat about our notifications like "Do I have any new "insert repo name" notifications since this morning"

[dgehriger]

I have added several hundred markdown files of internal documentation to a knowledge base repo, and configured various sets from it, using path expressions (e.g. /doc/DEV/**/*). The collections show up on the chat, but Copilot doesn't integrate the content in its answers.

Is there some delay between adding KB content and it being made available?

[arfon]

/ cc @timrogers here for visibility and suggestions for what might be going on.

[timrogers]

@dgehriger Thanks for flagging this, and sorry for the slow response - I totally missed it. It can take a while to index your documentation - especially for large repositories - but usually not more than half an hour. If you try again now, does it work?

[timrogers]

(We have plans to make that delay and limitation clearer!)

[hubwriter]

Sometimes I ask Copilot something in Chat on dotcom and I get back a particularly useful answer that I know I'll find useful to refer back to. At the moment I'm copying the answer into OneNote where I can search and find it. However, it would be better, I think, if I could just save it to something like a "My favorite answers" list in Chat on dotcom, giving it a name that's meaningful to me, so I could get back to it quickly.

Is this something we might add?

[some-natalie]

[arfon]

I agree! I'm not sure what is happening here, but I replicated the error. For background – we have a number of content filters being applied to keep Copilot on(developer) topics, and ensure the responses are safe. Something in the generated output here is being incorrectly detected as a 'violation' of our content policies but I'm not exactly sure what/why.

I'll file this as a bug and see if we can figure it out.

[some-natalie]

Seems fixed now ❤️

[arfon]

I'm pleased to see you got a good answer here @some-natalie. I've checked with the team and we're pretty sure this is a case of Copilot over-filtering some outputs. We're actively working to improve these.

[polajenko]

when I include some basic tests for copilot to look at this is the answer I get "I'm sorry, but your request is a bit unclear. Are you asking about a specific type of tests in a GitHub repository? This could be unit tests, integration tests, system tests, acceptance tests, etc. Or are you asking about a specific repository? If so, could you please provide the repository name? The more details you provide, the better I can assist you."

[hasan-dot]

Same here, when I include 1 line, it is somehow not included in the context of my question:

Screen.Recording.2024-05-16.at.09.01.12.mov

[LindseyB]

Hey, can I get some more context so I can help investigate this from our end:

• How big are the files you're looking at an experiencing issues with?
• Due to the non-deterministic nature sometimes things will work when they previously didn't, is this consistently failing for you?
• If this code you're having issues with is in a public repository can you link that so we can investigate more closely?

Thanks.

[hasan-dot]

@LindseyB Sorry I didn't notice your response earlier 🙇 , its either I didn't get a notification or I just didn't notice it somehow
I should've link the PR in my initial report, sorry for that too 😓

How big are the files you're looking at an experiencing issues with?

Small, like 50 lines of code in total for certain files (example)

is this consistently failing for you?

it is constantly happening for me for this PR

can you link that so we can investigate more closely?

I can reproduce the same behavior in this PR

Thanks for your assistance and please let me know in case you have any followup questions 🙇

[fredsh2k]

i got the same file change views repeatedly, where only one of them relates to a single point
https://github.com/copilot/c/44d5cbd3-463a-41db-9195-d050e0fae9ba

[eshwer]

Hey @fredsh2k, we can look into this. What was the question you asked that triggered this response?

[connor4312]

Copilot's seems very noisy, and often incorrect. For example, this suggestion is just wrong every statement it makes (it won't fail, we're checking not-nullish rather than truthy, and TS won't in fact complain)

[eshwer]

Can you share the prompt that led to this response?

[connor4312]

This was the default PR review message on microsoft/vscode#214186

[bryanhelms]

I just had Copilot review some markdown, and I used the 1. numbering approach where everything was 1. and I let Markdown auto sequence the numbers. Copilot saw this, said this was wrong, then told me to do exactly what I was doing (which it just said was wrong). Thought I should file that as feedback.

[LindseyB]

🤦🏻‍♀️

[taku333]

Will the Copilot pull request summaries be coming in multiple languages?

The only supported language for Copilot pull request summaries is English.

https://docs.github.com/en/copilot/github-copilot-enterprise/copilot-pull-request-summaries/about-copilot-pull-request-summaries