What's changed: New GH-300 Chapter 3 (data handling & flow = context gathering/prompt building, proxy filtering/post-processing, per-plan data usage/sharing/retention; suggestion lifecycle & LLM limits = context change → generate → present → accept/reject loop, unconfirmed inline candidate, probabilistic generation/knowledge cutoff/context dependence/no correctness guarantee)
3.2Suggestion Lifecycle and LLM Limitations
Visualize the lifecycle of a code suggestion from generation to accept/reject, and understand the limitations of LLMs and Copilot (context dependence, knowledge cutoff, probabilistic generation, no correctness guarantee).
Understanding the lifecycle of how a suggestion arises and disappears makes Copilot’s behavior more predictable. Roughly: detect context change → build prompt → model generates → present candidate → user accepts/rejects → next suggestion with new context—a loop. What you accept/reject influences the next context.
3.2.1The code-suggestion lifecycle
Each time the context changes (typing, cursor moves), Copilot builds a fresh prompt, sends it to the model, and presents a candidate. When the user accepts (e.g., Tab), it becomes part of the subsequent context. Rejecting discards it, and the next input yields a different candidate. In inline suggestions, the "grayed-out candidate" is this presentation stage—not in the code until confirmed. Grasping this loop lets you infer "why this suggestion appeared or didn’t" from context.
3.2.2Limitations of LLMs and Copilot
Copilot is built on an LLM, with limitations from its nature: (1) probabilistic generation—output varies even for the same context and isn’t always optimal; (2) knowledge cutoff—it doesn’t know information after training; (3) context dependence—heavily shaped by the quality/quantity of provided context, degrading with irrelevant/insufficient input; (4) no correctness guarantee—plausible can still be wrong (hallucination, Chapter 1). These are intrinsic model traits, not "bugs you can toggle off," so compensate operationally (validation, better prompts).
Common: (1) Suggestion lifecycle = context change → prompt → generate → present → accept/reject → next context (accepting affects the next context). (2) An inline candidate is not applied until confirmed. (3) LLM limitations = probabilistic generation / knowledge cutoff / context dependence / no correctness guarantee. (4) These are intrinsic traits, not toggle-off bugs—compensate operationally (validation, better prompts).
Watch out: (1) Output varying for the same context is not a bug—it’s probabilistic generation. (2) Whether a suggestion appears depends heavily on context—don’t assume a "feature failure." (3) Limitations can’t be "disabled in settings"—compensate via validation and better prompts. (4) The grayed-out inline candidate isn’t in your code until accepted.
3.2.3Section summary
- Suggestion lifecycle: context change → prompt → generate → present → accept/reject → next context
- The grayed-out inline candidate isn’t applied until confirmed
- LLM limits: probabilistic generation, knowledge cutoff, context dependence, no correctness guarantee
- Limits are intrinsic—compensate operationally via validation and better prompts
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. Which is correct about the "grayed-out candidate" shown in inline suggestions?
Q2. Copilot’s suggestion can differ slightly each time for the same code context. This is due to what?
Q3. What is the root cause of Copilot sometimes not reflecting the latest library spec?
Q4. When suggestions are low quality, which improvement follows from understanding the lifecycle?
Q5. What is the correct way to deal with LLM/Copilot limitations?
Q6. Which is correct about how an accepted suggestion affects subsequent behavior?