Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Theia AI] Terminal agent records its requests #14246

Merged
merged 4 commits into from
Oct 3, 2024
+32 −1
Merged

[Theia AI] Terminal agent records its requests #14246

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d457fa2
Terminal agent records its requests
JonasHelming Oct 2, 2024
9e3bb84
Update packages/ai-terminal/src/browser/ai-terminal-agent.ts
JonasHelming Oct 3, 2024
c09b896
Update packages/ai-terminal/src/browser/ai-terminal-agent.ts
JonasHelming Oct 3, 2024
e3b6742
Fix linting error
JonasHelming Oct 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 35 additions & 1 deletion packages/ai-terminal/src/browser/ai-terminal-agent.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,14 @@

import {
Agent,
CommunicationHistoryEntry,
JonasHelming marked this conversation as resolved.
Show resolved Hide resolved
CommunicationRecordingService,
getJsonOfResponse,
isLanguageModelParsedResponse,
LanguageModelRegistry, LanguageModelRequirement,
PromptService
} from '@theia/ai-core/lib/common';
import { ILogger } from '@theia/core';
import { generateUuid, ILogger } from '@theia/core';
import { inject, injectable } from '@theia/core/shared/inversify';
import { z } from 'zod';
import zodToJsonSchema from 'zod-to-json-schema';
Expand All @@ -33,6 +35,8 @@ type Commands = z.infer<typeof Commands>;

@injectable()
export class AiTerminalAgent implements Agent {
@inject(CommunicationRecordingService)
protected recordingService: CommunicationRecordingService;

id = 'Terminal Assistant';
name = 'Terminal Assistant';
Expand Down Expand Up @@ -153,6 +157,20 @@ recent-terminal-contents:
return [];
}

// since we do not actually hold complete conversions, the request/response pair is considered a session
const sessionId = generateUuid();
const requestId = generateUuid();
const requestEntry: CommunicationHistoryEntry = {
agentId: this.id,
sessionId,
timestamp: Date.now(),
requestId,
request: systemPrompt,
messages: [userPrompt],
};

this.recordingService.recordRequest(requestEntry);
JonasHelming marked this conversation as resolved.
Show resolved Hide resolved

try {
const result = await lm.request({
messages: [
Expand Down Expand Up @@ -181,12 +199,28 @@ recent-terminal-contents:
// model returned structured output
const parsedResult = Commands.safeParse(result.parsed);
if (parsedResult.success) {
const responseText = JSON.stringify(parsedResult.data.commands);
this.recordingService.recordResponse({
agentId: this.id,
sessionId,
timestamp: Date.now(),
requestId,
response: responseText,
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const responseText = JSON.stringify(parsedResult.data.commands);
this.recordingService.recordResponse({
agentId: this.id,
sessionId,
timestamp: Date.now(),
requestId,
response: responseText,
});
this.recordingService.recordResponse({
agentId: this.id,
sessionId,
timestamp: Date.now(),
requestId,
response: JSON.stringify(parsedResult.data.commands),
});

This is also to avoid the linting error:

@theia/ai-terminal: /home/runner/work/theia/theia/packages/ai-terminal/src/browser/ai-terminal-agent.ts
@theia/ai-terminal: 202:27 error 'responseText' is already declared in the upper scope on line 216 column 19
@typescript-eslint/no-shadow
@theia/ai-terminal:
@theia/ai-terminal: ✖ 1 problem (1 error, 0 warnings)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had this before, but especially with GPT 3.5 it "hanged" sometimes. I guess the error in parsing is not thrown correctly then, could this be?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happens only in the case below as in this case, the result is always pasrsable, but I wanted to make it consistent

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the linting error with the new commit

return parsedResult.data.commands;
}
}

// fall back to agent-based parsing of result
const jsonResult = await getJsonOfResponse(result);
const responseText = JSON.stringify(jsonResult);
this.recordingService.recordResponse({
agentId: this.id,
sessionId,
timestamp: Date.now(),
requestId,
response: responseText
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const responseText = JSON.stringify(jsonResult);
this.recordingService.recordResponse({
agentId: this.id,
sessionId,
timestamp: Date.now(),
requestId,
response: responseText
});
this.recordingService.recordResponse({
agentId: this.id,
sessionId,
timestamp: Date.now(),
requestId,
response: JSON.stringify(jsonResult)
});

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had this before, but especially with GPT 3.5 it "hanged" sometimes. I guess the error in parsing is not thrown correctly then, could this be?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the linting error with the new commit

const parsedJsonResult = Commands.safeParse(jsonResult);
if (parsedJsonResult.success) {
return parsedJsonResult.data.commands;
Expand Down
Loading