Server working with agent library #97

rbren · 2024-03-22T15:40:35Z

This PR modifies the websocket server so that it uses the main Agent API to start an agent.

It removes the docker container mgmt, which is now controlled by the DockerInteractive sandbox.

Sample output. The first line is the only message sent to the server, everything else is a response

❯ websocat ws://127.0.0.1:3000/ws
{"action": "start", {"args": {"task": "write a bash script that prints hello"}}
{"message":"Starting new agent..."}
{"action":"run","message":"Running command: ls","args":{"command":"ls"}}
{"action":"output","message":"Got output.","args":{"output":"LICENSE\nOpenDevinLogo.jpg\nREADME.md\nagenthub\nenv_name\nevaluation\nfrontend\nhello.sh\nopendevin\nrequirements.txt\nserver\nworkspace\n"}}
{"action":"read","message":"Reading file: hello.sh","args":{"path":"hello.sh"}}
{"action":"output","message":"Got output.","args":{"output":"#!/bin/bash\necho \"hello\""}}
{"action":"run","message":"Running command: bash hello.sh","args":{"command":"bash hello.sh"}}
{"action":"output","message":"Got output.","args":{"output":"hello\n"}}
{"action":"think","message":"I've successfully executed the bash script hello.sh which printed 'hello'. My primary task is complete. It's time to finalize my work.","args":{"thought":"I've successfully executed the bash script hello.sh which printed 'hello'. My primary task is complete. It's time to finalize my work."}}
{"action":"finish","message":"Finished!","args":{}}

kvchitrapu · 2024-03-22T20:02:46Z

The task hangs in git add hello.sh step. Add error checks around command execution and return exit on errors.

STEP 0
run {'command': 'ls'}
---
output {'output': 'LICENSE\nOpenDevinLogo.jpg\nREADME.md\nagenthub\nevaluation\nfrontend\nopendevin\nrequirements.txt\n'}
==============
STEP 1
think {'thought': "I should create a new bash script file. Let's call it hello.sh."}
---
==============
STEP 2
write {'path': 'hello.sh', 'contents': '#!/bin/bash\necho "hello"'}
---
output {'output': ''}
==============
STEP 3
run {'command': 'bash hello.sh'}
---
output {'output': 'hello\n'}
==============
STEP 4
think {'thought': "I've successfully created and executed the bash script that prints 'hello'. My next step should be to ensure the script is properly saved and can be executed by others. I should consider making the script executable by others and perhaps add a comment to explain what it does."}
---
==============
STEP 5
write {'path': 'hello.sh', 'contents': '#!/bin/bash\n# This script prints \'hello\'\necho "hello"'}
---
output {'output': ''}
==============
STEP 6
run {'command': 'chmod +x hello.sh'}
---
output {'output': ''}
==============
STEP 7
think {'thought': 'Now that the script is executable and has a helpful comment, I should ensure that my work is preserved for future reference or use. A good way to do this would be to commit the changes using git.'}
---
==============
STEP 8
run {'command': 'git add hello.sh'}

rbren · 2024-03-22T21:10:04Z

@kvchitrapu lmk if you're still able to repro this--I think I've fixed it.

kvchitrapu · 2024-03-23T00:50:59Z

@kvchitrapu lmk if you're still able to repro this--I think I've fixed it.

Happy path worked fine.

However, any syntax error in action json drops connection in session.py:56. How about sending back an error to the client indicating bad json? Here is a bad json message with extra '}' at the end to illustrate the problem on the client side.

Client side

$ websocat ws://127.0.0.1:3000/ws
{"message":"Control loop started"}
{"action":"start","args":{"task":"write a bash script that prints hello"}}}
<hangs>

Server side

$ uvicorn opendevin.server.listen:app --reload --port 3000
...
File "/var/OpenDevin-rbren/OpenDevin/opendevin/server/listen.py", line 15, in websocket_endpoint
    await session.start_listening()
  File "/var/OpenDevin-rbren/OpenDevin/opendevin/server/session.py", line 39, in start_listening
    data = await self.websocket.receive_json()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/OpenDevin-rbren/OpenDevin/.venv/lib/python3.11/site-packages/starlette/websockets.py", line 145, in receive_json
    return json.loads(text)
           ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_2/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_2/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 75 (char 74)
INFO:     connection closed

rbren · 2024-03-23T13:38:07Z

Good catch! Fixed the JSON parsing issue

I just got it to hang again, so I'm digging in a little further.

rbren · 2024-03-23T14:25:39Z

Figured out some of the hanging. It seems like await sometimes swallows errors--I had raise ValueError triggering, with no logs, which totally broke the control loop.

I've fixed it by adding try/except blocks into the controller, especially where it calls back to the agent or callbacks. But my guess is there's a better way.

kvchitrapu · 2024-03-23T21:51:20Z

Figured out some of the hanging. It seems like await sometimes swallows errors--I had raise ValueError triggering, with no logs, which totally broke the control loop.

I've fixed it by adding try/except blocks into the controller, especially where it calls back to the agent or callbacks. But my guess is there's a better way.

Nice work! Bad json input gave an error:

{"action": "start", "args": {"task": "write a bash script that prints hello"}}}
{"error":true,"message":"Invalid JSON"}

kvchitrapu · 2024-03-23T22:08:07Z

LGTM

kvchitrapu · 2024-03-23T22:13:24Z

For our next steps, it's important we address the following to enhance our project's stability and cleanliness:

Cleanup methods to ensure a smooth termination of WebSocket connections, sessions, agents, and controllers when errors occur. This should also extend to handling the cleanup of Docker containers started by agents.
A session manager to track sessions, which will aid in preventing resource leaks.
Add linting tools, such as ruff integrated into GitHub Actions, to catch and eliminate unused imports.

@rbren, if that sounds good to you, I'll proceed to create an enhancement issue and follow up with a pull request.

rbren · 2024-03-23T22:22:36Z

👍 totally agree!

re: cleanup, I think I'm doing some basic cleanup already, but it could probably be improved.

re: sessions, I think we can assume 1 session per server for MVP, but would be great to make sessions more robust

yimothysu

LGTM

xingyaoww

LGTM!

* server working with agent library * update readme * add messages to events * factor out steps * fix websocket messages * allow user to run arbitrary actions * allow user to run commands before a task is started * fix main.py * check JSON * handle errors in controller better * fix memory issue * better error handling and task cancellation * fix monologue len * fix imports * remove server from lint check * fix lint issues * fix lint errors

AbhisekOmkar approved these changes Mar 22, 2024

View reviewed changes

rbren mentioned this pull request Mar 23, 2024

WebSocket API #44

Closed

rbren mentioned this pull request Mar 23, 2024

Frontend/Backend: Connect chat interface to agent #56

Closed

rbren added 17 commits March 23, 2024 19:52

server working with agent library

f34564e

update readme

fb4c78f

add messages to events

9455c8a

factor out steps

f1a087a

fix websocket messages

32be994

allow user to run arbitrary actions

7885e4e

allow user to run commands before a task is started

58370f0

fix main.py

8a2a226

check JSON

b8d24dc

handle errors in controller better

2578467

fix memory issue

c48fb41

better error handling and task cancellation

cab671b

fix monologue len

cac35a1

fix imports

3dd1046

remove server from lint check

b01fdf6

fix lint issues

b3b803b

fix lint errors

da81dcc

rbren requested a review from xingyaoww March 24, 2024 02:28

rbren requested a review from yimothysu March 24, 2024 02:28

yimothysu approved these changes Mar 24, 2024

View reviewed changes

xingyaoww approved these changes Mar 24, 2024

View reviewed changes

xingyaoww merged commit 4aa24eb into All-Hands-AI:main Mar 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Server working with agent library #97

Server working with agent library #97

Server working with agent library #97

Server working with agent library #97

Conversation

Client side

Server side

Choose a reason for hiding this comment

Choose a reason for hiding this comment