PL j! tUdZddlmZddlmZmZdddiddgd d d d dgd dd ddddddddddddiddddddgddd dddgdd d!d"d#dd$dd%dd&dd'dddiddd(dd)dddiddd*dd+dgd,d-d d.dd/dd0dd1dd2dd3dd4dd5dd6d7d8dd9d:dd9d;dd<dgd=d>Zd?ed@<dDdBZdCS)EuNSchema for the generic `computer_use` tool. Model-agnostic. Any tool-calling model can drive this. Vision-capable models should prefer `capture(mode='som')` then `click(element=N)` — much more reliable than pixel coordinates. Pixel coordinates remain supported for models that were trained on them (e.g. Claude's computer-use RL). ) annotations)AnyDict computer_useuDrive the macOS desktop in the background — screenshots, mouse, keyboard, scroll, drag — without stealing the user's cursor, keyboard focus, or Space. Preferred workflow: call with action='capture' (mode='som' gives numbered element overlays), then click by `element` index for reliability. Pixel coordinates are supported for models trained on them. Works on any window — hidden, minimized, on another Space, or behind another app. macOS only; requires cua-driver to be installed.objectactionstring) captureclick double_click right_click middle_clickdragscrolltypekey set_valuewait list_apps focus_appu Which action to perform. `capture` is free (no side effects). All other actions require approval unless auto-approved. Use `set_value` for select/popup elements and sliders — it selects the matching option directly without opening the native menu (no focus steal).)renum descriptionmode)somvisionaxuCapture mode. `som` (default) is a screenshot with numbered overlays on every interactable element plus the AX tree — best for vision models, lets you click by element index. `vision` is a plain screenshot. `ax` is the accessibility tree only (no image; useful for text-only models).appzOptional. Limit capture/action to a specific app (by name, e.g. 'Safari', or bundle ID, 'com.apple.Safari'). If omitted, operates on the frontmost app's window or the whole screen.)rrelementintegerzoThe 1-based SOM index returned by the last `capture(mode='som')` call. Strongly preferred over raw coordinates. coordinatearrayrzPixel coordinates [x, y] in logical screen space (as returned by capture width/height). Only use this if no element index is available.)ritemsminItemsmaxItemsrbutton)leftrightmiddlezMouse button. Defaults to left. modifiers)cmdshiftoptionaltctrlfn)rrz%Modifier keys held during the action.)rr#r from_elementzSource element index (drag). to_elementzTarget element index (drag).from_coordinatez3Source [x,y] (drag; use when no element available). to_coordinatez3Target [x,y] (drag; use when no element available). direction)updownr'r(zScroll direction.amountzScroll wheel ticks. Default 3.valuezFor action='set_value': the value to set on the element. For AXPopUpButton / select dropdowns, pass the option's display label (e.g. 'Blue'). For sliders and other AXValue-settable elements, pass the numeric or string value.textz+Text to type (respects the current layout).keyszUKey combo, e.g. 'cmd+s', 'ctrl+alt+t', 'return', 'escape', 'tab'. Use '+' to combine.secondsnumberzSeconds to wait. Max 30.booleanuOnly for action='focus_app'. If true, brings the window to front (DISRUPTS the user). Default false — input is routed to the app without raising, matching the background co-work model.zIf true, take a follow-up capture after the action and include it in the response. Saves a round-trip when you need to verify an action's effect.) raise_window capture_after)r propertiesrequired)namer parametersDict[str, Any]COMPUTER_USE_SCHEMAreturnctS)z2Return the generic OpenAI function-calling schema.)rF=/home/kuhnn/.hermes/hermes-agent/tools/computer_use/schema.pyget_computer_use_schemarLs rJN)rGrE) __doc__ __future__rtypingrrrF__annotations__rLrIrJrKrQsR#"""""  ;Y   H%Y 6  ///-  7Y N  BOY b !'cY r  ),5  sY H  333@IY R $KKK G SY d Y-KMMeY h 9+IKKiY l  ),1T   mY x  ),1T yY F  7772GY P !?QY Z  S[Y n  LoY v  ;wY D  9EY P"="BcY Y Y tJy]]j'j'jjjjZrJ