Manual

This section list commands used to control robot

Refer to DEVICE_CLIENT_CONFIGS in carerobotapp.configs.device_clients module for more details.

[CLIK TO SOURCE]

How to use

Create a new project and customize configs
- Customize your required devive clients similar with DEVICES.
- Customize your robot similar with ROBOT.
- Customize your tasks similar with TASKS.
Create node_prompt, node_taskmanager, node_skill_servers, node_observe(Optional)

from pyconnect.ros.node_prompt import run_prompt_node, get_prompt_cfg
from carerobotapp.configs.tasks import PROMPT_CONFIGS

cfg = get_prompt_cfg(**PROMPT_CONFIGS)
run_prompt_node(cfg=cfg)

from pyconnect.ros.node_taskmanager import run_task_manager_node
from pyconnect.utils import parse_keys_values, path2module
from pyconnect.ros.node_prompt import get_prompt_cfg
from pyconnect.ros.get_configs import get_prompt_cfg, get_observation_cfg
from carerobotapp.configs.tasks import PROMPT_CONFIGS, SKILL_CONFIGS, DEVICE_CLIENT_CONFIGS, OBSERVATION_NODE_CONFIGS, LLM_SERVES

"""Main entry point to initialize and run the ManagerNode."""
prompt_cfg = get_prompt_cfg(**PROMPT_CONFIGS)
observe_cfg = get_observation_cfg(**OBSERVATION_NODE_CONFIGS)
llm_cfg = LLM_SERVES['llama']
llm_cfg['name'] = 'llama'
run_task_manager_node(prompt_cfg=prompt_cfg, observe_cfg=observe_cfg, llm_cfg=llm_cfg, skill_cfgs=SKILL_CONFIGS, device_cfgs=DEVICE_CLIENT_CONFIGS)

from pyconnect.ros.node_skill_servers import run_skill_servers
from carerobotapp.configs.tasks import DEVICE_CLIENT_CONFIGS, SKILL_CONFIGS, RobotParam

run_skill_servers(device_cfgs=DEVICE_CLIENT_CONFIGS, skill_cfgs=SKILL_CONFIGS, robot_params=RobotParam)

from pyconnect.ros.node_observe import run_observe_node
from pyconnect.ros.get_configs import get_observation_cfg
from carerobotapp.configs.tasks import OBSERVATION_NODE_CONFIGS

run_observe_node(cfg=get_observation_cfg(**OBSERVATION_NODE_CONFIGS))

Make new skill

touch new_skill.py

# add new_skill to SKILL_LISTS  in your own tasks config file

Execute new skill

# Run in 3 different terminals
python3 -m carerobotapp.node_prompt
python3 -m carerobotapp.node_taskmanager
python3 -m carerobotapp.node_skill_servers

# input in node_prompt running window
new_skill::[INPUTS]

Init Arm

Fuction	Prompt Use	Dev Use
Clear error	clear_error:	node.agents[‘clean_error’].send({})
Enable motion	enable_motion:	node.agents[‘enable_motion’].send({‘inputs’:0})
Set mode	set_mode::[INT]	node.agents[‘set_mode’].send({‘inputs’:[INT]})
Set state	set_state::[INT]	node.agents[‘set_state’].send({‘inputs’:[INT]})
Init Arm	init_arm:	from carerobotapp.skills.goto_ready import init_arm init_arm(node=node)

Arm Control

Fuction	Usage	Inputs
Fuction	Usage	Inputs
Arm movel	# PROMPT USE movel::[INPUTS] example: movel::x=200, y=700 # DEV USE from carerobotapp.skills.arm import movel movel(node=node, x=200, y=700)	x, y, z [Default: Current x, y, z] dx, dy, dz [Default: 0] wait [Default: True] Predefined locations: afterpnp: location afetr picking or placing
Arm movej	# PROMPT USE movej::[INPUTS] example: movej::fold # DEV USE from carerobotapp.skills.arm import movej movej(node=node, inputs='fold')	r0, r1, r2, r3, r4, r5, r6 [Default: Current joints] dr0, dr1, dr2, dr3, dr4, dr5, dr6 [Default: 0] wait [Default: True] Predefined locations: home: home pose ready: ready pose give: give me pose fold: arm fold unfold: arm unfod
Move tool	# PROMPT USE movet::[INPUTS] example: movet::dz=100 # DEV USE from carerobotapp.skills.arm import movel movet(node=node, dz=100)	dx, dy, dz [Default: 0] wait [Default: True]
Movel + movet	# PROMPT USE movelf::[INPUTS] example: movelf::x=200, rx=-135 # DEV USE from carerobotapp.skills.arm import movelf movelf(node=node, x=200, rx=-135)	x, y, z [Default: Current x, y, z] dx, dy, dz [Default: 0] rx, ry, rz: Roll, Pitch, Yaw [Default: Current angles] drx, dry, drz: [Default: 0] wait [Default: True]

Gripper / Litf/ Head / Mobile Control

Fuction	Usage	Inputs
Grip	# PROMPT USE grip::[INT] example: grip::200 # DEV USE from carerobotapp.skills.grip import grip grip(node=node, inputs=200)	inputs [0, 1000] Predefined locations: open: inputs=1000 close: inputs=0
Lift	# PROMPT USE lift::[INT] example: lift::200 # DEV USE from carerobotapp.skills.grip import grip lift(node=node, inputs=200)	inputs [115, 650] Predefined locations: lowest: inputs=115 highest: inputs=650
Head	# PROMPT USE moveh::[INPUTS] example: moveh::left, straight # DEV USE from carerobotapp.skills.head import moveh moveh(node=node, inputs='left, straight')	inputs: ry, rz [deg] [Default: current angle] Predefined locations: front: rz=0 lefet: rz=-90 right: rz=90 straight: ry=-25 up: ry=0 down: ry=-50
Mobile move	# PROMPT USE move::[INPUTS] example: move::desk # DEV USE from carerobotapp.skills.mobile import move move(node=node, inputs='desk')	offset_x, offset_y [mm] [Default: in ENV meta] offset_yaw [deg] [Default: in ENV meta]

Recognition Skills

Fuction

Usage

Inputs

Find

# PROMPT USE
find::[INPUTS]
example: find::banana@desk

# DEV USE
from carerobotapp.skills.recognition import find
find(node=node, inputs='banana@desk')

inputs [caption@location]
   e.g: yellow cup@desk
       . if no location:
          find at current loc
       . else:
          move to loc first
detector:
   [Default: groudingdino]
   [groudingdino, fastsam,
   Groundedsam, groundedsam_grasp]
camera:
   [Default: head]
   [head, rs_raw]
once:
   [Default: True]
   True:
     Find at loc only
   False:
     Search all loc in ENV

Find_arm

# PROMPT USE
find_arm::[INPUTS]
example: find_arm::banana

# DEV USE
from carerobotapp.skills.recognition import find_arm
find_arm(node=node, inputs='banana')

inputs [caption]

detector:
   [Default: groundedsam_grasp]
   [groudingdino, fastsam,
   groundedsam, groundedsam_grasp]
camera:
   [Default: rs_raw]
   [head, rs_raw]

Approach Skills

Fuction

Usage

Inputs

Approach_close
Approach

# PROMPT USE
approach_close::[INPUTS]

example 1:
   approach_close::banana@desk

example 2:
   find::banana@desk
   approach_close::banana

example 3:
   approach_close::[x,y,z,wrist_angle]

example 4:
   approach_close::desk@living_room

# DEV USE
from carerobotapp.skills.approach import approach_close
approach_close(node=node, inputs='banana@desk')

inputs
   [caption, location,
   caption@location, pose_3d]
       . if no location:
          find at current loc
       . else:
          move to loc first

       . if caption:
           find [caption] first
         else:
           move to location
           approach to placepose

       . if pose_3d:
          approach to pose_3d


init_pose_fixed
   [Defalt: False]

All find input params

Pick Skills

Fuction

Usage

Inputs

Pick

# PROMPT USE
pick::[INPUTS]

example 1:
   pick::banana@desk

example 2:
   find::banana@desk
   pick::banana

example 3:
   pick::[x,y,z,wrist_angle]

# DEV USE
from carerobotapp.skills.pick import pick
pick(node=node, inputs='banana@desk')

inputs
   [caption, location,
   caption@location, pose_3d]
       . if no location:
          find at current loc
       . else:
          approach to loc first

init_pose_fixed
   [Defalt: True]

All find input params

Place Skills

Fuction

Usage

Inputs

Place

# PROMPT USE
place::[INPUTS]

example 1:
   place::banana@shelf>>desk@living_room

example 2:
   pick::banana@shelf@kitchen
   place::wooden_dish@desk@living_room

example 3:
   place::[x,y,z,wrist_angle]

example 4:
   place::desk@living_room

# DEV USE
from carerobotapp.skills.place import place
place(node=node, inputs='banana@desk')

inputs
   [caption, location,
   caption@location, pose_3d]
       . if no location:
          find at current loc
       . else:
          move to loc first

       . if caption:
           find [caption] first
         else:
           move to location
           approach to placepose

       . if pose_3d:
          approach to pose_3d

All approach input params

Wipe Skills

Fuction

Usage

Inputs

Wipe

# PROMPT USE
wipe::[INPUTS]

example 1:
   wipe::

example 2:
   wipe::spill@dining table

example 3:
   wipe::towel@table@living room>>spill@dining table

# DEV USE
from carerobotapp.skills.place import wipe
wipe(node=node, inputs='spill@dining table')

inputs
   [caption, location,
   caption@location, pose_3d]
       . if no location:
          find at current loc
       . else:
          move to loc first

       . if caption:
           find [caption] first
         else:
           move to location
           approach to placepose


All approach input params

Open/Close Drawer Skills

Fuction

Usage

Inputs

Open/Close Drawer

# PROMPT USE
open_drawer::[INPUTS]
close_drawer::[INPUTS]

example 1:
   open_drawer::
   close_drawer::

example 2:
   open_drawer::drawer table
   close_drawer::drawer table

# DEV USE
from carerobotapp.skills.pick import open_drawer, close_drawer
open_drawer(node=node, inputs='drawer table')
close_drawer(node=node, inputs='drawer table')

inputs
   [location]


All approach input params