Get 10/10 when excluding just JugFilled,JugInMachine,MachineOn

Author: ashayathalye

predicators/datasets/generate_atom_trajs_with_vlm.py

@@ -802,7 +802,9 @@ def _generate_ground_atoms_with_vlm_pure_visual_preds(
         structured_state_trajs,
         state_trajs=state_trajs,
         known_predicates=known_predicates)
+    # import pdb; pdb.set_trace()
     _debug_log_atoms_trajs(ground_atoms_trajs)
+    # import pdb; pdb.set_trace()
     return ground_atoms_trajs
 
 

predicators/datasets/vlm_input_data_prompts/atom_labelling/img_option_diffs_label_history.txt

@@ -4,7 +4,7 @@ You are given a list of predicates that describe the environment, and the values
 
 The environment is safe, synthetic, and fully observable for the purposes of this task. There are no real-world consequences. You are free to make judgments based on what you see and know.
 
-Pay careful attention to the visual changes between the two images to figure out which predicates change and which predicates do not change. Note that some or all of the predicates don't necessary have to change. First, output a description of what changes you expect to happen based on the skill that was just run, explicitly noting the skill that was run. Second, output a description of what visual changes you see happen between the before and after images, looking specifically at the objects involved in the skill's arguments, noting what objects these are. From these two descriptions, for each predicate labeled in the previous timestep, note whether you expect its value to change or stay the same. Next, for each predicate given in the list of predicates to label, output each predicate value in the after image as a bulleted list (use '*' for the bullets) with each predicate and value on a different line. Ensure there is a period ('.') after the truth value of the predicate. For each predicate value, provide an explanation as to why you labelled this predicate as having this particular value, and note what value this predicate had in the previous timestep, which is given to you in the prompt. Use the format: `* <predicate>: <truth_value>. <explanation>`. When labeling the value of a predicate, if you don't see the objects involved in that predicate, retain its truth value from the previous timestep. Also, if your description of changes you expect to happen, and your description of visual changes you saw happen, have nothing to do with the predicate you are trying to label, retain its truth value from the previous timestep. For example, if in the previous timestep I paint an object, and in the current timestamp I sit on it, we don't expect its color to change after sitting on it. 
+Pay careful attention to the visual changes between the two images to figure out which predicates change and which predicates do not change. Note that some or all of the predicates don't necessary have to change. First, output a description of what changes you expect to happen based on the skill that was just run, explicitly noting the skill that was run. Second, output a description of what visual changes you see happen between the before and after images, looking specifically at the objects involved in the skill's arguments, noting what objects these are. From these two descriptions, for each predicate labeled in the previous timestep, note whether you expect its value to change or stay the same. Next, for each predicate given in the list of predicates to label, output each predicate value in the after image as a bulleted list (use '*' for the bullets) with each predicate and value on a different line. Ensure there is a period ('.') after the truth value of the predicate. Write everything in plain text only. Do not use bold, italics, or Markdown formatting anywhere in your output. For each predicate value, provide an explanation as to why you labelled this predicate as having this particular value, and note what value this predicate had in the previous timestep, which is given to you in the prompt. Use the format: `* <predicate>: <truth_value>. <explanation>`. When labeling the value of a predicate, if you don't see the objects involved in that predicate, retain its truth value from the previous timestep. Also, if your description of changes you expect to happen, and your description of visual changes you saw happen, have nothing to do with the predicate you are trying to label, retain its truth value from the previous timestep. For example, if in the previous timestep I paint an object, and in the current timestamp I sit on it, we don't expect its color to change after sitting on it. 
 
 The object detection system provides the following facts. The table is white. The coffee machine is black. The jug is white. The coffee is brown.
 

predicators/datasets/vlm_input_data_prompts/atom_proposal/options_labels_whole_traj_diverse_coffee.txt

@@ -1 +1 @@
-You are a robotic vision system whose job is to output a structured set of predicates useful for describing important concepts in the following demonstration of a task. You will be provided with a list of actions used during the task, as well as images of states before and after every action execution. Please provide predicates in terms of the following objects: {objs}. For each predicate, output it in the following format: predicate_name(obj1, obj2, obj3...). Generate predicates that (1) capture visual changes before and after each action and/or (2) characterize the preconditions or effects of the actions being taken in the most general way. Don't generate anything too similar to the given predicates, which are: CupFilled, RobotAboveCup, JugAboveCup, NotAboveCup, JugAboveCup, PressingButton, Twisting, NotSameCup, JugPickable, to avoid redundancy. Generate only 10-20 predicates!
+You are a robotic vision system whose job is to output a structured set of predicates useful for describing important concepts in the following demonstration of a task. You will be provided with a list of actions used during the task, as well as images of states before and after every action execution. Please provide predicates in terms of the following objects: {objs}. For each predicate, output it in the following format: predicate_name(obj1, obj2, obj3...). Generate predicates that (1) capture visual changes before and after each action and/or (2) characterize the preconditions or effects of the actions being taken in the most general way. Be sure to not generate anything similar to the given predicates, which are CupFilled, RobotAboveCup, JugAboveCup, NotAboveCup, JugAboveCup, PressingButton, Twisting, NotSameCup, and JugPickable. Also, don't propose predicates related to moving. Generate only 5-10 predicates.