Simulator implementation

Iniital state from this comment from Jeong

Troubleshooting

For numerical debugging, I was thinking of applying the same rosbag data and offline observer to a MuJoCo simulation, that I already made it as ROS2 node before, in which torque command can be subscribed and sensor data (e.g. end-effector pose/twist/acc, ...) can be published, so that I can get some hints for debugging.
But, I found that:
- My local ROS2 node cannot communicate with Nix's ROS1 node via ros1_bridge (it may require quite complicate set up.)
- Also, even local ROS1 node cannot communicate with Nix's ROS1 node (I don't know why. I thought if roscore is run, and I set ROS_MASTER_URI as the same, they can communicate topics that have the same topic name.) I could check rostopic list in each shells, but they could not communicate.
So, I tried to install MuJoCo in Nix env by editing flake.nix as below (referring to this, and I think it is installed without any problem. But, when run the simulator, it didn't work. It looks like there is some problem with rendering (OpenGL), but I have no idea.

  outputs = inputs@{ self, ... }:
    let
      system = "x86_64-linux";
      
      # ...   

      rospkgs = pkgs.rosPackages.noetic;  
      rosPackages = with rospkgs; [
        # ...
      ];

      mujocoDependencies = [
        pkgs_2411.mujoco
        pkgs_2411.glfw
        #pkgs.python3Packages.mujoco-py
      ];

    in
      {
        devShells.${system} = {        
          default = pkgs.mkShell {
            packages = [
              # ...
            ];

            # MuJoCo env variable setting 
            shellHook = ''
              export LD_LIBRARY_PATH=${pkgs.mujoco}/lib:$LD_LIBRARY_PATH
              export LD_LIBRARY_PATH=${pkgs.mesa}/lib:${pkgs.libglvnd}/lib:$LD_LIBRARY_PATH
              export CPATH=${pkgs.mesa}/include:$CPATH
            '';
          };
        };
      };

created branch 6-simulator-implementation to address this issue

Hi @jeong, got the mujoco env running with graphical interface.

The standard trick is to use nixGL when you need to create a graphical interface, this simulates GL functionalities in the nix world. From 7fc8cd00 you should be able to do

nix develop .#sim
cd /nix/store/vijj76i147v2h8nbcgdnxffjn0xwcb2n-mujoco-3.1.6
nixGLIntel ./simulate

and get a visualization env as seen below

Hello @hanikevi , I hope you have a great time during your vacation. Thank you so much for providing me a solution for running mujoco in Nix environment. It seems like I should have added additional work in flake.nix which I would never know by myself. You must spent quite lots of effort to fix it. I appreciate so much

Thanks to you, I wrote a mujoco node for ROS1 interface at feature/6-simulator-implementation branch, in which I can command joint torque command and perturbation from a controller node. I made it as a separate branch to keep your original work. When the code is stabilized, the branch could be merged later.

Also, I modified the original offline_observer node at feature/5-observer-implementation branch, in which impedance controller is integrated. We can use it both for simulation and real robot with an additional code modification. This branch also could be merged to the original 5-observer-implementation branch later.

Since both branches are originated in the same repo, to run each nodes simultaneously, I separated them as a independent repo for testing, for example, ~/nix-project/dob_impedance_controller and ~/nix-project/ros1_mujoco_interface, and here's how it runs:

The small red arrow represents perturbation.

When I tested it with both real robot data from rosbag, and simulation data, I found there's some bug in the estimation of DOB and SOSML, so I am working on debugging the observers. This may solve the numerical difference problem in estimation that I mentioned in previous. I will keep update with the results.

Thank you so much again, and I hope you have a great holidays =)

mentioned in issue #7 (closed)

changed the description

assigned to @jeo65286 and unassigned @hanikevi

marked the checklist item Re-structure branch 6- as sub-package so we can merge more easily with this repo as completed

Hello @hanikevi , I restructured 6-simulator... branch as sub-package so that it can be merged into the main branch easily @f37d277a. Some minor features could be done for this package in the future, but at least it can be done at the main branch independently. If it seems that it doesn't have any problem. I will merge it into the main branch.

Commit looks good! You tested after the restructure? I guess this is what you mean with 'it seems that it doesnt have any problem'. If so, go ahead and do the merge. If you have other features you'd like to think about in the future, Id suggest to write them down as an issue and, if they're not blocking, put the 'backlog' tag on them.

Yes I tested after restructuring, and it works. Okay then I will merge it into the main branch with 5-observer-... and comment some features that can be further developed. Thank you for your review =)

Hello @hanikevi , I created merge request for 6-simulator-... branch here, but there was merge conflicts. I checked if there's some conflicts in structure, but I couldn't find what was the problem cause I already had updated 6-simulator-... branch both in remote and local as the same. So, I just merged the branch into main in my local git and then pushed it to remote.

Hi @jeong, The issue for !2 (merged) was that it was behind main, i.e. the main branch contains commits that #6 (closed) did not. I merged main into #6 (closed), but then I noticed that the main already contains the package for the fr3_impedance_controller_sim. Is there any code in branch 6-simulator-... which is not in main?

Hello @hanikevi , oh I see. No the package for simulator in main branch and 6-simulator-... branch were the same, then I think I can just merge 6-simulator-... branch into the main now. It seems that there's no merge conflict in Merge requests. I will let you know if there's another issue for merging. Thank you for solution =)

mentioned in merge request !2 (merged)

mentioned in commit 941910ac

closed with merge request !2 (merged)

Simulator implementation

Designs

Child items ...

Activity

Troubleshooting