🚀 The Goal: Bringing Robots to Life (Virtually!)

Imagine controlling a robot with a joystick—or even better, letting it move on its own like a self-driving car! That’s what we’re doing here. Using PyBullet (a cool physics engine), we’ll simulate robots that can:

• 🤖 Move autonomously (think: robot vacuum cleaners that avoid walls)

• 🎮 Be controlled by a human (like a video game, but for robots!)

By the end, you’ll have a virtual robot playground where you can test different ways to make robots smart and responsive

📜 TL;DR (Too Long; Didn’t Read) – Explained Like You’re 10

🔹 What’s PyBullet? → A tool that lets robots move realistically in a computer world.
🔹 Autonomous Robot? → A robot that thinks for itself (like a Roomba).
🔹 Teleoperated Robot? → A robot you control (like a remote-control car).
🔹 Why Simulate? → Testing robots in real life is expensive. Simulation = cheap & safe!

🔧 Setting Up the Simulation

1. Installing PyBullet

First, we need PyBullet. It’s free and works with Python!

pip install pybullet

2. Loading a Robot

PyBullet comes with pre-built robots. Let’s load a Franka Emika Panda (a fancy robot arm). Also, you can use your robot using the URDF file. To learn more about URDF, check out my article on URDF here.

Create your project folder, e.g sample-robot and create a file main.py The content of the main.py is shown below

import pybullet as p
import pybullet_data

# Start the simulation
physicsClient = p.connect(p.GUI)  # GUI = Graphical window
p.setAdditionalSearchPath(pybullet_data.getDataPath())
p.loadURDF("plane.urdf")  # Ground
robot = p.loadURDF("franka_panda/panda.urdf", [0, 0, 0],  useFixedBase=True)
p.setGravity(0, 0, -10)  # Set gravity

while True:
    p.stepSimulation()  # Step the simulation
    p.setRealTimeSimulation(1)  # Real-time simulation
    p.setTimeStep(1.0 / 240.0)  # Time step

If you run the command python main.pyNow you should see a robot arm on the floor!,

---

N.B.: If it does not work or you are having an error, check if PyBullet is properly installed*

🤖 Autonomous Control: Making the Robot Think

We’ll make the robot move on its own using simple AI.

TL;DR: How Autonomous Control Works

→ The robot senses its surroundings (like a self-driving car).
→ It decides where to move (like avoiding obstacles).
→ It moves without human help.

Example: Making the Robot Wave

First, we will get all the joints of the robots, then we’ll move its joints in a loop to make it wave.

To get all the joints of the robot

# Existing code still remains intact
num_joints = p.getNumJoints(robot)
movable_joint_indexes = []
for joint_index in range(num_joints):
    joint_info = p.getJointInfo(robot, joint_index)
    joint_type = joint_info[2]
    if joint_type != p.JOINT_FIXED:
        movable_joint_indexes.append(joint_index)
print("Movable joints:", movable_joint_indexes)

The output should be similar to the image below:

Now your robot can configure the joints of the robot to move in your preferred direction.

🎮 Teleoperated Control: Human in the Loop

Now, let’s control the robot with a keyboard!

TL;DR: How Teleoperation Works

→ You press keys (like in a game).
→ The robot moves based on your commands.

Keyboard-Controlled Robot

Now, since we have our robot joint information, we will program the joint to move when the keys Forward and Backward are pressed, and change the selected joint if keys n or p are pressed.

import pybullet as p
import pybullet_data
import time
## Existing code still remains intact

target_positions = {}
for j in movable_joint_indexes:
    # Get the current joint position (using joint state)
    current_state = p.getJointState(robot, j)[0]
    target_positions[j] = current_state

# Control settings
increment = 0.05  # how much to change per key press
selected_idx = 0  # index in movable_joint_indexes list

print("Controlling joint index:", movable_joint_indexes[selected_idx])
print("Press RIGHT/LEFT arrow to increase/decrease joint angle")
print("Press 'n' (next) / 'p' (previous) to change the selected joint")

while True:

    keys = p.getKeyboardEvents()
    for k, v in keys.items():
        if v & p.KEY_WAS_TRIGGERED:
            # Select next/previous joint
            if k == ord("n"):
                selected_idx = (selected_idx + 1) % len(movable_joint_indexes)
                print("Selected joint:", movable_joint_indexes[selected_idx])
            elif k == ord("p"):
                selected_idx = (selected_idx - 1) % len(movable_joint_indexes)
                print("Selected joint:", movable_joint_indexes[selected_idx])
            # Adjust joint angle (using arrow keys)
            elif k == p.B3G_RIGHT_ARROW:
                selected_joint = movable_joint_indexes[selected_idx]
                target_positions[selected_joint] += increment
                print(
                    "Joint", selected_joint, "target:", target_positions[selected_joint]
                )
            elif k == p.B3G_LEFT_ARROW:
                selected_joint = movable_joint_indexes[selected_idx]
                target_positions[selected_joint] -= increment
                print(
                    "Joint", selected_joint, "target:", target_positions[selected_joint]
                )

    # Apply control for the selected joint
    selected_joint = movable_joint_indexes[selected_idx]
    p.setJointMotorControl2(
        robot,
        selected_joint,
        p.POSITION_CONTROL,
        targetPosition=target_positions[selected_joint],
    )

    p.stepSimulation()
    time.sleep(1.0 / 240.0)

The block of code above simply selects a joint if n is pressed, then move it forward or backward when the forward or backward keys are pressed.

Now you can move the robot with the Forward (→) and Backward (←) keys.

🌐 Combining Both: Smart + Human Control

What if the robot mostly moves independently, but you can take over when needed?

Obstacle Avoidance + Human Control

• The robot moves forward automatically.

• If it detects an obstacle, it stops.

• You can override its movement with keys.

# (Pseudocode)
while True:
    if obstacle_detected():
        robot.stop()  # Autonomous safety
    else:
        robot.move_forward()  # Autonomous mode

    if human_pressed_key():  # Teleoperation override
        robot.follow_human_command()

---

🎉 Conclusion: Why This Matters

✅ Safe Testing → No broken robots in real life!
✅ Fun Learning → Play with robots like a video game.
✅ Real-World Use → Self-driving cars, drones, and factory robots use similar tech!

Happy learning and see you in the next robotics byte-release! 🚀