While reading this morning I recalled how robots (industrial robots in particular) use “teaching pendants” to program the robots movements in addition to writing traditional control code.
The only pendant I’ve used is this one from the Heathkit HERO-1 robot.
The pendant is a physical device akin to a remote control that can be used to move the robot though various motions. The difference between this and a simple remote is that the motions can be recorded, creating a program that allows the robot to repeat the action without human control.
I haven’t worked with these machines directly but I imagine that the training process produces a program in the robots memory (perhaps in gcode?) that can be edited and adjusted to fine-tune the action. Maybe even inserting variables or other sources of dynamic control (input from sensors, etc.).
Some robots can be trained by moving the robot by hand, guiding the robot through the action by taking its “hand” and manually moving it. I imagine this uses the position encoders of the robots limbs to produce a similar program.
When I’ve thought about ways to build programming tools in virtual environments I’ve thought in terms of physical construct representing programming language logic. For example, cubes and pipes representing the flow of a flowchart. However this doesn’t feel very natural. On the other hand, a “training” process like this might achieve the goals of programmability in a way that feels more natural within the virtual environment.
It’s easy to imagine constructing the “physical” structure of an object in a virtual space, then move the parts of the object to “train” it and have the results recorded as a program than can be later refined. In this example the refinement could be through manipulation of the symbols representing the program, but perhaps more interesting it could be altered through interacting directly with the object during “playback”.
Another robotics technique that might apply here is Inverse_kinematics where you describe the target positions for the robot and let software decide how best to move the robots joints to reach these points, but I’ll leave that for another post.