Reading comprehension of natural language instructions by robots

Abstract

We address the problem of robots executing instructions written for humans. The goal is to simplify and speed-up the process of robot adaptation to certain tasks, which are described in human language. We propose an approach, where semantic roles are attached to the components of instructions which lead to robotic execution. However, extraction of such roles from the sentence is not trivial due to the prevalent non determinism of human language. We propose algorithms for extracting actions and object names with roles and explain, how it leads to the robotic execution via attached sub-symbolic information of previous execution examples for rotor assembly and bio(technology) laboratory scenarios. The precision for the main action extraction is 0.977, for the main, primary and secondary objects is 0.828, 0.943 and 0.954, respectively