Human-centred Robotics Systems Challenges
The main challenges in the key sectors of manufacturing, construction and agriculture targeted by SOPRANO relate to adopting reconfigurability and flexibility of a production process to support changing conditions. Also, sustainability in production goes hand-in-hand with flexibility, by optimizing the use of resources adjusted to the dynamic situation and current needs, and in consequence, reducing waste and overproduction.
In the Industry 5.0 paradigm, with the human element in focus, robotics is an enabling technology explored alongside the skills and capabilities of human workers, as traditional robotic systems cannot handle complex tasks on their own, nor can they easily be reconfigured and repurposed to satisfy changing needs. The challenge is to make the technology valuable to the user without disjoining the development from the actual end user, addressing acceptance and trust, and ultimately raising the wellbeing and safety of workers, especially in shared workspaces where safety-critical tasks are jointly performed. These challenges apply both to large as well as small and medium-sized industries.
Recent research highlights both a) the synergistic task execution by humans and robots as part of the collaborative robot paradigm and b) the strategic role of intelligent multi-robot systems where distributed and interconnected robots, with different characteristics and functions, are intelligently orchestrated to perform tasks whose complexity and cost are too demanding for a single robot to accomplish on its own. For the former, usually a single robotic platform is exploited that is constrained by the physical properties and hardware design, thus supporting a limited task repertoire. For the latter, the human element is often neglected and not fully integrated in dynamic multi-robot systems used in industrial environments. This negatively impacts flexibility and ability to adapt to the characteristics of the worker, the team, the dynamic environment (indoors or outdoors) and the various processes such as inspection, material handling, pick and place, or assembly.
SOPRANO combines these research strands to deliver the next generation of agile industries in manufacturing, construction and agriculture, underpinned by trustworthy and acceptable multi-human multi-robot (MH-MR) systems.
​
Project Concept
Project Objectives, Motivations and Challenges
​
Develop advanced human-centric robotic capabilities
Motivation: The interplay of robots and humans in industrial settings is an inherently complex task as robots need to ground their actions and decisions on situational constraints of the operating context. Developed robot capabilities should cope with a dynamically populated environment; maintain location awareness in indoor and outdoor environments; detect, track and manipulate objects with challenging characteristics; exploit collaborative sensing to improve the perception of individual robots; analyse the activities of humans; plan robot actions in real time based on time-inclusive and human-centric criteria.
Challenge: Overcome nuances and variability in human motion and actions, complexity and unpredictability of the environment, challenging appearances of industrial parts. Predicting, prioritizing and orchestrating interaction and collaboration with humans combining dynamic task allocation & synchronization, and team cognition.
Ensure trustworthy and dependable operation in MH-MR synergistic tasks
Motivation: Seamless collaboration in MH-MR systems when executing safety-critical tasks entails making credible decisions on safety and security under uncertainty to uphold the mandated dependability levels at runtime.
Challenge: Achieving truly peer-based synergy in MH-MR systems that enables the team to remain undisturbed and productive, coping with context uncertainties. Modelling these uncertainty sources at design time, and quantifying and explaining their origin and rationale at runtime are indispensable components to anticipate and mitigate safety-critical situations when performing human-robot interactions.
Provide modular and reconfigurable tools to aid deployability and adaptability
Motivation: Tools and techniques are needed for the development of highly flexible, reconfigurable, and modular solutions, allowing fast response to re-purposing changes in industrial requirements, reducing considerably the programming effort and configuration time needed for new products and tasks. The lack of easy-to-use configurable tools leads to solutions incurring high maintenance cost and technology stagnation.
Challenge: Develop reconfigurable, easy-to-use by non-experts tools and (model-based) abstractions for programming and configuring quickly new products and tasks. Support optimization and intelligent resource scheduling.
Realize the next generation of MH-MR teams in agile industries
Motivation: Demonstrate the effectiveness of the SOPRANO technological offering in three novel use cases, advancing and shaping the realization of the next generation of MH-MR teams in the key application sectors of manufacturing, construction and agriculture.
Challenge: Transitioning from the status quo in conventional industrial environments that targets excessive automation and use of robotic systems optimised to perform very specific tasks to achieve productivity and sustainability, to a new regime that promotes adaptivity, flexibility and fast response to re-purposing changes.
Project Technology Pillars
​
Pillar I: Advanced Collaborative Human-Centric Robotic Capabilities
Advancing the perception and cognition capabilities of robots, to effectively cope with uncertainty in context awareness by: a) exploiting and integrating data from low-cost visual and non-visual heterogeneous sensors distributed in the working environment, on mobile and static robots and human(s), being part of an IIoT environment; b) adopting a collective exploration and perception approach that combines data from different sensors to support navigation, scene interpretation and object detection/localization, resolve potential ambiguities and uncertainties, leading to superior robustness in dynamic environments both indoor and outdoor; c) addressing object detection and localization with challenging surface characteristics that are commonly found in industrial environments for object grasping, item picking, quality checking, etc.; d) analysing human motion patterns and activities to support more precise inferences on the human state, coping with the different levels of semantics in human analysis; e) considering the social and motivational aspects of human-robot interaction combining psychological and occupational science research methods with robotics and AI; and f) considering the skills of each team member and building upon the notion of time to orchestrate human-robot collaboration at the team (global) level and also enhance the short-term efficiency of goal-driven interaction at the human-robot dyad (local) level.
Pillar II: Trustworthy and Dependable AI-based MH-MR Teaming
A threefold strategy, driven by AI, safety engineering, simulation-based assurance, and digital twins will be utilised to manage uncertainty both at design time and at runtime, helping the MH-MR team to make credible decisions on safety and security. This strategy involves a) simulation-based testing, informed by hazards and threats identified during safety assessment, to launch targeted testing campaigns and assess the robustness and resilience of the MH-MR system across different scenarios helping to surface and rectify corner-case behaviours before deployment; b) advanced techniques for trustworthy and responsible AI to assess AI components used in MH-MR, while assured AI techniques can detect distribution shifts at runtime and prevent safety-critical situations before they emerge; and c) digital twins supporting the modelling of human-robot and robot-robot tasks and workflows on a fine-grained level based on “digital replicas” of machines, equipment and humans.
Pillar III: Modular, Reconfigurable and Flexible Engineering for Rapid Reconfiguration and Deployability
Multiple frameworks, tools and engineering approaches will be exploited to provide the desired modularity and reconfigurability in the project technologies and toolkits, supporting different operating contexts and tasks. These will rely on: a) context models relevant for the industry processes, exploiting ontologies that are easy to adapt and re-use in relevant contexts; b) model-based tools that enable both developers and non-expert users to reconfigure control architectures by modifying control policies at runtime; c) AI-driven robot programming tools enabling parameterization of robotic actions to fulfil the promised capability; d) mechanisms facilitating the configuration of AI-supported tools for different environments taking advantage of transfer learning principles, and examine inductive, transductive and unsupervised scenarios; and e) establishing an underlying computing and network infrastructure based on open-source edge cloud architectures, exploiting intelligent resource scheduling and optimization solutions for high performance in cloud-edge, using lightweight encryption mechanisms and supporting scalability of the solution.
​
