“Time” is the most important limiting factor in an Urban Search And Rescue (USAR) mission. Actually the chance of surviving victims drastically decreases 24 hours after a disaster. In this critical situation, rescue personnel should avoid any risky tasks. Published texts indicate that surviving a trapped rescuer takes about 4 hours for a team of 10 rescuers and it’ll take almost 10 hours if the rescuer is injured or entombed!
Rescue teams can reduce the risk of surviving victims by means of robotics. Robots can provide accurate information about the collapsing walls, hazard materials etc. what rescue dogs cannot do. One may want to put a camera on a rescue dog to inspect collapsed buildings but, these video streams certainly won’t provide the environmental awareness that a smoothly controlled tele-operated robot can. In other words the “accuracy and quality” of environmental awareness (identification of victims can also be considered as a part of environmental awareness) provided by robots is the most significant advantage of these instruments in comparison with rescue dogs. We can also use these two parameters to have a rough estimation about the success of rescue phase in a USAR mission: “the more accurate and usable information one gets in a search phase, the more victims are expected to be survived in rescue time.”
In RoboCup Rescue scenario that is a standardized simulation of real disasters, rescue teams very well know how many people are injured, who they are and where they are located (rescuers may get this information from a survived person in real situations). The task is “to explore the arena by a team of robots with different levels of autonomy in a limited time to gather accurate and reusable information about the surroundings and victims.”
Like in a real USAR mission, here we can expect that a victim will be survived if a safe path to his/her location is discovered. Therefore, a victim should first be detected then his/her exact location be marked on the currently generated map of the environment to start rescuing. If the team cannot find a safe path to the location of detected victim, the rescuers should probably put their own life in a serious danger to survive that victim. This means that a “search phase” is not accomplished unless rescuers can prepare a map of the environment having accurate location of victims with a path for following. If we suppose that the chance of rescuing an identified victim linearly increases with the quality and accuracy of generated map, the following formulas can be used for evaluating general performance of a rescue team:
As an instance, assume that a team could find 5 victims out of 12 in a mission and they could identify all the signs of life (victim identification score = 30) while they could generate a half accurate map (mapping score = 10). Their final score is calculated as the followings:
Score of each detected victim = (10 / 20) x (30 / 30) = 0.5
Final score = (100 / 12) x (5 x 0.5) = 20.8%
This shows that the mission was 20.8% successful!
How would it be if they could find 3 victims with a completely accurate map?
Score of each detected victim = (20 / 20) x (30 / 30) = 1.0
Final score = (100 / 12) x (3 x 1.0) = 25.0%
It seems that they really need to improve their mapping for the next year!
Now let’s take a closer look at this scoring method:
Since final result of each team is calculated by adding all identified victims’ score which itself is the product of mapping and victim identification results, the final score highly depends on efficiency of applied algorithms (Computer Science) and robustness of utilized electro-mechanical systems (Mechatronics). Obviously a team should be very well prepared in terms of mechatronics, computer science and human resources if it wants to be successful in the competitions. Unlike the existing formula, a team will not be able to completely cover its weakness in one of the aforementioned areas by concentrating on its capabilities in the other fields. This can also be considered as a step towards eliminating operator’s skill in success of rescue teams (the end users of rescue robots won’t necessarily have all the capabilities of a well trained operator in RC RRL).
Advantages of proposed method:
At the end I think the proposed scoring formula can help to improve the quality of our highly valuable RoboCup Rescue Robot League.
Rescue teams can reduce the risk of surviving victims by means of robotics. Robots can provide accurate information about the collapsing walls, hazard materials etc. what rescue dogs cannot do. One may want to put a camera on a rescue dog to inspect collapsed buildings but, these video streams certainly won’t provide the environmental awareness that a smoothly controlled tele-operated robot can. In other words the “accuracy and quality” of environmental awareness (identification of victims can also be considered as a part of environmental awareness) provided by robots is the most significant advantage of these instruments in comparison with rescue dogs. We can also use these two parameters to have a rough estimation about the success of rescue phase in a USAR mission: “the more accurate and usable information one gets in a search phase, the more victims are expected to be survived in rescue time.”
In RoboCup Rescue scenario that is a standardized simulation of real disasters, rescue teams very well know how many people are injured, who they are and where they are located (rescuers may get this information from a survived person in real situations). The task is “to explore the arena by a team of robots with different levels of autonomy in a limited time to gather accurate and reusable information about the surroundings and victims.”
Like in a real USAR mission, here we can expect that a victim will be survived if a safe path to his/her location is discovered. Therefore, a victim should first be detected then his/her exact location be marked on the currently generated map of the environment to start rescuing. If the team cannot find a safe path to the location of detected victim, the rescuers should probably put their own life in a serious danger to survive that victim. This means that a “search phase” is not accomplished unless rescuers can prepare a map of the environment having accurate location of victims with a path for following. If we suppose that the chance of rescuing an identified victim linearly increases with the quality and accuracy of generated map, the following formulas can be used for evaluating general performance of a rescue team:
(1) Score of a detected victim = (mapping score at the point / maximum mapping score) x (victim identification score / maximum score of victim identification)
(2) Final score of a mission = (100 / QTY of all victims) x (sum of victim’s scores)
(2) Final score of a mission = (100 / QTY of all victims) x (sum of victim’s scores)
As an instance, assume that a team could find 5 victims out of 12 in a mission and they could identify all the signs of life (victim identification score = 30) while they could generate a half accurate map (mapping score = 10). Their final score is calculated as the followings:
Score of each detected victim = (10 / 20) x (30 / 30) = 0.5
Final score = (100 / 12) x (5 x 0.5) = 20.8%
This shows that the mission was 20.8% successful!
How would it be if they could find 3 victims with a completely accurate map?
Score of each detected victim = (20 / 20) x (30 / 30) = 1.0
Final score = (100 / 12) x (3 x 1.0) = 25.0%
It seems that they really need to improve their mapping for the next year!
Now let’s take a closer look at this scoring method:
Since final result of each team is calculated by adding all identified victims’ score which itself is the product of mapping and victim identification results, the final score highly depends on efficiency of applied algorithms (Computer Science) and robustness of utilized electro-mechanical systems (Mechatronics). Obviously a team should be very well prepared in terms of mechatronics, computer science and human resources if it wants to be successful in the competitions. Unlike the existing formula, a team will not be able to completely cover its weakness in one of the aforementioned areas by concentrating on its capabilities in the other fields. This can also be considered as a step towards eliminating operator’s skill in success of rescue teams (the end users of rescue robots won’t necessarily have all the capabilities of a well trained operator in RC RRL).
Advantages of proposed method:
- More accurate and still fair: As it was discussed, a team should be the best one in mechatronics, computer science and human resources if it wants to win the place award because these three factors have less effect on each other comparing to the existing formula.
- Easy to understand scoring result: Everyone (competitors and audiences) can easily understand how the performance of a team is even without comparing it with other teams’ scores when it is presented in percentage.
- Flexibility: Since the final score is a fraction of maximum available score, it does not depend on how many victims are placed in the arena. Therefore, the committee can freely add or remove some victims in an event while the performance of participants will be easily evaluated with previous events.
- Encouraging development of accurate real-time SLAM algorithms: Generally accuracy and quality of mapping algorithms reduce when the speed of a mobile platform increases. So, teams need to develop their mapping algorithms to have more accurate maps when they drive their robots faster to find more victims.
- Necessity of search strategy: One may want to find more victims to have grater score while another one may decide to get complete score of each victim. So, having a search strategy based on technical capabilities besides compromising between finding more victims and having more accurate maps will be inevitable.
- Shortening the score gap between auto-based and tele-based teams: Most auto-based teams produce very high quality maps. So, their mapping coefficient will be greater than tele-based teams. As a result, they can probably get more score if they find a victim.
- Map scoring: The presented scoring formula highly depends on mapping score. Since the automatic map scoring system is still under development by the RC RRL committee, final results will rely on human factor till a reliable automatic map scoring system becomes available.
At the end I think the proposed scoring formula can help to improve the quality of our highly valuable RoboCup Rescue Robot League.
