Air-Cobot

This is a good article. Click here for more information.
Source: Wikipedia, the free encyclopedia.

Air-Cobot
CountryFrance
TypeCobot
Websiteaircobot.akka.eu

Air-Cobot (Aircraft Inspection enhanced by smaRt & Collaborative rOBOT) is a French research and development project of a wheeled collaborative mobile robot able to inspect aircraft during maintenance operations. This multi-partner project involves research laboratories and industry. Research around this prototype was developed in three domains: autonomous navigation, human-robot collaboration and nondestructive testing.

Air-Cobot is presented as the first wheeled robot able to perform visual inspections of aircraft. Inspection robots using other types of sensors have been considered before, such as the European project Robair. Since the launch of the project, other solutions based on

image processing began to be developed, such as EasyJet with a drone, the swarm of drones from Toulouse company Donecle and the Aircam project of the aerospace manufacturer
Airbus.

Since the beginning of the project in 2013, the Air-Cobot robot is dedicated to inspect the lower parts of an aircraft. In the continuation of the project, there is the prospect of coupling with a drone to inspect an aircraft's upper parts. In October 2016, Airbus Group launched its research project on the hangar of the future in Singapore. The robots from the Air-Cobot and Aircam projects are included in it.

Project description

Objectives

Launched in January 2013,

Air France Industries.[5]

Partners

Air France Industries.[5]

The project leader is Akka Technologies. There are two academic partners; Akka Technologies and four other companies make up the five commercial partners.[6]

Academic partners
Industrial partners

Project finance

Project finance is provided by

Aquitaine Regional Council, the Pyrénées-Atlantiques Departemental Council, the Midi-Pyrénées Regional Council and by the European Union.[12]

Expected benefits

Aircraft are inspected during maintenance operations either outdoors on an airport between flights, or in a hangar for longer-duration inspections. These inspections are conducted mainly by human operators, visually and sometimes using tools to assess defects.[A 1] The project aims to improve inspections of aircraft and traceability. A database dedicated to each aircraft type, containing images and three-dimensional scans, will be updated after each maintenance. This allows for example to assess the propagation of a crack.[4][13]

The human operator's eyes fatigue over time while an automatic solution ensures reliability and repeatability of inspections. The decrease in time taken for inspections is a major objective for aircraft manufacturers and airlines. If maintenance operations are faster, this will optimize the availability of aircraft and reduce maintenance operating costs.[4][13]

Robot equipment

Air France Industries.[A 1]

All electronics equipment is carried by the 4MOB mobile platform manufactured by Sterela. The off-road platform, equipped with four-wheel drive, can move at a speed of 2 metres per second (7.2 kilometres per hour (4.47 mph)).[11] Its lithium-ion battery allows an operating time of eight hours. Two bumpers are located at the front and at the rear. These are obstacle detection bumpers. They stop the platform if they are compressed.[11]

The cobot weighs 230 kilograms (507 lb). It has two computers, one running

geofencing tasks in outdoor environments.[3][7]

Autonomous navigation

The autonomous navigation of the Air-Cobot robot is in two phases. The first, navigation in the airport or the factory, allows the robot to move close to the aircraft. The second navigation, around the aircraft, allows the robot to position itself at control points referenced in the aircraft virtual model. In addition, the robot must insert itself in a dynamic environment where humans and vehicles are moving. To address this problem, it has an obstacle avoidance module. Many navigation algorithms are constantly running on the robot with real time constraints. Searches are conducted on optimizing the computing time.[citation needed][clarification needed]

Navigation in the airport or the factory

In an outdoor environment, the robot is able to go to the inspection site by localizing through

geofencing. At the airport, the robot operates in dedicated navigation corridors respecting speed limits. Alerts are sent to the operator if the robot enters a prohibited area or exceeds a given speed.[10][A 2]

Another algorithm based on computer vision provides, in real-time, a lane marking detection. When visible, painted lanes on the ground can provide complementary data to the positioning system to have safer trajectories.[A 3] If in an indoor environment or an outdoor environment where GPS information is not available, the cobot can be switch to follower mode to move behind the human operator and follow her or him to the aircraft to inspect.[14][A 2]

Navigation around the aircraft

To perform the inspection, the robot has to navigate around the aircraft and get to the checkpoints called up in the aircraft virtual model. The position of the aircraft in the airport or factory is not known precisely; the cobot needs to detect the aircraft in order to know its position and orientation relative to the aircraft. To do this, the robot is able to locate itself, either with the laser data from its laser range finders,[A 4] or with image data from its cameras.[A 1][A 5]

Near the aircraft, a point cloud in three dimensions is acquired by changing the orientation of the laser scanning sensors fixed on pan-tilt units. After filtering data to remove floor- or insufficiently large dot clusters, a registration technique with the model of the aircraft is used to estimate the static orientation of the robot. The robot moves and holds this orientation by considering its wheel odometry, its inertial unit and visual odometry.[A 4]

"Matching clouds"
  • 3D laser acquisition in an outside environment.[A 4]
    3D laser acquisition in an outside environment.[A 4]
  • Matching of the outside data with the aircraft model.[A 4]
    Matching of the outside data with the aircraft model.[A 4]
  • 3D laser acquisition in an inside environment.[A 4]
    3D laser acquisition in an inside environment.[A 4]
  • Matching of the inside data with the aircraft model.[A 4]
    Matching of the inside data with the aircraft model.[A 4]
Air-Cobot can estimate its position relative to an aircraft by using visual landmarks on the fuselage.[A 5]

Laser data are also used horizontally in two dimensions. An algorithm provides a real-time position estimation of the robot when enough elements from the landing gears and engines are visible. A confidence index is calculated based on the number of items collected by lasers. If good data confidence is achieved, the position is updated. This mode is particularly used when the robot moves beneath the aircraft.[A 4]

For visual localization, the robot estimates its position relative to the aircraft using visual elements (doors, windows, tires, static ports etc.) of the aircraft. During the evolution of the robot, these visual elements are extracted from a three-dimensional virtual model of the aircraft and projected in the image plane of the cameras. The projected shapes are used for

Speeded Up Robust Features (SURF) approach. A pairing is performed between images of each element to be detected and the actual scene experienced.[A 1]

By detecting and tracking visual landmarks, in addition to estimating its position relative to the aircraft, the robot can perform a visual servoing.[A 6] Research in vision is also conducted on simultaneous localization and mapping (SLAM).[A 7][A 8] A merger of information between the two methods of acquisition and laser vision is being considered. Artificial intelligence arbitrating various locations is also under consideration.[A 4][A 1]

Obstacle avoidance

In both navigation modes, Air-Cobot is also able to detect, track, identify and avoid obstacles that are in its way. The laser data from laser range sensors and visual data from the cameras can be used for detection, monitoring and identification of the obstacles. The detection and monitoring are better in the two-dimensional laser data, while identification is easier in the images from the cameras; the two methods are complementary. Information from laser data can be used to delimit work areas in the image.[A 6][A 9][A 10]

The robot has several possible responses to any obstacles. These will depend on its environment (navigation corridor, tarmac area without many obstacles, cluttered indoor environment etc.) at the time of the encounter with an obstacle. It can stop and wait for a gap in traffic, or avoid an obstacle by using a technique based on a spiral, or perform

path planning trajectories.[A 6][A 10]

Computing time optimization

Given the number of navigation algorithms calculating simultaneously to provide all the information in real time, research has been conducted to improve the computation time of some numerical methods using field-programmable gate arrays.[A 11][A 12][A 13] The research focused on visual perception. The first part was focused on the simultaneous localization and mapping with an extended Kalman filter that estimates the state of a dynamic system from a series of noisy or incomplete measures.[A 11][A 13] The second focused on the location and the detection of obstacles.[A 12]

Non-destructive testing

Air-Cobot can inspect the blades of a turbofan engine.[A 14]

Image analysis

After having positioned to perform a visual inspection, the robot performs an acquisition with a

pan-tilt-zoom camera. Several steps take place: pointing the camera, sensing the element to be inspected, if needed repointing and zooming with the camera, image acquisition and inspection. Image analysis is used on doors to determine whether they are open or closed; on the presence or absence of protection for certain equipment; the state of turbofan blades or the wear of landing gear tires.[A 14][A 15][A 16][A 17]

The detection uses pattern recognition of regular shapes (rectangles, circles, ellipses). The 3D model of the element to be inspected can be projected in the image plane for more complex shapes. The evaluation is based on indices such as the uniformity of segmented regions, convexity of their forms, or periodicity of the image pixels' intensity.[A 14]

The

static ports being covered or not covered. A pairing is performed between images of the element to be inspected in different states and that present on the scene. For these simple items to be inspected, an analysis during navigation is possible and preferable due to its time saving.[A 1][A 18]

Point cloud analysis

After having positioned to perform a scan inspection, the pantograph elevates the

3D scanner at the fuselage. A pan-tilt unit moves the scan device to acquire the hull. By comparing the data acquired to the three-dimensional model of the aircraft, algorithms are able to diagnose any faults in the fuselage structure and provide information on their shape, size and depth.[15][A 19][A 20]

By moving the pan-tilt units of the laser range finders, it is also possible to obtain a point cloud in three dimensions. Technical readjustment between the model of the aircraft and the scene point cloud is already used in navigation to estimate the static placement of the robot. It is planned to make targeted acquisitions, simpler in terms of movement, to verify the absence of chocks in front of the landing gear wheels, or the proper closing of engine cowling

latches.[A 4]

Collaboration human-robot

As the project name suggests, the mobile robot is a cobot – a collaborative robot. During phases of navigation and inspection, a human operator accompanies the robot; he can take control if necessary, add inspection tasks, note a defect that is not in the list of robot checks, or validate the results. In the case of pre-flight inspections, the diagnosis of the walk-around is sent to the pilot who decides whether or not to take off.[7][14][A 21]

Other robotic inspection solutions

The drones can inspect upper parts of the aircraft such as the tail and simplify maintenance checks.

European project Robair

The inspection robot of the European project Robair, funded from 2001 to 2003, is designed to mount on the wings and

thermographic techniques. It detects loose rivets and cracks.[16][17][18]

EasyJet drone

Airline EasyJet is interested in the inspection of aircraft with drones. It made a first inspection in 2015. Equipped with laser sensors and high resolution camera, the drone performs autonomous flight around the aeroplane. It generates a three-dimensional image of the aircraft and transmits it to a technician. The operator can then navigate in this representation and zoom to display a high-resolution picture of some parts of the aircraft. The operator must then visually diagnose the presence or absence of defects. This approach avoids the use of platforms to observe the upper parts of the aeroplane.[19]

Donecle drone

Donecle's autonomous drone inspecting an aircraft.

Founded in 2015, Donecle, a Toulouse start-up company, has also launched a drone approach which was initially specialized in the detection of lightning strikes on aeroplanes.[20][21] Performed by five people equipped with harnesses and platforms, this inspection usually takes about eight hours. The immobilization of the aircraft and the staff are costly for the airlines, estimated at $10 000 per hour. The solution proposed by the start-up lasts twenty minutes.[21]

Donecle uses a swarm of drones equipped with laser sensors and micro-cameras. The algorithms for automatic detection of defects, trained on existing images database with a

pitot probes, rivets, openings, text, defects, corrosion, oil stains. A damage report is sent on the operator's touch pad with each area of interest and the proposed classification with a probability percentage. After reviewing the images, the verdict is pronounced by a qualified inspector.[21]

Project continuation

In 2015, in an

Airbus A320. The CEO also revealed that Akka Technologies plans to work on a duo of robots for inspection: the same mobile platform for the lower parts, and a drone for the upper parts. If funding is allocated then this second phase would take place during the period 2017–2020.[3]

At the

new technologies can be tested. Upon entering the hangar, cameras study the aircraft to detect damages. Mobile robots, such as the one of the Air-Cobot project, and drones, such as the one of the Aircam project, carry out more detailed inspections.[23]

During the 14th International Conference on Remote Engineering and Virtual Instrumentation in March 2017, Akka Research Toulouse, one of the centers for

Sud-Aviation SE 210 Caravelle.[25]

Communications

Airbus A320 in a hangar.[A 4]

On 23 October 2014, a patent was filed by

conference proceedings and four journal articles.[30] Part of publications is centered on navigation and/or inspection by Air-Cobot while the rest focuses on specific numerical methods or hardware solutions related to the issues of the project. During the international conference Machine Control and Guidance (MCG) of 2016, the prize for the best final application is awarded to the authors of the publication Human-robot collaboration to perform aircraft inspection in working environment.[31]

On 17 April 2015, Airbus Group distributed a project presentation video, made by the communication agency Clipatize, on its YouTube channel.

Laboratoire d'analyse et d'architecture des systèmes during development, researchers or engineers working on the project regularly present a demonstration to visitors (external researchers, industrial partners, or students); it was also demonstrated to the general public during the 2015 Feast of Science.[34] Airbus Group, on 17 February 2016, broadcast a YouTube video presentation of its vision of the hangar of the future in which it plans to use Air-Cobot.[22]

See also

Wikimedia Commons has media related to Air-Cobot.

Notes and references

Research publications of the project

  1. ^ a b c d e f Villemot, Larnier & Vetault 2016, RFIA
  2. ^ a b c d e Donadio et al. 2017, REV
  3. ^ Bauda, Bazot & Larnier 2017, ECMSM
  4. ^ a b c d e f g h i j Frejaville, Larnier & Vetault 2016, RFIA
  5. ^ a b c Jovancevic et al. 2016, ICPRAM
  6. ^ a b c Futterlieb, Cadenat & Sentenac 2014, ICINCO
  7. ^ Esparza-Jiménez, Devy & Gordillo 2014, FUSION
  8. ^ Esparza-Jiménez, Devy & Gordillo 2016, Sensors
  9. ^ Lakrouf et al. 2017, ICMRE
  10. ^ a b Leca et al. 2019, ECC
  11. ^ a b Tertei, Piat & Devy 2014, ReConFig
  12. ^ a b Alhamwi, Vandeportaele & Piat 2015, ICVS
  13. ^ a b Tertei, Piat & Devy 2016, CEE
  14. ^ a b c Jovancevic et al. 2015, JEI
  15. ^ Jovancevic et al. 2015a, QCAV
  16. ^ Jovancevic et al. 2015b, CMOI
  17. ^ Jovancevic et al. 2016, MECO
  18. ^ Leiva et al. 2017, ECMSM
  19. ^ Jovancevic et al. 2017, I2M
  20. ^ Bauda, Grenwelge & Larnier 2018, ETRSS
  21. ^ Donadio et al. 2016, MCG

Proceedings

Journal articles

PhD thesis reports

Other references

  1. ^ a b (in French) Xavier Martinage (17 June 2015). "Air-Cobot : le robot dont dépendra votre sécurité". lci.tf1.fr. La Chaîne Info. Archived from the original on 3 January 2016. Retrieved 12 July 2016.
  2. ^ a b (in French) "Air-Cobot : un nouveau mode d'inspection visuelle des avions". competitivite.gouv.fr. Les pôles de compétitivité. Archived from the original on 11 October 2016. Retrieved 12 July 2016.
  3. ^ a b c d e f (in French) Olivier Constant (11 September 2015). "Le projet Air-Cobot suit son cours". Air et Cosmos (2487). Retrieved 12 July 2016.
  4. ^ a b c (in French) "Rapport d'activité 2013–2014 de l'Aerospace Valley" (PDF). aerospace-valley.com. Aerospace Valley. Archived from the original (PDF) on 24 September 2016. Retrieved 12 July 2016.
  5. ^ a b (in French) "News du projet Air-Cobot". aircobot.akka.eu. Akka Technologies. Archived from the original on 10 July 2016. Retrieved 12 July 2016.
  6. ^
    Capital. 1 July 2014. Archived from the original
    on 25 June 2016. Retrieved 14 July 2016.
  7. ^ a b c d e f g h i (in French) "Air-Cobot, le robot qui s'assure que vous ferez un bon vol !". Planète Robots (38): 32–33. March–April 2016.
  8. Laboratoire d'analyse et d'architecture des systèmes. Archived from the original
    on 14 September 2015. Retrieved 17 July 2016.
  9. ^ (in French) "Akka Technologies : une marque employeur orientée sur l'innovation". Le Parisien. 15 February 2016. Retrieved 17 July 2016.
  10. ^ a b "M3 Systems Flagship Solution". M3 Systems. Archived from the original on 6 August 2016. Retrieved 17 July 2016.
  11. ^ a b c (in French) "4MOB, plateforme intelligente autonome" (PDF). Sterela Solutions. Archived from the original (PDF) on 9 August 2016. Retrieved 17 July 2016.
  12. ^ (in French) "Financeurs". aircobot.akka.eu. Akka Technologies. Archived from the original on 4 August 2016. Retrieved 15 July 2016.
  13. ^ a b (in French) Véronique Guillermard (18 May 2015). "Aircobot contrôle les avions avant le décollage". Le Figaro. Retrieved 14 July 2016.
  14. ^
    YouTube
  15. ^ (in French) Pascal NGuyen (December 2014). "Des robots vérifient l'avion au sol". Sciences et Avenir (814). Archived from the original on 8 August 2016. Retrieved 17 July 2016.
  16. ^ (in French) "Robair, Inspection robotisée des aéronefs". European Commission. Retrieved 16 July 2016.
  17. ^ "Robair". London South Bank University. Retrieved 16 July 2016.
  18. doi:10.1108/01439910710832093
    .
  19. ^ (in French) Newsroom (8 June 2015). "Easy Jet commence à utiliser des drones pour l'inspection de ses avions". Humanoides. Archived from the original on 12 October 2015. Retrieved 16 July 2016.
  20. ^ (in French) Florine Galéron (28 May 2015). "Aéronautique : la startup Donecle invente le drone anti-foudre". Objectif News, la Tribune. Retrieved 16 July 2016.
  21. ^ a b c (in French) Arnaud Devillard (20 April 2016). "Des drones pour inspecter des avions". Sciences et Avenir. Archived from the original on 8 August 2016. Retrieved 16 July 2016.
  22. ^
    YouTube
  23. ^ "Pimp my Hangar: Excelling in MRO". airbusgroup.com. Airbus. Archived from the original on 21 December 2016. Retrieved 21 December 2016.
  24. ^ (in French) Éric Parisot (21 June 2013). "Co-Friend, le système d'analyse d'images qui réduit les temps d'immobilisation des avions". Usine Digitale. Retrieved 24 February 2018.
  25. ^ (in French) Aeroscopia, ed. (August 2017). "Le Musée accueille le projet AIR-COBOT". musee-aeroscopia.fr. Archived from the original on 14 October 2017. Retrieved 24 February 2018.
  26. ^ "Espacenet – Bibliographic data – Collaborative robot for visually inspecting an aircraft". worldwide.espacenet.com. Retrieved 1 June 2016.
  27. ^ (in French) Juliette Raynal; Jean-François Prevéraud (15 June 2015). "Bourget 2015 : les dix rendez-vous technos à ne pas louper". Industrie et Technologies. Retrieved 16 July 2016.
  28. ^ (in French) "Akka Technologies au Salon du Bourget". Maurice Ricci. 21 June 2015. Archived from the original on 4 April 2016. Retrieved 16 July 2015.
  29. ^ "Singapore Airshow 2016 Trends: Emerging Technologies Take Off – APEX | Airline Passenger Experience". apex.aero. Retrieved 1 June 2016.
  30. ^ "Communications du projet Air-Cobot". aircobot.akka.eu (in French). Akka Technologies. Archived from the original on 11 August 2016. Retrieved 14 July 2016.
  31. ^ "Best MCG2016 Final Application Award" (PDF). mcg2016.irstea.fr. Machine Control and Guidance. October 2016. Retrieved 22 February 2020.
  32. ^ "AirCobot – Introducing Smart Robots for Aircraft Inspections". clipatize.com. Clipatize. Archived from the original on 6 August 2016. Retrieved 15 August 2016.
  33. YouTube
  34. ^ Air-Cobot, le robot d'assistance aux inspections des aéronefs (PDF). Programme de la fête de la science (in French). 2015. Retrieved 17 July 2016.

External links

Retrieved from "https://en.wikipedia.org/w/index.php?title=Air-Cobot&oldid=1192251894"