Adaptive learning robots to complement the human workforce

Imagine a world where outdoor robots can be trained to work alongside humans in harsh and highly dynamic environments, taking on the risky jobs and protecting human workers. 

Hero image for project IMG 3168 770x400px
The robot-mule developed by the Spearhead team as an exemplar for supporting humans; here forestry workers by delivering seedlings to human planters somewhere in a very steep, rough terrain.

This Spearhead project is studying the underlying differences to standard robotics where methods are available for simple environments (roads, warehouses, factories etc.). Sensing technologies and new algorithms, e.g. learning capabilities, are required to operate robots outdoors in very challenging, highly unstructured environments (e.g. see next image). The project will also provide unique prototypal hard- and software to display the project outputs, integrated into one unit for a practical exemplar.



  • Industry is increasingly demanding robots that can safely complete tasks and adapt to a variety of environments as they work in close quarters with humans.

  • Guided by industry advisors, our researchers are developing algorithms, software and robotic prototypes to build smarter robots with safety as the key priority.

  • This project represents an invaluable opportunity to build New Zealand’s capability in robotics, while generating new automation options in everything from forestry to high-value manufacturing.

Hero image Robotics tentacle
An example of easily adaptive configuration and hardware (‘tentacle’) on the Demonstrator, featuring a high level of flexibility that is needed for changing robotic tasks.

Helping humans and robots work together

Robots are already present in industrial settings such as car manufacturing, automated milking, and meat processing. The outdoors, however, is a complex and changing environment and human workers are unpredictable and liable to move quickly.  Safety is a particular focus for this project. Robots in factories can be safely operated in cages or away from humans. Field robots need new abilities that allow them to perceive what is happening around them and use that information to learn and adapt.

These newer forms of robots are needed to support industries including small-medium primary industries, such as the seafood, horticultural and forestry sectors.

Hero image Robotics Person detection
Person-detection using neural network sensor fusion. Resulting data will be stored 3D-voxels an algorithmic representation of the dynamic environment.

Small resolution voxels - so many uses

Voxels are the equivalent of pixels in 3D. They can be recorded in very small resolution and stored in a smart and compressed way. This ‘Octree Compression’ allows a robot using this data to ‘learn’.

For example, researchers in this project have built a database of leaves so a robot can identify a particular type of tree. This could allow a robot to remove wild pine seedlings on the side of a rugged hill that is being restored to native bush.

The researchers have also sent robots into unknown rooms to see if they can predict how to manoeuvre around unobstructed (see next maps). This reinforcement learning has been very successful in simulations.

Hero image Robotics Trajectories
Trajectories of different methods after 1000 time steps. From left to right: our approach, ‘look-around’ option disabled and ‘frontier method’ (results from arbitrary point method and atomic action method not shown). Green areas represent obstacles while light blue represents free space. Red lines indicated the trajectories of agent. Orange dots are the frontiers. Unexplored areas are in grey.

An adaptive forestry robot to deliver seedlings

To develop adaptive learning technology further, our researchers use sensors and artificial intelligence so a robot-mule can perceive and understand its surroundings. The robot ‘teaches’ itself to flexibly handle new situations so that it could travel over rough, unpredictable, steep ground to carry seedlings to workers for planting.

To do this the robot-mule has to perceive the distance and position of objects (including humans) and collect data that meant it does not skid or tip over. The robot-mule stores 3D-maps to allow it to navigate its surroundings and anticipate any risks. Successful learning is reinforced and reintegrated into the robot’s representation of the environment, so the captured ‘past experience’ could be shared with other robots and used in further testing and development.

Hero image Robotics sm scale robot
Small-scale robot used to test skidding and slipping prediction/prevention algorithms.

Developing a niche market

The team envisage that after 8-10 years, Aotearoa-New Zealand will have developed niche adaptable robots in contrast to mass-produced single-task robots. The technology is expected to help the country’s industries thrive globally and create an international hub for innovative robotics development.

Introducing robots to unstructured environments and to closer working interaction with humans poses unique challenges. Tackling has required innovations, such that this project is now well placed to help Aotearoa prepare for a future where humans and robots work together in harmony.

Project background information

This Mission-led Spearhead Project commenced in 2017, ending in June 2022 and has received a total of $5M funding. Find out how we develop our Mission-led projects.