— Salah, could you tell us how relevant agricultural robotics is today, and how this relevance can be explained? Can it be explained, for instance, by the low productivity of human or mechanized labor, the shortage of manpower, the undesirable environmental effects of human activity?

I think it can be explained by all of the factors you have mentioned. By around 2050 there will be about 10 billion people on our planet, and the questions that will definitely arise are those of all these people’s consolidated income and the quality of their diet. Of course, this means that we will have to produce more food, but the Earth’s resources are limited, so we will have to produce it in a both more sustainable and more economically efficient manner. Besides, these days we are facing the problem of climate change, which leads to the changes in the flora and fauna, i.e. the pests that farmers have to deal with change, diseases and bacteria change, new soil problems and water resources problems emerge.

But there is another aspect: most farmers all over the world are middle-aged or senior citizens, because young people are generally not very willing to pursue careers in agriculture. And a lot of countries see the solution to that problem in the introduction of robotics into the agricultural industry.

Here in Australia agriculture is one of the key, fundamental branches of economy, and we export about 80% of our agricultural products. Australia has quite a large area, but the population of the country is just about 24 million people, and most of them live in big cities.

Besides, historically in Australia we take robotics very seriously. For instance, the mining industry here has been almost entirely automated by now. The same can be said about logistics and aviation, where automation is also quite well-developed. Which is why it seems logical that the next industry to be automated is agriculture.

— Can you tell us what your field robotics center generally does, and what particular systems do you develop? What results have you achieved as of now?

The Australian Center for Field Robotics is a Sydney-based R&D laboratory that was founded about 20 years ago. We are one of the largest research groups in the sphere of robotics in the world, with about 130 members. Of course, our primary area is field robotics, i.e. machines that do field work. But we don’t just work in agriculture but also the mining industry, the aviation industry, the transport industry and logistics. It’s only in the last 10-15 years that we’ve focused our primary efforts on agriculture.

At this moment we have a wide range of robotic systems for agriculture. We have systems for mechanical weeding of fields with grain crops, systems for orchard crops, systems for measuring the area of pastures, transporting livestock and so on.

About 95% of our funding comes from the agricultural industry itself, so we don’t get any government funding at all. We work directly with customers and we implement research and development ideas (including the ideas of our customers) turning them into actual machines, going all the way from laboratory prototypes to operational prototypes. In some cases we also commercialize our technologies. Naturally, our customers are interested in bringing their ideas to life so that the end result is functional and working. We often work on a certain model for a year or more, and even when we have the prototype ready, it may take one to two more years to modify and rework it, which is always done in cooperation with the customer. The readiness of a certain system to be commercialized heavily depends on whether it has been improved so as to reduce the expenses of its production as much as possible, i.e. it needs to be affordable and economically viable for the customer, so that they can buy it in the amount that they need. So the follow-up revision and improvement of our system is often not just functionality-related but also cost-related: we are trying to cut the cost of the technology as much as possible after we have finished working on the functionality.

© Australian Centre for Field Robotics

— So, the customer plays a key role in your business?

Yes, the most important thing is what the customers think and what their business model is. The technology is also important, of course, but we have been working in this industry for a long time, and we already know most key aspects of this technology. We have qualified engineers and project managers that handle their work really well. We have the necessary information technology and we are familiar with the latest A.I. developments, so I’d say that this aspect causes us the least amount of problems, and we also know exactly how to put together the team for every type of a project. I wouldn’t say it’s extremely easy, but it’s relatively simpler than everything else. At least we understand what obstacles and pitfalls there are with regard to this aspect, what problems exist in the sphere of information technology, and we know how to solve these problems. A much more difficult task comes after you have created a robotic system: you need to understand how the business model for this system will work, how it is going to be used, and how to get a good return on investment using this system, which is what the customer ultimately wants.

— How affordable are your robotic systems for most farmers? Can owners of small farms afford them, or are your customers only large agricultural enterprises?

We have completely different robots designed for different types and scales of agricultural production. There are systems designed specifically for large agricultural companies, and in that case, of course, the main customers and purchasers of the technology are the market majors. But we also had projects where we designed smaller systems for developing countries. For example, we introduced robotic systems in countries such as Fiji, Samoa, Indonesia, and we tried to make them as affordable as possible.

But there is also another thing: large agricultural enterprises often purchase or rent the machinery, while small companies very rarely do that. The systems for these small companies are usually bought by a separate company that does business in agricultural consulting, and this company offers the farmers their services if they want to use these robotic systems.

© Australian Centre for Field Robotics

© Australian Centre for Field Robotics

— Is machine labor more economically efficient than human labor in most cases?

Yes, it is more viable economically and also more precise. And, as strange as it may seem, it is a more topical issue in developing countries, where a lot of farmers come across the problem of finding skilled labor. It’s not that difficult to find cheap labor, but qualified personnel is few and far between. So it’s often less costly for these farmers to hire a consultant that would bring the robotics to their farm and perform certain works with it, like weeding or fertilization, than to find and hire qualified workers, even for a very short period of time.

— What are agricultural robots generally capable of, if you compare them to human workers? What is their main advantage?

The main factors in agriculture are the need to perform a lot of continuously repeated actions and often the necessity of working in the field non-stop 24/7. I have to admit that human labor is still superior in certain types of works, specifically those that require manual dexterity: for instance, in harvesting, especially harvesting fruit and berries, where fine motor skills of human hands are required. This work is more suitable for human hands and fingers rather than robots. But we are currently working in this direction and in the future, I’m sure, robotic systems will catch up with humans in this regard as well, but today it’s still the way it is. However, robots are better at handling monotonous, repeated actions and works like spot irrigation and spot pest control.

Another sphere that many people seem to forget about is monitoring the growth of agricultural crops: robotics are really good at tracing the different phases of growth and ripening of agricultural plants. And it’s a very important type of works that involves, for instance, determining the necessary nutrients or fertilizers for a particular plant at the current moment, etc. Automated systems do these calculations much more effectively than people.

© Australian Centre for Field Robotics

© Australian Centre for Field Robotics

— Do you think that the development of robotics will influence the basic principles of agricultural business? Will the methods of working in the field become more environmentally friendly? For example, we know that traditionally pest control chemicals are pulverized across the entire field, while robots are capable of performing selective spot pulverization.

Well, you can say to yourself: «Now we are doing this and that on our farm, so let’s automate these particular types of works.» But you can also ask yourself: «What is my ultimate goal on this farm, what result am I trying to achieve? And what would these tasks be if people didn’t participate in this process at all?» And then automation will take a slightly different turn, because the approaches and methods become different.  

To answer your question, I’ll say that your assumptions are absolutely correct. There is always the question of whether it’s more effective to spot-spray the chemicals rather than pulverize them across the entire field; or to weed selectively instead of weeding the whole area. By using robotics, we can estimate the need for weeding or using chemicals at the level of individual plants. So, robotic systems allow us to implement a customized approach to every individual plant and to make separate decisions about each individual plant as opposed to the entire field.

But it would also be great to ask yourself this question: «If we try to imagine a farm with only robots working on it 10 years from now, what would it look like, and what would the process look like without any human participation?» And we ask ourselves this question all the time, and it influences the way we think and come up with ideas. Let’s say I’m a farmer and I have some fruit trees on my farm. So, if I’m planning to automate my farm, I’m going to plant them in such a way so that it doesn’t get in the way of automation but instead contributes to its efficiency, and that would be a very different way of planting. And when you start thinking that way you may even prefer to lose some of your yield but instead organize things so that it’s helps the robotic systems to do the harvesting, because it might be more economically viable in the long run. If I’m an entrepreneur who owns a farm, I have to think about these things, too.

© Australian Centre for Field Robotics

© Australian Centre for Field Robotics

— How do you work on machine vision? That is, how do you teach robots to see, and what visual information are they capable of recognizing at this point?

It’s a combination of several different things: the sensor technology, which has a lot to do with recognizing the amount of light, and computational capabilities in order to process the visual information. It can be photographic images-based technology, or it can be laser-based technology. And if we are planning to design a system that is capable of seeing and visually recognizing certain information, we employ a certain type of technology and teach the machine to detect and determine what we need. If, for instance, we need a robot that detects weeds, we teach it to do exactly that. And if we need to detect the size and ripeness of fruit, then we set up a system and train it for that type of activity. For each particular task we develop a special algorithm of machine learning, and then we go to the field, test these systems, modify the algorithm, then test them again, modify the algorithm again and repeat this cycle until the robot starts working with the 100% (or nearly 100%) precision. One of the difficult things about it is that we often have to analyze enormous amounts of data in order to make sure that the same algorithm works with different types of soils, different planting formats, in different areas, etc.

© Australian Centre for Field Robotics

© Australian Centre for Field Robotics

— How easy to use is your technology? Because it’s one thing to purchase a robotic system for a farm and it’s quite another thing to learn how to operate and maintain it. Do your machines need specifically trained engineers to operate them?

Of course, everything depends on the specific features of a particular type of robotic systems. If we’re talking about large-scale agricultural production, large companies normally have their own internal department that is responsible for this sort of technological devices, and their own production team: engineers, agricultural technologists, machinery operators, etc. The same department usually is the one to decide how effective is a certain type of technology for their business. Besides, large companies always have engineers that are capable of doing maintenance for this sort of machinery, because it’s not all that different from doing maintenance for ordinary tractors or combine harvesters. If engineers and repair servicemen are sufficiently qualified, they will be able to do maintenance for our systems.

Speaking of small farms and minor productions, then, as I have already mentioned, they mostly employ the services of consultant firms whenever they need to adopt some technology. If a company hires a consultant, they conclude a contract with them, and this consultant is completely responsible for the machinery and technology used. So then it’s up to the consulting company have to hire the people who will be able to maintain these robotic systems.

— Are there any aspects in agriculture that are very difficult to automate, where human labor is hard to replace?

I think there are two main angles to this question: the comparison between human and machine labor in terms of physical activity and their comparison in terms of mental activity. At the physical level we have everything people do with their fingers such as trimming the branches of trees or cutting the flowers or, for example, pollination — all of that is quite hard to do with robotics. Of course, these tasks are potentially solvable in the measurable future, and the technology development is heading in that direction. So I think pretty soon we will be able to recreate or imitate a human hand.

As for the mental labor, then we need to remember that operational mindset and strategic thinking is very important for farmers. They need to think about a lot of things: weather, supplies of resources and materials, water and electricity, logistics, local and regional peculiarities of agriculture, the social aspects of this business. All those things require highly organized strategic thinking. And naturally it’s quite difficult to recreate that using robotic systems. They are good for certain repeated actions, but it’s still too early to speak about robots substituting people in the way of strategic evaluation or making important decisions. Maybe in agriculture the processes of strategic thinking or evaluation shouldn’t be automated at all. I feel that robotic systems here serve more as an assistance to people, and it’s up to the farmers themselves to decide how much of this help they want to use. And also there is one thing that needs to be said about food production: it plays a very special role in our lives, and traditionally people have been doing it by themselves. So I’d say that the role of robotics in food production is more that of tools which assist farmers.

— Which of your projects was the most difficult one, what problems did you have to face while doing it, and how did you solve them?

I think that the hardest projects were the animal farming ones.

We had to deal with cattle here, and these animals can weigh up to 200-500 kilograms. They usually graze in pastures that aren’t always flat, that contain some relief and ground features such as rocks, valleys, large stones, etc. This is why even when robots are designed to simply measure the area of such pastures, lots of issues arise: for instance, they need to move and overcome the obstacles, including the animals, that also move around all the time.

The second difficult task in agricultural robotics is monitoring animals’ health. Here we get a combination of several problems: firstly, we need to create very complex, advanced and labor-consuming robotic systems; secondly, we have to make their production cost relatively low, so that they are affordable for the end users; thirdly, here we use very advanced systems of sensors as well as complex machine learning systems. You could say that this is robotics of the highest level and, frankly speaking, it’s still a long way for us to go till we get really good machines and technologies in this sphere.

© Australian Centre for Field Robotics

— Can you think of some examples of systems you designed that weren’t very successful, or ones that there hasn’t been any real demand for yet? Maybe something that is a little ahead of its time?

As I have mentioned, we are generally very customer-oriented, but we also do some fundamental research, too. There are some developments that we think will become relevant in about 5-10 years. It’s even possible that some of this research has no practical value at all, but we learn a lot from it. To give you an example, we had a project that had to with drones. However, those were not the usual propeller-based drones but the ones with wings that looked like small airplanes. And we were trying to teach those drones to rise up using waves of hot air, similarly to how eagles fly, catching rising currents of air and lifting themselves up. Our goal was to make the drones hang in the air for a long time without losing altitude. And we managed to do it: our drones were indeed able to catch hot waves, lift themselves up pretty high and stay in the air for quite a long time, monitoring the environment, and then they would move to a different area, where they would catch those waves again, and so on. This was a very good project, because it taught us a lot in terms of the theory, particularly of machine learning. But also it’s not a very practical project for the simple reason that this technology is still very expensive. Probably this project is one of those that are slightly ahead of their time, and it might become practical in about 10 years.

— As a university professor, how do you see the role of universities and colleges in the technological advances, and are there people among your graduate or post-graduate students that are capable of developing this in the future?

20 years ago, when we had only started designing robotics, the hardest thing to do was to build one robot that would work. And that was the role of universities back then: to push science in that direction. Today building a robot is not that difficult, they are used everywhere, and the questions we ask ourselves are different. They are more serious questions than the ones that we asked ourselves 20 years ago: how to make robots more intelligent, what are the effective algorithms of interaction between different robots, between robots and people, etc. A lot of my students work in companies such as Google, Apple, Facebook, as well as companies that produce both ground-based and airborne robotic systems. But there are also students that want to do research at the university and work on fundamental problems related to artificial intelligence, or study new materials and how the creation of these materials may influence the production of robotic systems and increase their efficiency. I’m talking about things like 3D printing, new polymeric compounds, new types of plastic and so on.

— What are you working on right now? What robots can we hope to see in the next 5-10-15 years and how, in your opinion, will agricultural lands be organized in the future?

Right now I completely devote myself to agricultural projects. Firstly because it’s interesting for me personally, and secondly because I feel that improving the effectiveness of automation in this particular sphere is an important task. Of course, except robotics, there are a lot of other things to work on in agriculture as well, but robotics is a particularly important area today, when a lot of modern technology is very rapidly developing, which is why agricultural robotics are my current focus. I think that in about 15 years robots on farms will become a common and ordinary thing, the same way as a vacuum cleaner is in every home. But I’m also sure that you will be able to see a lot of robots in large agricultural companies even sooner than that: I’d say it’s definitely happening in about 5 years.