Artificial Swarm Intelligence, Autonomous Systems and Drones
This presentation examines artificial swarm intelligence for autonomous and semi-autonomous robotic systems, with particular emphasis on drones. It develops the idea that complex, resilient collective behaviour can emerge from cooperation between relatively simple AI agents, drawing on natural swarm intelligence, emergent behaviour and biological evolution. It then outlines a Swarm AI design trained using modified deep reinforcement learning and evolutionary optimisation, before demonstrating the approach in simulations of grid search, a self-healing communications repeater chain and three forms of coordinated autonomous swarm attack.
How You Might Train a Dog
A dog-fetching example explains the basic reinforcement-learning cycle. An agent observes its environment, chooses an action and receives a reward when it performs the desired behaviour, gradually learning which actions produce successful outcomes.
Deep RL
Deep reinforcement learning applies the same principle using an artificial neural network as the policy model. Sensor and state information are mapped to actions, while numerical rewards drive training so that the policy increasingly selects actions with higher expected future reward.
Deep-RL: A Two-Dimensional Robot Learning to Walk
A two-dimensional Bipedal Walker example illustrates deep reinforcement learning using PPO, an actor-critic algorithm. A policy network chooses motor actions while a value network estimates future reward, and both are improved through repeated interaction with the simulated environment.
A Problem with Multi-Agent Systems
Multi-agent reinforcement learning can fail if agents are rewarded only for individual success. A football analogy shows why training must encourage collective performance; the approach used here relies on reward shaping and cooperation factors to promote useful teamwork.
Simulation Examples of Swarm AI
The simulation section introduces five examples: Grid Searcher, a dynamic drone communications repeater chain, and three autonomous swarm attacks. The examples demonstrate pseudorandom search, resilience, attack deconfliction, synchronisation and cooperation between different agent types.
Grid Searcher
Grid Searcher uses ten simple ant-like agents to explore a 100 by 100 grid within 1,000 time steps. The agents have only local proximity sensing, no position information, no map, no memory and no communication, making the search task deliberately difficult.
Grid Searcher Simulation
Despite their severe limitations, the Grid Searcher agents typically cover around 60–70 percent of the grid. Their apparently disorganised motion makes constructive use of pseudorandom behaviour, illustrating that an effective learned swarm strategy may not be intuitively understandable to a human observer.
Swarm Repeaters
A swarm of autonomous drones is tasked with maintaining a dynamic communications chain between a human pilot and a moving attack drone. The repeater drones must continually reposition as the attack drone moves while also coping with losses caused by enemy fire.