It’s late in the 4th quarter of Super Bowl LI. The Falcons are facing 3rd and goal on the Patriots’ 5 yard line. Matt Ryan takes the snap and hands off to Devonta Freeman, already running hard at the goal line. Then, with a crunch audible to the topmost rows of NRG Stadium, Freeman is brought down by Dont’a Hightower right at the goal line.
Silence falls as all eyes turn… not to the referees on the sidelines (there aren’t any) but to giant LCD panels behind the end zones. The screens remain black for several long moments until “TOUCHDOWN” lights up. A roar erupts from about half of the stadium.
Where were the referees in this fictional account of the upcoming Super Bowl LI? They’ve been automated by artificial intelligence systems hooked up to networks of sensors worn by the players and high-speed cameras strategically positioned throughout the stadium.
Does all of this sound speculative? It is, but not as much as you might think when you take a look around the world of professional sports. “Precursor” technologies that provide the sensory input data for yet-to-be-invented AI algorithms are already in use. The distance between today’s smart sensors and tomorrow’s fully automated officiating is closing fast.
We won’t see AI refs in Super Bowl LI, but the question of whether there’s room for automated officials in the NFL and other professional sports is quickly becoming a matter of policy, not technology.
Automated officiating is born one technology at a time
Ask any soccer fan about the Hand of God goal, and you’ll hear not a lecture on divine intervention, but a passionate recounting of one of the biggest referee mistakes in sports history. During the 1986 World Cup quarterfinals, Argentinian midfielder Diego Maradona scored a goal by swatting the ball into the net by hand, a clear no-no in soccer. The referees all missed it, and the goal stood despite the handball, which was described by Bleacher Report as “one of the most egregious” mistakes in World Cup history. Because of this error, Argentina ended up winning 2-1 over England.
Refs are only human, and we can’t expect them to be right all of the time. Some calls, such as the Hand of God, occur simply because refs can’t watch every single detail of every single play. Mistakes are inevitable. But the days of bad calls may soon be at an end. Goal-line technology, wearables, nanosensors, and even artificial intelligence are being adopted to help reduce referee errors and level the playing fields.
Do referees have the toughest jobs in sports?
The Hand of God goal wasn’t the first referee mistake and it certainly won’t be the last. Questionable calls have landed refs in the crosshairs of irate players and fans since humans first began competing over athletic prowess. The only difference between then and now is that bad calls are recorded and preserved online forever. Everything from simple oversights to politically charged decisions can be witnessed and dissected by every sports fan with a smartphone and a data plan.
Consider the 1972 Olympics, when officials nearly reignited the Cold War by handing the gold to the Soviet Union’s men’s basketball team. The U.S. team had remained undefeated throughout the Olympics and as time wound down on the gold medal match, looked like they would remain so. But officials added time to the clock during the last seconds of the matchup, a decision ESPN dubs one of the worst calls in sports history. The Soviets eked out a one-point lead in those final moments, unseating the American champions.
Although the world narrowly avoided political disaster then, referees have continued to confound players and enrage fans with mystifying calls. But what if officials—at the next World Cup, Olympics, or Super Bowl—had access to real-time data on players’ movements and could make faster, more accurate decisions? By reducing human error, there would be fewer opportunities for bad judgment and confusion to alter the outcomes. This would benefit not only the players, but the refs as well. That’s a good thing, right? Well, it’s not that simple.
The era of goal-line technology
Referees are just as much a part of the game as players and coaches, facing their own unique brand of pressure every time they set foot on a court or field. The decisions they make determine the course of high-stakes matchups, and coaches, players, and spectators alike don’t hesitate to let them know when they disagree with their calls. That’s where advances in tracking technology prove useful.
In football or soccer, for instance, goal-line technology could become referees’ first line of defense against disgruntled players and fans who dispute their calls. Goal-line technology determines the exact moment when a ball crosses the threshold to count as a score. These systems use high-speed camera or microchips implanted in the balls to track their movements, and referees can use the real-time data from the systems to confirm and support their judgments.
The NFL has yet to fully adopt goal-line technology, but FIFA implemented it for the 2014 World Cup. European Premier League teams have begun using it as well. England’s Premier League coach spoke in favor of adopting the system, saying it would prevent “gross injustices” from occurring in the sport. Referees can access goal data within a second, mitigating lengthy game delays. Rather than relying on their own observations or debating their colleagues, officials will have instant proof of whether a goal should stand.
Such technology would help referees in other sports as well, allowing them to defend their calls when they’re accused of bias. Case in point: Game 2 of the 1998 NBA finals, when observers noted that many of the refs’ calls favored the Boston Celtics over the Los Angeles Lakers. Emotions flare when championships are on the line, and hard data can help cooler heads prevail.
Wearables and AI technology are game-changing…literally
Bad calls make great headlines, but the occasional bad call doesn’t always indicate a trend. When the NBA began reviewing officials’ calls in 2015, it found that referees were correct 86% of the time in the crucial final minutes of games. Rod Thorn, then-head of the NBA’s basketball operations, said the referee reviews not only increased transparency, but added “a humanity factor” and proved that “the vast majority of the calls are right.”
See also: What happened when the NFL found IoT?
A little humanity can go a long way, especially in the fraught world of sports officiating. Referees often find themselves the targets of insults and vitriol, particularly from fans. There are even online forums where fans go to brainstorm creative barbs to unleash on officials during games.
Technology is giving refs better cover. Multiple professional leagues are increasingly equipping their athletes with wearable devices that send constant updates to coaches and referees who use the data to assess their performances, spot signs of injuries, and analyze goals and plays.
In high-profile events, such as the 2016 Rio Olympics, data was often streamed to television broadcasters so spectators could understand what was happening in real-time. Former Olympian Barbara Kendall has said that live stats make sporting events more interactive for fans watching at home.
But wearables are also invaluable for accurate judging and scoring. In fencing, for instance, sensors indicate the precise timing and location of athletes’ hits. Camera feeds record exactly what referees see in several sports, eliminating controversies around their decisions. It’s hard to dispute officials’ logic when you’re looking at the same numbers and visuals and can see why they ruled the way they did.
Cameras may also provide a buffer for human errors, which is a real concern for referees. In one memorable instance, officials accidentally allowed the University of Colorado Boulder football team five downs, giving them a game-defining advantage over the University of Missouri. The error occurred in 1990, but it was so widely known, it has its own Wikipedia page. Widespread use of on-field cameras could help referees catch those oversights, as would automated tracking of downs, yards gained and lost, and other critical aspects of the game.
Artificial intelligence is also already here, helping coaches call plays during games. “Moore’s Law predicts that computational power doubles roughly every two years, so by Super Bowl 100, in 2066, computers should be several million times faster than today. Imagine a robot Bill Belichick flicking through a digital playbook of trillions of moves during the 40-second gap between plays.”
Immersion and empathy through technology
Wearables and smart technology are transforming sports like football, soccer, and basketball in unprecedented ways. Athletes and coaches are gaining deeper understandings than ever before about plays, strategies, and players’ abilities. If coaches can monitor athletes’ vitals and performances, they can detect when someone is injured or at risk of hurting themselves. Players can then seek care in time to prolong their careers instead of being devastated by an unforeseen break or trauma.
Fans are getting in on the action, too. Jerseys equipped with smart sensors give them a feel for what it’s like to be on the field with their favorite players. This smart apparel uses haptic feedback to transmit football plays to viewers as they’re watching them happen.
Even those who don’t slip into a smart jersey can get up close and personal through wearables. Devices such as Ref Cams and GoPros allow viewers to feel as though they’re on the court or field. The WNBA introduced Ref Cams in 2013 and FoxSports installed GoPros in referees’ hats to innovate in its coverage of the December 2016 Big 10 Championship Game.
Although refs and players don’t always see eye to eye, they’re likely to agree that facial recognition technology will change sporting environments for the better. Stadiums in Australia are exploring the use of such systems in keeping known troublemakers out of their venues. People who are known to start fights and antagonize refs and players may soon be contained before they’ve had a chance to rile people up and become distractions for officials, players, and fans who are trying to enjoy their games.
Wearables and other smart technologies will help officials improve their accuracy and do their jobs more effectively. But the greatest benefit may be that when fans and coaches get a ref’s-eye view of the game, they become a little less hostile and a lot more empathetic.
Will AI replace refs?
We’ve been looking at how new technologies are having an impact in professional sports around the world. These changes are, for the most part, evolutionary: players, coaches, and officials benefit but the games remain largely the same. As long as the chips, dips, and buffalo wings don’t run out, Super Bowl LI won’t feel much different.
When we start talking about artificial intelligence in sports, we enter into a completely new realm. Because when you link the sensors and cameras we’ve been talking about to AI systems, you have the recipe for fully automated officiating.
As we saw with soccer, on-field tracking systems can detect handballs, identify penalties, and evaluate offside calls. These are working systems already in use; the capabilities of autonomous AI systems will only grow from here. Which is why, in one research study, referees and umpires face a 98% chance of being replaced by AI.
Proponents of automated officiating say that AI could reduce corruption and more accurately enforce rules, and it seems likely that the technology will play an increasingly prominent role in athletics. But the transition – if it happens – won’t occur overnight.
Although referees are often maligned by angry sports fans, people see officiating a game as a complex task that requires human capabilities. Increased uses of sensors and cameras might provide an AI system with the data to make certain calls, but they may need something closer to a theory of mind before they seem human enough to be embraced by sports teams and fans. As we see with technologies like self-driving cars, the technology is here; it’s the policy and regulation that needs to catch up.
So when you’re watching the Super Bowl this weekend, keep an eye on the referees. Can you picture the game without the zebra jerseys on the sidelines? Are we ready to welcome automated play calling to the wide world of sports?
Additional article contributors include Mehdi Ghafourifar and Brian Walker.