Bug-Hunting Computers to
Compete in DARPA Cyber Grand Challenge
By Cheryl Pellerin, DoD News,
Defense Media Activity.
Washington D.C. — (DOD
News) — July 18, 2016 — On Aug. 4 in Las Vegas, seven computers will compete
in the first all-machine cyber defense tournament, the result of a multiyear
effort by the Defense Advanced Research Projects Agency to bring autonomy to the
problem of making
Today, finding and reacting to unknown flaws in software is entirely manual
Mike Walker, program manager for DARPA’s Cyber Grand
Challenge, discussed what the agency and the seven finalist teams set out to do
and what the world will see during the final hours of a competition that
eventually could deliver cybersecurity at network speeds by computers that look
at all the bits, all the time, without human help.
Today, finding and reacting to unknown flaws in software is
entirely manual, as demonstrated by these assessment team members who are
collecting data to analyze blue and red team attacks and defenses during
exercise Cyber Shield 2016, at Camp Atterbury, Ind., April 28, 2016. On Aug. 4,
2016, the Defense Advanced Research Projects Agency will hold a Cyber Grand
Challenge for seven finalist teams whose computers will compete autonomously in
a capture the flag competition that eventually could make machine-speed
cybersecurity around the world a reality. Army photo by Sgt. Stephanie A.
“Today the comprehension [of] and reaction to unknown flaws
in software is entirely manual,” Walker said during a recent media
The best statistics indicate that when intruders have access
to an unknown flaw and are using it to break into computers, on average they can
use the flaw for 312 days before it's discovered, and software vendors have
about 24 median days to patch, he said. Both of those times are coming down, he
added, but the amount of time it takes to discover, comprehend and react to an
unknown flaw is about a year.
“We want to build autonomous systems that can arrive at their
own insights about unknown flaws, do their own analysis, make their own
risk-equity decisions about when to field a patch and how to manage that
patching process autonomously,” Walker said, “and bring that entire … timeline
down from a year to minutes or seconds.”
DARPA launched the challenge in 2013 and has so far spent $55
million on the effort. In October that year, it opened up a track for teams who
wanted to submit a proposal and receive initial funding to compete, and an open
track for anyone in the world who wanted to enter their own intellectual
property without DARPA funding.
Walker said development and work on the challenge began in
June 2014, and the qualifier stage for those who entered the competition ran
until June 2015.
“At the end of the qualifier stage, we held a contest that
was executed live on the internet for 24 hours,” he said, “where we gave 131
pieces of unexamined software to all competitors simultaneously and asked the
machines to bug hunt those pieces of software in 24 hours and submit bug reports
directly to DARPA.”
The results of the contest showed that, of the 590 known
flaws in the publicly available software corpus, the machines mitigated 100
percent of them, Walker said, noting that no individual competitor achieved that
result or even came close. Only by taking the best solution from each competitor
in the field could it be achieved, he said, and all the teams learned from one
another. Individually though, the machines successfully bug-hunted 73 percent of
the challenges, he added, finding and proving at least one security-critical
flaw in the software.
“We don't require systems to write exploits, but they do have
to prove vulnerability and gain very specific control of software and indicate
that to a DARPA referee,” Walker said, adding that the goal is to create
defenses that can prevent vulnerability from happening.
In Las Vegas, Walker said, he’ll be most excited to see the
mix the machines decide to use of generic binary armoring, which doesn't target
specific bugs and is all over the program, slowing it down, and point patching,
which very quickly fixes specific bugs but requires a lot of expertise.
“I will say that in all the results all of our machines
released in 2015 as the result of our qualifiers, we did see point patching --
very effective point patching written by an expert system,” Walker said, “and
that was actually one of the reverse engineering tests that was most convincing”
when he and his team were thinking about executing the second year of the Cyber
Stand and Compete
When the seven finalist teams meet in Las Vegas next month,
the field of battle will be the Paris Hotel and Conference Center. The teams
will compete in a cyber capture-the-flag event for nearly $4 million in prizes.
The machines themselves are DARPA-constructed
high-performance computers with about 1,000 Intel Xeon cores and 16 terabytes of
RAM. They’ll operate on an open-source operating system extension called DECREE
-- for DARPA Experimental Cybersecurity Research Evaluation Environment -- built
only for computer security research and experimentation.
What each team will do with its autonomous system, Walker
said, “is program it with what we call a cyber reasoning system that they will
eventually be disconnected from on the day before the grand challenge. And when
they are disconnected from it, that cyber reasoning system will stand and
compete entirely on its own, and they will be spectators to its victory or its
The results will be open-source to the world as they happen,
and every single piece of software the machines have written and will write will
go on a public server in perpetuity, DARPA officials said.
Walker said one thing that's important to understand about
the final event is that the compute time during which the event will happen and
the audience time are different timescales.
These racks will compete in the DARPA’s Cyber Grand Challenge finals in Las Vegas, Aug.
On Aug. 4, the machines will compute the event for 10 hours
without an audience, then at 5 p.m., Walker and his team will do a three-hour
recap for the audience. But the live event and the rest of the computing will
finish at the same time. “So the beginning will be a recap, but the end will be
live, and that's because a three-hour timescale for a live event was much more
manageable,” he explained.
When the live event begins at 5 p.m., the audience in the
3,000-seat auditorium will watch a capture-the-flag competition among seven
autonomous machines occur in rounds of about five minutes each, Walker said.
“We have a video we call an arena view that shows who's proving vulnerability
against who, whose software is broken, whose software is well defended, and it's
going to unfold as a graphical 3-D visualization, all driven by data occurring
inside the game on screen,” he said.
Two announcers -- one astrophysicist and one hacker -- will
talk the audience through the action.
“Then we have a second view called trace viewer that you can
think of as a software microscope that is actually going to let people see what
the structure of a good patch looks like, what the structure of a failed patch
looks like, and what the structural feel of the software armor that these
systems are constructing looks like,” he said. “You can see multiple samples
from a single system and start to identify the visual field.”
The awards ceremony will take place the next day at 10 a.m.
A Seat at the Table
The Cyber Grand Challenge is co-located this year with the
world series of hacking: Def Con, one of the world’s largest hacker conventions.
The day after DARPA’s event, Walker said, the autonomous
system that wins the Cyber Grand Challenge has been challenged to play in a Def
Con community capture-the-flag contest, a competition with at least two decades
“You win a qualifying competition, where [that] has to be
global entry open competition, and the winners of other competitions feed into
Def Con capture the flag and earn a seat there,” Walker explained. “Teams fly in
from around the world to play. It's an annual contest, and this will be the
first time that a machine will play at a table rather than a team of experts.
“That contest is actually post-DARPA's involvement with the
technology,” he added, “and could actually be considered the first step in the
open technology revolution.”
(Follow Cheryl Pellerin on Twitter @PellerinDoDNews)