The US Department of Defense is reportedly working with the startup Scale AI for testing artificial models intelligencefor military use.
Scale AI will create a framework of tools and data that the Pentagon can use to evaluate language models. The framework will be able to be used to "measure model performance, provide real-time feedback to warfighters, and create specialized public domain evaluation suites for testing AI models in military support applications," the San Francisco-based startup says.
The ability of language models to analyze and generate text could enhance the Pentagon's ability to gather intelligence, plan operations, and thus be able to guide decision making in the battlefield.
"Imagine a world where combatant commanders can see everything they need to make strategic decisions," said Craig Martell, the ministry's head of digital and artificial intelligence;
“Imagine a world where these combatant commanders don't get that information via email and so the turnaround time for situational awareness shrinks from a day or two in ten minutes", he stated during his Advantage DoD 2024: Defense Data and AI Symposium.
Η artificial intelligence can process large amounts of information quickly. Military data, however, is often highly sensitive, and officials worry that if they start using it, API abuse attacks could leak information.
The biggest obstacle to the military application of LLMs is their tendency to produce inaccurate or false information. With Scale AI's work, the Pentagon believes it can test the performance of different models to identify potential threats before using the information to support soldiers in combat.
The startup will reportedly compile “standby datasets” containing examples of effective responses to input prompts that will be useful to the military. DoD officials will then be able to compare the responses of different models to the same prompts and evaluate their usefulness.
Last year, the Ministry of Defense launched the Task Force Lima, μια μονάδα με επικεφαλής τον Martell, πρώην επικεφαλής της μηχανικής εκμάθησης στη ride-share biz Lyft, για τη διερεύνηση στρατιωτικών εφαρμογών για την τεχνητή νοημοσύνη.
“It is imperative for the Department of Defense to responsibly pursue the adoption of AI models, identifying appropriate safeguards and mitigating national security risks. better safetys that can arise from issues such as mismanagement of training data,” Martell said at the time. "We must also consider the extent to which our adversaries will use this technology to seek to disrupt our own use of AI-based solutions."
However, tools like ChatGPT are blocked internally. The US Space Force banned its personnel from using the software, lest military secrets be revealed or leaked.
