Velocity by Booz Allen

While it will never replace classified intelligence collection and analysis, OSINT is the “INT” that best balances the traditional need for secrecy with the increased need for rapid information sharing to address developing and emergent threats.

Apart from building advanced analytic engines to automate the exploitation, this data challenge requires analysts to determine what data to prioritize and target, based on the mission problem, in order to limit the volume required for aggregation. Accomplishing this can provide analysts with previously unavailable insights, including patterns of life; messaging trends; social, financial, and supply chain networks; breaking news updates; and more. Let’s take a closer look at the range of sources available and which need to be curated to paint a complete picture. While OSINT aggregation, processing, and analysis is not free, many of the sources that form the basis for OSINT insights are. Free and open, publicly available information can range extensively in terms of data type and content. Though by no means an exhaustive list, some examples include: • news articles • public social media posts • marine or air traffic monitoring sites

I n light of today’s national security threats, the need for the U.S. intelligence community (IC) to swiftly process and disseminate information has never been more urgent. Dynamic and multifaceted challenges demand unprecedented speed and agility, and this imperative has been further magnified by the exponential rise of publicly available information. Open-source intelligence (OSINT), as defined by the SANS Institute, is “intelligence produced by collecting, evaluating, and analyzing publicly available information with the purpose of answering a specific intelligence question.” Beyond mere information, OSINT can contextualize, enhance, and validate analysis and provide opportunities for increased dissemination. While it will never replace classified intelligence collection and analysis, OSINT is the “INT” that best balances the traditional need for secrecy with the increased need for rapid information sharing to address developing and emergent threats. In a panel about the future IC workforce, Patrice Tibbs, deputy chief of the Open Source Enterprise at the Central Intelligence Agency, called OSINT the “INT of first resort” and said that OSINT brings a new perspective not only to the information being collected but also to the entire process of intelligence collection. But OSINT sources are far from static and have a scope that continues to expand. What was once limited to historical research or news monitoring now encompasses a rich array of technology and collection methods. Even technical domains traditionally associated with geospatial intelligence or signals intelligence can now be integrated and analyzed alongside more traditional OSINT sources, giving rise to more robust and holistic intelligence. Director of National Intelligence Avril Haines affirms that various intelligence disciplines and IC agencies are all diligently developing their versions of open-source capabilities. “However, we are not in a position where we feel as if the entire intelligence community is leveraging … the best of what we can do in this space yet, and that is something that we have been focused on,” she said, in a hearing to the Senate Committee on Armed Services.

With its advantages, OSINT also disrupts the intelligence playing field. According to former Defense Intelligence Agency Director Robert Ashley and former Principal Executive of the Office of the Director of National Intelligence (ODNI) Neil Wiley, “the ubiquity and accessibility of this public data” narrows the advantage of the IC’s niche and proprietary intelligence sources and methods, which emphasizes the need to continuously evolve and adapt how publicly available information is integrated with classified resources. National security organizations must evolve their understanding of OSINT to keep pace with emerging adversarial capabilities in big data aggregation, cloud computing, AI, and machine learning (ML) analytics. Data, Data Everywhere Big data stands as the formidable linchpin within the OSINT domain, shaping the very frameworks of collection and processing methodologies. According to Statista, in this year alone, 120 zettabytes of data will be created, transformed, captured, copied, and consumed and that number will grow by 20–30 zettabytes annually thereafter. To put that into perspective, one zettabyte is equal to one trillion gigabytes, or over 570 million years of YouTube videos. Even if the IC is only expected to ingest and analyze a fraction of that, it’s a nearly inconceivable task.

• international trade databases • company registration databases • government public records • nongovernmental organization (NGO) reports • civil imagery • human geography and infrastructure data

license agreement that affords a continuing right of access.” The report further notes that “there is today a large and growing amount of CAI that is available to the public, including foreign governments (and their intelligence services) and private-sector entities, as well as the IC. CAI clearly provides intelligence value, whether considered in isolation and/or in combination with other information, and whether reviewed by humans and/or by machines.” As the IC increases investment in commercially available information in alignment with the nation’s strategic imperatives, it will need to continue developing new OSINT policies and frameworks. These guidelines can help ensure the IC handles and collects information properly, maintains vigilance against potential counterintelligence threats, and allocates resources comprehensively to support the development of tools, training, and tradecraft.

It’s a matter of knowing that information exists, then collecting, vetting, and processing it to address intelligence requirements. However, an additional level of vetting is often required for free sources of publicly available information to ensure the data can be validated and avoid leveraging mis- or disinformation. Extracting value from freely accessible information sources is often a race against the clock. For example, in 2023, Twitter stopped allowing unregistered viewers to see individual tweets and limited the number of tweets non-paying users could view per day. Many aggregators (though not all) were locked out and no longer able to leverage the Twitter application programming interface (API). Similarly, the United Nations’ Comtrade database recently updated its subscription plan and now only allows limited access and no downloads on their free public user license. There is also a subset of publicly available information that is commercially sold, which includes geolocation; commercial satellite and airborne imagery; radio frequency data; subscription news and journal articles; and databases of pre-aggregated public records, among many others. An ODNI report from 2022 describes that the purchase of commercially available information (CAI) can be made one time or on a subscription basis and may involve a purchaser “directly ingesting the CAI or obtaining a

A zettabyte is 1,000,000,000,000,000,000,000 bytes (or more than 570 million years of YouTube videos)

Deciphering and Accelerating the Value of OSINT Modern technologies are turning the tide and

empowering national security organizations to harness the immense potential of public and commercially available data and analytic sources. Advanced modeling and automation capabilities, decentralized data processing, and AI/ML are alleviating the tasks that were once arduously executed through hours of manual research, exploitation, integration, and analysis by the IC.