In a world where data volumes are growing by the second, open-source information from places like news sites, social media and other online sources is becoming an increasingly consequential dataset to analyze. But just because open-source information exists, this by itself doesn't make it intelligence.

Open-source intelligence (OSINT) is information that has been analyzed, identified, deliberated, discovered, disseminated and/or distilled into an output. Said another way: Turning data into something that is actionable is what differentiates information from intelligence – and this is why OSINT is so incredibly important at informing the larger intelligence picture. It serves as the foundation, and then the appropriate classified sources of information fill in the remaining gaps and verify what is needed, amplify insights and enhance traditional intelligence collection.

Ingesting an Ever-Growing Data Pipeline

A major focus in the OSINT field has been ingesting data and making it usable for various AI/ML-based intelligence tools. This makes logical sense: in a target-rich environment, the faster we can accelerate the pace at which we collect open-source information and automate organizing and processing the data using AI/ML tools, the faster analysts can verify the results and begin analysis and production. This includes developing models that will look for patterns and indications that may be the first warning of an event or a trend that a human analyst should investigate.

Many commercially available tools aggregate and normalize data, and allow a user to search and drill down on specifics or even create visualizations and alerts. Social media data, which is one type of open-source information, has a richness that presents both opportunity and challenge. The opportunity is the value in the plethora of data created – photos, videos, locations, relationships and more. The challenge is in processing what can be high volumes of data to produce valuable and/or timely intelligence. This points to the importance of tools that help with collection/ingest, processing, analysis, visualization, and reporting.

Understanding OSINT Tradecraft

There are absolutely risks to consider when assessing open-source information and looking for true intelligence. Adversaries are operating in the same spaces, and they can see what our intelligence teams are searching for through a variety of “tells”. Certain tradecraft is required to mine open-source information safely and securely. Tools like VPNs, Managed Attribution or Non-Attributable Systems and other layered security measures which provide a separation between the device and government organization are valuable, because they help to stay vigilant against adversary observance. There are also certain techniques associated with open-source information collection that can throw off adversaries and are important skills for analysts working in the OSINT field.

Added challenges arise given the amount of potential circular reporting, and the history and risk of misinformation (e.g., deception, propaganda, disinformation, etc.). Though their capabilities are rapidly improving, AI tools are not adept yet at filtering all misinformation. Conversely, human analysts can often spot this type of information and filter it out well before it finds its way into any type of finished intelligence.

Adhering to Regulations

Of course, there are also laws around data protection and privacy that OSINT analysts must abide by, which our adversaries are not concerned with adhering to. In the United States, we have the Constitution and the Fourth Amendment (which protects against unreasonable search and seizure) and a host of other federal and state data protection laws. This all needs to be considered and factored into open-source information collection so that any collection is done appropriately, legally and safely.

Both the volume and the dimension of open-source information is exploding. This presents opportunities for the intelligence community, as well as challenges for the analysts working with it. However, the impact of turning open-source information into true intelligence cannot be understated.

It can complement existing intelligence collection efforts across SIGINT, GEOINT, HUMINT and more, and results in faster, better intelligence products and interactions. For these reasons, the Department of Defense has developed an entire OSINT Strategy and has called OSINT “the premier source of intelligence information for decisionmakers and warfighters.”

We agree and work every day to help customers navigate the continuum from open-source information to true valuable, actionable OSINT. The tools, the raw data and breadth and depth of potential is immense, and the pace at which they will continue to grow is accelerating constantly.