Palo Alto Networks Minemeld - Part I - An Introduction

Minemeld is an awesome tool provided by Palo Alto Networks that is available for multiple use cases, and comes in both open source and paid subscriptions (via AutoFocus). In my opinion, the open source version should only be used in test environments and not in production for the reasons below:

• The paid subscription is hosted and maintained as a SaaS service. Every time I have deployed the open source version, either myself or with customers, I have ran into issues with reliability, upgrades, and configuration.
• The paid subscription comes with access to the Palo Alto Networks Technical Assistance Center, which can be extremely useful when needing configuration or troubleshooting help.
• The paid subscription comes with AutoFocus. AutoFocus is a contexual threat intelligence service that allows security professionals to deeply analyze all data collected by WildFire. This service becomes even more powerful when used in conjunction with Minemeld, as WildFire data can be cross referenced with third-party data, WildFire data can be exported for use by third party security tools, etc. There are many examples, some of which I will be sharing in this series of posts.

The purpose of the first post in the series is to familiarize you with how Minemeld works using a very simple use case. This is a great place to start in order to understand how Minemeld actually works and see it in action. Subsequent posts will dive deeper into configuration, as well as other use cases and scenarios.

NOTE: The post(s) associated with this topic assume(s) that you already have an active Minemeld instance running (either open source or via AutoFocus), and that you already have a firewall or third party security tool configured in a manner that can consume the data. More information about initial configuration can be found here (for hosted) and here (for open source).

Important Minemeld Jargon

• Prototype - A prototype is what is used to create a Miner, Processor, or Output Node. There are many prototypes already present by default within Minemeld that can be used to create nodes. However, they can also be used to create new prototypes for specific use cases. In the example below, I used an existing prototype called ETOpen.blockIPs to create a new prototype called minemeldlocal.cisco_talos so that I can assign specific attributes, a source URL, etc. to be referenced in my configuration. 

• Miner Node - A miner is a resource created from a prototype that mines data from a specific source (i.e. Cisco Talos IP Blacklist in the screenshot above). Miners contain the specific attributes defined within the prototype, which can prove useful. An example of an attribute would be confidence level (i.e. the prototype above has a value of 80). Confidence value is an attribute that pertains to the operator's confidence of the validity of the data collected by the miner. For example, a feed that gets updated very often may be assigned a confidence level of 100, whereas a feed that gets updated once a month may have a confidence level of 25. By default, a miner with a value >75 is considered high, 50-75 is considered medium, and <50 is considered low. Below is a screenshot of the cisco_talos miner that was created from the minemeldlocal.cisco_talos prototype shown in the previous screenshot.  

• Processor Node - A processor is a resource created from a prototype that acts as a data aggregator for multiple miners that contain the same type of data (i.e. IP addresses). This function allows the data from multiple miners to be consolidated into a single source. In the screenshot below, I have created the aggregatorIPv4 processor node from the stdlib.aggregatorIPv4Inbound prototype to aggregate IPv4 addresses. 

• Output Node - An output node acts as a mechanism to distribute the data aggregated by a processor node so that it can be consumed by a security tool (i.e. a firewall). Output nodes can be customized based on attributes. For example, an output node can be configured to only distribute data that has a confidence value >75. It is also important to note that data can be distributed from an output node to an external destination (i.e. a firewall) via either a push (dynamic address group) or a pull (external dynamic list) method. I will provide more information on this in a subsequent post. In the screenshot below, I have created the feedHC output node from the stdlib.feedHCGreen prototype to distribute high confidence data. 

Bringing it All Together

The cisco_talos miner has collected 939 indicators, which based on the attributes we assigned to the various prototypes, are passed to the aggregatorIPv4 processor node, and subsequently the feedHC output node. This is permitted based on the attributes described above (i.e. confidence level value of 80).

My next post will provide a walk through of the configuration referenced in this post.